SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model trained on the Petitepoupoune/Cyberattacks_aviation dataset that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
Classification head: a LogisticRegression instance
Maximum Sequence Length: 512 tokens
Number of Classes: 10 classes
Training Dataset: Petitepoupoune/Cyberattacks_aviation

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
0	'The radar display suddenly shows multiple ghost aircraft.' 'Engine parameters display fluctuates, but engine runs fine.' 'Pilot receives incorrect weather data from ground station.'
1	'Navigation coordinates keep shifting without any inputs.' 'Navigation system reports inconsistent coordinates.'
2	'Unable to establish secure communication with the ground.' 'Pilot headset communication filled with static noises.' 'GPS fails to lock onto satellites during flight.'
3	'Unexpected engine alert appeared without apparent malfunction.' 'Air Traffic Control reports conflicting position data.' 'Ground proximity warnings trigger in normal flight conditions.'
4	'Passenger internet shows anomalies, potentially exposing data.' 'Unusual network activity detected in cockpit systems.' 'Passengers report unauthorized access to personal devices.'
5	'Pilots unable to update flight plan due to system freeze.' 'Cabin displays turn off intermittently without reason.' 'Unusual delay in system response when adjusting controls.'
6	'Incorrect altitude data reported by onboard instruments.' 'Cockpit alarm indicates incorrect fuel levels.' 'Unexpected power fluctuation in avionics systems.'
7	'Aircraft is directed off course by autopilot without input.' 'Sudden and unexplained decrease in engine thrust.' 'Aircraft enters unexpected descent despite normal controls.'
8	'In-flight entertainment malfunctions and reboots frequently.' 'Unexpected system update initiated during flight.' 'Sudden reboot of all electronic systems mid-flight.'
9	'Minor turbulence encountered during flight.' 'Pilot reports fatigue after long flight hours.' 'Passenger complains about seatbelt malfunction.'

Evaluation

Metrics

Label	Accuracy
all	0.6667

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("Petitepoupoune/SetFit_Cyberaviation")
# Run inference
preds = model("Radar detects a non-existent aircraft nearby.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	5	6.7857	10

Label	Training Sample Count
0	4
1	2
2	6
3	3
4	4
5	6
6	3
7	4
8	4
9	6

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (1, 1)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 20
body_learning_rate: (2e-05, 2e-05)
head_learning_rate: 2e-05
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
l2_weight: 0.01
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0095	1	0.2581	-
0.4762	50	0.1219	-
0.9524	100	0.0351	-

Framework Versions

Python: 3.10.12
SetFit: 1.1.0
Sentence Transformers: 3.2.1
Transformers: 4.42.2
PyTorch: 2.5.1+cu121
Datasets: 3.2.0
Tokenizers: 0.19.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

Petitepoupoune
/

SetFit_Cyberaviation