mDeBERTa-v3-base-xnli-multilingual-zeroshot-v3.0-only-non-nli

This model is a fine-tuned version of MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2718
  • F1 Macro: 0.9088
  • F1 Micro: 0.9089
  • Accuracy Balanced: 0.9089
  • Accuracy: 0.9089
  • Precision Macro: 0.9092
  • Recall Macro: 0.9089
  • Precision Micro: 0.9089
  • Recall Micro: 0.9089

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 128
  • seed: 20241201
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.06
  • num_epochs: 3

Training results

Training Loss Epoch Step Validation Loss F1 Macro F1 Micro Accuracy Balanced Accuracy Precision Macro Recall Macro Precision Micro Recall Micro
0.2798 1.69 200 0.3328 0.8677 0.8677 0.8681 0.8677 0.8678 0.8681 0.8677 0.8677

Eval results

Datasets asadfgglie/nli-zh-tw-all/test asadfgglie/BanBan_2024-10-17-facial_expressions-nli/test eval_dataset test_dataset
eval_loss 0.667 0.294 0.381 0.272
eval_f1_macro 0.711 0.901 0.868 0.909
eval_f1_micro 0.713 0.901 0.868 0.909
eval_accuracy_balanced 0.71 0.901 0.867 0.909
eval_accuracy 0.713 0.901 0.868 0.909
eval_precision_macro 0.711 0.901 0.868 0.909
eval_recall_macro 0.71 0.901 0.867 0.909
eval_precision_micro 0.713 0.901 0.868 0.909
eval_recall_micro 0.713 0.901 0.868 0.909
eval_runtime 568.387 4.571 0.829 3.382
eval_samples_per_second 14.955 206.945 227.909 223.805
eval_steps_per_second 0.118 1.75 2.412 1.774
epoch 2.99 2.99 2.99 2.99
Size of dataset 8500 946 189 757

Framework versions

  • Transformers 4.33.3
  • Pytorch 2.5.1+cu121
  • Datasets 2.14.7
  • Tokenizers 0.13.3
Downloads last month
17
Inference Examples
Unable to determine this model's library. Check the docs .

Model tree for asadfgglie/mDeBERTa-v3-base-xnli-multilingual-zeroshot-v3.0-only-non-nli

Dataset used to train asadfgglie/mDeBERTa-v3-base-xnli-multilingual-zeroshot-v3.0-only-non-nli