Hugging Face's logo

mtzig
/
v3d_mistral_lora

PEFT
Safetensors
Generated from Trainer
Model card Files Community
v3d_mistral_lora / README.md
mtzig's picture
mtzig
Model save
cabe0ea verified
preview code
|
raw
history blame contribute delete
3.52 kB
metadata
library_name: peft
base_model: peiyi9979/math-shepherd-mistral-7b-prm
tags:
  - generated_from_trainer
metrics:
  - accuracy
  - precision
  - recall
  - f1
model-index:
  - name: v3d_mistral_lora
    results: []

v3d_mistral_lora

This model is a fine-tuned version of peiyi9979/math-shepherd-mistral-7b-prm on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3848
  • Accuracy: 0.8329
  • Precision: 0.7398
  • Recall: 0.4527
  • F1: 0.5617

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 765837
  • distributed_type: multi-GPU
  • num_devices: 4
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 64
  • total_eval_batch_size: 32
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss Accuracy Precision Recall F1
No log 0 0 0.6112 0.7506 0.3659 0.0746 0.1240
0.5265 0.0532 20 0.6081 0.7647 0.5455 0.0299 0.0566
0.4127 0.1064 40 0.5412 0.7682 0.7 0.0348 0.0664
0.3347 0.1596 60 0.5020 0.7741 0.7368 0.0697 0.1273
0.3077 0.2128 80 0.4463 0.7965 0.6944 0.2488 0.3663
0.307 0.2660 100 0.4498 0.8 0.6782 0.2935 0.4097
0.2413 0.3191 120 0.4316 0.8141 0.7087 0.3632 0.4803
0.326 0.3723 140 0.4107 0.8235 0.7257 0.4080 0.5223
0.2404 0.4255 160 0.4615 0.8094 0.7671 0.2786 0.4088
0.2962 0.4787 180 0.4205 0.8282 0.7619 0.3980 0.5229
0.2727 0.5319 200 0.4830 0.8 0.7627 0.2239 0.3462
0.2844 0.5851 220 0.4187 0.8259 0.7524 0.3930 0.5163
0.2632 0.6383 240 0.4037 0.8235 0.7339 0.3980 0.5161
0.2499 0.6915 260 0.3885 0.8247 0.7241 0.4179 0.5300
0.2121 0.7447 280 0.3953 0.8224 0.7232 0.4030 0.5176
0.2704 0.7979 300 0.3849 0.8329 0.736 0.4577 0.5644
0.2333 0.8511 320 0.3878 0.8318 0.7417 0.4428 0.5545
0.2896 0.9043 340 0.3886 0.8306 0.7395 0.4378 0.55
0.2398 0.9574 360 0.3848 0.8329 0.7398 0.4527 0.5617

Framework versions

  • PEFT 0.13.2
  • Transformers 4.46.0
  • Pytorch 2.5.1+cu124
  • Datasets 3.1.0
  • Tokenizers 0.20.3