results

This model is a fine-tuned version of microsoft/Phi-3-mini-4k-instruct on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.2406

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
2.2518	0.0530	100	1.3453
1.3137	0.1059	200	1.2820
1.2681	0.1589	300	1.2684
1.2611	0.2118	400	1.2625
1.2599	0.2648	500	1.2587
1.2709	0.3177	600	1.2561
1.2607	0.3707	700	1.2537
1.2502	0.4236	800	1.2515
1.2475	0.4766	900	1.2494
1.2479	0.5295	1000	1.2476
1.2535	0.5825	1100	1.2469
1.2546	0.6354	1200	1.2455
1.2498	0.6884	1300	1.2440
1.2445	0.7413	1400	1.2433
1.247	0.7943	1500	1.2423
1.2438	0.8472	1600	1.2418
1.2434	0.9002	1700	1.2413
1.2425	0.9531	1800	1.2406

Framework versions

• PEFT 0.12.0
• Transformers 4.42.4
• Pytorch 2.3.1+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1