mms-1b-bemgen-balanced-model

This model is a fine-tuned version of facebook/mms-1b-all on the BEMGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.2753
• Wer: 0.4201

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.8327	0.1031	100	0.8480	0.7889
0.5906	0.2062	200	0.3704	0.5819
0.4809	0.3093	300	0.3327	0.5039
0.4495	0.4124	400	0.3172	0.4894
0.4266	0.5155	500	0.3102	0.4632
0.4167	0.6186	600	0.3075	0.4716
0.4151	0.7216	700	0.2996	0.4829
0.3955	0.8247	800	0.2985	0.4712
0.3802	0.9278	900	0.2960	0.4926
0.392	1.0309	1000	0.2839	0.4374
0.375	1.1340	1100	0.2837	0.4318
0.3885	1.2371	1200	0.2812	0.4257
0.3824	1.3402	1300	0.2825	0.4255
0.3906	1.4433	1400	0.2794	0.4290
0.3465	1.5464	1500	0.2807	0.4283
0.3564	1.6495	1600	0.2773	0.4238
0.3617	1.7526	1700	0.2750	0.4452
0.3808	1.8557	1800	0.2783	0.4229
0.3661	1.9588	1900	0.2761	0.4517
0.3952	2.0619	2000	0.2753	0.4201

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0