xls-r-300m-bemgen-combined-hp-tuning-test-model

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the BEMGEN - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.3925
• Wer: 0.4224

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005819322340474784
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 128
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	2.7986	100	2.9112	1.0
No log	5.5704	200	1.6968	0.9971
No log	8.3422	300	0.3893	0.6158
No log	11.1141	400	0.3453	0.5107
31.2339	13.9127	500	0.3384	0.4909
31.2339	16.6845	600	0.3592	0.4619
31.2339	19.4563	700	0.3676	0.4336
31.2339	22.2282	800	0.3925	0.4228

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0