indictrans-en-ne-checkpoint-1B-2

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1300
• Bleu: 37.7435
• Chrf: 61.2886
• Num Input Tokens Seen: 196608000

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 256
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: inverse_sqrt
• lr_scheduler_warmup_steps: 500
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Chrf	Input Tokens Seen
0.1232	0.2897	1000	0.1388	36.0713	60.2577	65536000
0.1306	0.5795	2000	0.1324	37.0443	60.8360	131072000
0.1309	0.8692	3000	0.1300	37.7435	61.2886	196608000

Framework versions

• Transformers 4.44.2
• Pytorch 2.2.1+cu121
• Datasets 3.0.0
• Tokenizers 0.19.1