output_LiLT_test_03

This model is a fine-tuned version of nielsr/lilt-xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1452
• Precision: 0.7549
• Recall: 0.8276
• F1: 0.7896
• Accuracy: 0.9577

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 300
• num_epochs: 45

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	0.0808	100	0.5589	0.0259	0.0020	0.0037	0.8724
No log	0.1617	200	0.3198	0.3704	0.3698	0.3701	0.9009
No log	0.2425	300	0.2222	0.6005	0.6743	0.6353	0.9346
No log	0.3234	400	0.1950	0.7231	0.6316	0.6743	0.9446
0.4262	0.4042	500	0.1684	0.7115	0.7215	0.7165	0.9491
0.4262	0.4850	600	0.1885	0.6328	0.7602	0.6907	0.9366
0.4262	0.5659	700	0.1776	0.6609	0.8097	0.7278	0.9437
0.4262	0.6467	800	0.2086	0.6420	0.8034	0.7137	0.9398
0.4262	0.7276	900	0.1702	0.8241	0.7469	0.7836	0.9611
0.1027	0.8084	1000	0.1496	0.8028	0.7558	0.7786	0.9601
0.1027	0.8892	1100	0.1620	0.7354	0.7754	0.7549	0.9513
0.1027	0.9701	1200	0.1404	0.8062	0.7997	0.8029	0.9627
0.1027	1.0509	1300	0.1626	0.7976	0.7564	0.7764	0.9597
0.1027	1.1318	1400	0.1741	0.7253	0.8089	0.7648	0.9485
0.0764	1.2126	1500	0.1278	0.8170	0.7798	0.7979	0.9645
0.0764	1.2935	1600	0.1452	0.7549	0.8276	0.7896	0.9577

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.1+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1