roberta-large-ner-qlorafinetune-runs-colab

This model is a fine-tuned version of FacebookAI/xlm-roberta-large on the biobert_json dataset. It achieves the following results on the evaluation set:

• Loss: 0.0742
• Precision: 0.9394
• Recall: 0.9563
• F1: 0.9478
• Accuracy: 0.9810

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0004
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• training_steps: 1820
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
0.0513	0.0327	20	0.0990	0.9421	0.9307	0.9364	0.9776
0.0862	0.0654	40	0.1131	0.9052	0.9417	0.9231	0.9730
0.0883	0.0980	60	0.1008	0.9070	0.9417	0.9240	0.9736
0.0779	0.1307	80	0.1018	0.9014	0.9447	0.9226	0.9733
0.1107	0.1634	100	0.1126	0.9109	0.9451	0.9277	0.9705
0.1039	0.1961	120	0.0843	0.9291	0.9451	0.9370	0.9770
0.0875	0.2288	140	0.0921	0.9127	0.9422	0.9272	0.9739
0.0838	0.2614	160	0.0949	0.9208	0.9555	0.9378	0.9765
0.0785	0.2941	180	0.0847	0.9176	0.9549	0.9359	0.9768
0.0752	0.3268	200	0.0989	0.9068	0.9497	0.9277	0.9721
0.0891	0.3595	220	0.0990	0.9014	0.9555	0.9277	0.9738
0.1096	0.3922	240	0.0907	0.9073	0.9583	0.9321	0.9746
0.1101	0.4248	260	0.0848	0.9137	0.9500	0.9315	0.9754
0.0745	0.4575	280	0.0824	0.9126	0.9384	0.9253	0.9751
0.0778	0.4902	300	0.0802	0.9271	0.9561	0.9414	0.9785
0.0823	0.5229	320	0.0955	0.9030	0.9402	0.9212	0.9732
0.0866	0.5556	340	0.1133	0.8918	0.9252	0.9082	0.9667
0.0855	0.5882	360	0.0853	0.9205	0.9564	0.9381	0.9777
0.0819	0.6209	380	0.0891	0.9162	0.9543	0.9349	0.9760
0.0646	0.6536	400	0.0872	0.9127	0.9473	0.9297	0.9757
0.0661	0.6863	420	0.0943	0.9177	0.9561	0.9365	0.9768
0.0802	0.7190	440	0.0892	0.9392	0.9363	0.9378	0.9771
0.0736	0.7516	460	0.0833	0.9204	0.9421	0.9311	0.9759
0.0838	0.7843	480	0.0835	0.9255	0.9622	0.9435	0.9780
0.0756	0.8170	500	0.0805	0.9302	0.9491	0.9396	0.9781
0.0431	0.8497	520	0.0924	0.9184	0.9560	0.9368	0.9771
0.0708	0.8824	540	0.0833	0.9225	0.9540	0.9380	0.9774
0.0796	0.9150	560	0.0883	0.9104	0.9476	0.9286	0.9754
0.0826	0.9477	580	0.0792	0.9200	0.9616	0.9403	0.9786
0.091	0.9804	600	0.0832	0.9324	0.9531	0.9426	0.9780
0.0818	1.0131	620	0.0790	0.9273	0.9546	0.9408	0.9780
0.0412	1.0458	640	0.0770	0.9312	0.9515	0.9413	0.9796
0.0535	1.0784	660	0.0868	0.9383	0.9519	0.9450	0.9788
0.0633	1.1111	680	0.0825	0.9239	0.9592	0.9412	0.9788
0.0495	1.1438	700	0.0869	0.9263	0.9615	0.9436	0.9790
0.0612	1.1765	720	0.0729	0.9409	0.9586	0.9497	0.9816
0.0483	1.2092	740	0.0890	0.9218	0.9472	0.9344	0.9777
0.0749	1.2418	760	0.0765	0.9375	0.9538	0.9456	0.9802
0.0732	1.2745	780	0.0824	0.9169	0.9434	0.9300	0.9768
0.0614	1.3072	800	0.0773	0.9381	0.9562	0.9471	0.9799
0.0598	1.3399	820	0.0796	0.9307	0.9599	0.9451	0.9800
0.0608	1.3725	840	0.0810	0.9315	0.9466	0.9390	0.9778
0.0594	1.4052	860	0.0738	0.9367	0.9577	0.9471	0.9801
0.0575	1.4379	880	0.0784	0.9441	0.9504	0.9473	0.9802
0.0616	1.4706	900	0.0853	0.9243	0.9563	0.9401	0.9771
0.042	1.5033	920	0.0824	0.9266	0.9442	0.9353	0.9767
0.044	1.5359	940	0.0813	0.9263	0.9570	0.9414	0.9789
0.0629	1.5686	960	0.0828	0.9307	0.9601	0.9452	0.9784
0.0502	1.6013	980	0.0743	0.9393	0.9521	0.9457	0.9804
0.0488	1.6340	1000	0.0849	0.9257	0.9568	0.9410	0.9783
0.0391	1.6667	1020	0.0779	0.9386	0.9570	0.9477	0.9799
0.0488	1.6993	1040	0.0787	0.9318	0.9611	0.9462	0.9799
0.0461	1.7320	1060	0.0807	0.9304	0.9600	0.9450	0.9792
0.0546	1.7647	1080	0.0761	0.9303	0.9502	0.9401	0.9796
0.0507	1.7974	1100	0.0747	0.9365	0.9549	0.9456	0.9802
0.0583	1.8301	1120	0.0706	0.9394	0.9566	0.9479	0.9806
0.0546	1.8627	1140	0.0736	0.9373	0.9539	0.9455	0.9810
0.0581	1.8954	1160	0.0829	0.9176	0.9531	0.9350	0.9775
0.0541	1.9281	1180	0.0792	0.9410	0.9577	0.9493	0.9814
0.0552	1.9608	1200	0.0807	0.9368	0.9562	0.9464	0.9808
0.0536	1.9935	1220	0.0759	0.9324	0.9580	0.9450	0.9805
0.036	2.0261	1240	0.0768	0.9338	0.9587	0.9461	0.9806
0.0337	2.0588	1260	0.0749	0.9367	0.9530	0.9448	0.9807
0.0302	2.0915	1280	0.0759	0.9380	0.9556	0.9467	0.9805
0.0241	2.1242	1300	0.0746	0.9376	0.9609	0.9491	0.9819
0.0326	2.1569	1320	0.0780	0.9317	0.9550	0.9432	0.9803
0.0586	2.1895	1340	0.0808	0.9251	0.9489	0.9368	0.9785
0.0357	2.2222	1360	0.0775	0.9333	0.9561	0.9445	0.9801
0.0302	2.2549	1380	0.0797	0.9345	0.9543	0.9443	0.9801
0.0361	2.2876	1400	0.0819	0.9344	0.9550	0.9446	0.9796
0.0324	2.3203	1420	0.0804	0.9363	0.9557	0.9459	0.9802
0.0302	2.3529	1440	0.0775	0.9400	0.9548	0.9473	0.9812
0.0359	2.3856	1460	0.0792	0.9363	0.9567	0.9464	0.9805
0.0371	2.4183	1480	0.0750	0.9399	0.9525	0.9461	0.9810
0.0449	2.4510	1500	0.0771	0.9300	0.9513	0.9405	0.9796
0.0444	2.4837	1520	0.0758	0.9306	0.9544	0.9423	0.9795
0.037	2.5163	1540	0.0773	0.9355	0.9576	0.9464	0.9808
0.0372	2.5490	1560	0.0758	0.9348	0.9591	0.9468	0.9811
0.0403	2.5817	1580	0.0777	0.9309	0.9585	0.9445	0.9803
0.0401	2.6144	1600	0.0749	0.9381	0.9593	0.9486	0.9812
0.032	2.6471	1620	0.0751	0.9397	0.9581	0.9488	0.9810
0.0319	2.6797	1640	0.0768	0.9369	0.9582	0.9475	0.9804
0.0329	2.7124	1660	0.0752	0.9377	0.9586	0.9480	0.9806
0.0376	2.7451	1680	0.0748	0.9350	0.9551	0.9449	0.9803
0.0277	2.7778	1700	0.0761	0.9328	0.9563	0.9444	0.9800
0.0388	2.8105	1720	0.0771	0.9318	0.9573	0.9444	0.9799
0.0364	2.8431	1740	0.0759	0.9347	0.9583	0.9464	0.9805
0.0218	2.8758	1760	0.0755	0.9361	0.9574	0.9466	0.9806
0.0321	2.9085	1780	0.0750	0.9379	0.9567	0.9472	0.9808
0.0414	2.9412	1800	0.0742	0.9393	0.9561	0.9476	0.9810
0.031	2.9739	1820	0.0742	0.9394	0.9563	0.9478	0.9810

Framework versions

• PEFT 0.13.2
• Transformers 4.46.3
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.20.3