bart-large-cnn-finetuned-Kaggle-Science-LLM

This model is a fine-tuned version of facebook/bart-large-cnn on the None dataset. It achieves the following results on the evaluation set:

• Loss: 6.4896
• Rouge1: 29.4886
• Rouge2: 10.2696
• Rougel: 22.611
• Rougelsum: 23.6936
• Gen Len: 70.1

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	90	2.9814	32.5407	12.8638	25.9593	28.0874	66.05
No log	2.0	180	3.1081	33.6875	13.0896	25.2244	26.9945	68.25
No log	3.0	270	3.4845	33.889	12.8396	26.2138	28.2817	70.55
No log	4.0	360	3.8911	31.8492	12.0458	23.4026	25.8547	66.25
No log	5.0	450	4.3530	31.2083	11.0996	23.9196	26.1564	72.25
1.4121	6.0	540	4.4582	29.7758	11.1798	22.9812	24.9141	72.2
1.4121	7.0	630	4.5299	30.3925	11.41	23.9357	25.4386	74.15
1.4121	8.0	720	5.0756	30.1282	10.1879	22.5263	24.3294	71.05
1.4121	9.0	810	5.2213	29.1958	11.9758	22.9344	25.3243	70.95
1.4121	10.0	900	5.0236	32.2902	12.9557	24.9154	26.9866	71.85
1.4121	11.0	990	5.2231	29.9105	11.4629	22.5421	24.7261	73.15
0.1808	12.0	1080	5.4899	30.6426	10.8586	23.0649	25.4052	69.35
0.1808	13.0	1170	5.5205	31.4239	12.4297	24.2742	25.8058	64.9
0.1808	14.0	1260	5.4710	31.3377	11.5225	23.4415	25.9487	68.3
0.1808	15.0	1350	5.3894	30.5681	11.3301	22.5992	25.0445	67.1
0.1808	16.0	1440	5.7293	30.7485	10.2947	23.2461	25.1156	67.8
0.0634	17.0	1530	5.8342	27.8846	9.4002	20.5223	22.8928	73.7
0.0634	18.0	1620	5.7280	31.3703	12.7091	24.947	27.6756	68.7
0.0634	19.0	1710	6.0204	29.311	10.8717	22.2206	23.6151	66.05
0.0634	20.0	1800	5.8662	30.3449	10.9645	22.7105	25.3131	75.6
0.0634	21.0	1890	6.0514	29.4108	10.9479	22.1319	23.8446	70.6
0.0634	22.0	1980	5.9087	30.1637	10.7748	21.7979	23.8345	71.6
0.0281	23.0	2070	6.1406	30.3179	11.0906	23.2057	24.9556	69.65
0.0281	24.0	2160	6.0541	29.7931	11.492	22.7251	24.4958	68.9
0.0281	25.0	2250	6.4349	29.6705	11.3079	22.1845	24.0782	68.2
0.0281	26.0	2340	6.2949	30.3573	9.7319	22.8766	25.5102	68.65
0.0281	27.0	2430	6.3606	30.2358	10.7457	22.9097	24.7486	69.8
0.0167	28.0	2520	6.2235	29.131	11.0196	23.0364	24.7254	69.0
0.0167	29.0	2610	6.2203	30.0767	10.4042	23.0845	24.5571	71.15
0.0167	30.0	2700	6.3899	29.524	11.0226	22.7426	24.7137	71.45
0.0167	31.0	2790	6.4216	29.9921	11.1592	22.7774	25.4653	70.35
0.0167	32.0	2880	6.4758	29.4138	10.1446	22.5501	24.4203	68.0
0.0167	33.0	2970	6.4529	30.7129	9.9512	23.3078	25.1444	70.1
0.0086	34.0	3060	6.3910	32.0673	11.8157	24.4371	26.4378	67.4
0.0086	35.0	3150	6.4725	31.0417	11.8642	23.9718	25.9358	65.5
0.0086	36.0	3240	6.5413	31.2471	11.9972	24.537	25.6679	66.6
0.0086	37.0	3330	6.6040	30.6614	11.4845	23.6335	26.3165	72.15
0.0086	38.0	3420	6.4808	30.1209	10.4855	22.7931	24.9675	74.75
0.0053	39.0	3510	6.4196	29.9709	11.1147	23.3882	25.1429	73.3
0.0053	40.0	3600	6.4798	32.6666	11.6476	24.0167	25.8167	67.7
0.0053	41.0	3690	6.4364	31.7081	11.4081	23.8924	25.3477	67.35
0.0053	42.0	3780	6.4463	31.371	11.3334	23.8642	25.5894	67.85
0.0053	43.0	3870	6.4507	29.6148	11.0601	22.5613	24.2758	70.95
0.0053	44.0	3960	6.5410	30.9704	10.054	22.8276	25.1106	66.25
0.0036	45.0	4050	6.4484	30.6993	10.2855	22.8241	25.1591	69.3
0.0036	46.0	4140	6.4579	29.6269	10.353	21.9677	23.4709	71.15
0.0036	47.0	4230	6.4931	29.8756	10.4957	23.039	24.2656	69.0
0.0036	48.0	4320	6.4831	29.6629	10.0869	22.8167	24.0125	70.35
0.0036	49.0	4410	6.4871	29.908	10.3116	22.9103	24.0365	71.9
0.0023	50.0	4500	6.4896	29.4886	10.2696	22.611	23.6936	70.1

Framework versions

• Transformers 4.33.3
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3