LLM_Teached_Pegasus_50k

This model is a fine-tuned version of google/pegasus-xsum on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.5934
• Rouge1: 0.4774
• Rouge2: 0.2259
• Rougel: 0.3926
• Rougelsum: 0.3926
• Gen Len: 26.5556
• Precision: 0.9117
• Recall: 0.9103
• F1: 0.9108

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	F1	Gen Len	Validation Loss	Precision	Recall	Rouge1	Rouge2	Rougel	Rougelsum
No log	1.0	390	0.9034	26.2967	1.8258	0.9049	0.9023	0.4338	0.1906	0.3496	0.3498
2.1621	2.0	781	0.9054	26.2727	1.7537	0.9068	0.9044	0.4449	0.2005	0.3633	0.3633
1.8794	3.0	1172	0.9066	26.4345	1.7268	0.9078	0.9058	0.4518	0.2061	0.3696	0.3695
1.8271	4.0	1560	0.9069	26.3971	1.7157	0.9082	0.906	0.4539	0.2075	0.3716	0.3714
1.8271	5.0	1951	0.9074	26.3015	1.7033	0.9087	0.9065	0.4561	0.2098	0.3735	0.3734
1.8067	6.0	2340	0.9077	26.4389	1.6897	0.9089	0.9069	0.4592	0.2114	0.3762	0.3759
1.7833	7.0	2731	0.9079	26.3745	1.6819	0.9092	0.9071	0.4598	0.2115	0.3764	0.376
1.7683	8.0	3120	0.9083	26.6204	1.6763	0.9094	0.9076	0.4621	0.2133	0.3791	0.3789
1.7559	9.0	3511	0.9086	26.424	1.6662	0.9098	0.9078	0.4632	0.215	0.38	0.3799
1.7559	10.0	3902	0.9089	26.5425	1.6594	0.9099	0.9082	0.4651	0.2168	0.3812	0.3812
1.7357	11.0	4293	0.9091	26.6051	1.6555	0.91	0.9086	0.4663	0.2178	0.3824	0.3823
1.7297	12.0	4680	0.9092	26.4393	1.6508	0.9103	0.9084	0.4668	0.2175	0.3823	0.3822
1.7165	13.0	5071	0.9094	26.6385	1.6451	0.9103	0.9089	0.4687	0.2191	0.3834	0.3834
1.7165	14.0	5462	0.9095	26.4156	1.6405	0.9106	0.9087	0.4691	0.2193	0.3845	0.3844
1.7068	15.0	5853	0.9097	26.4571	1.6383	0.9108	0.9089	0.4699	0.2204	0.3853	0.3853
1.7004	16.0	6240	0.9097	26.4247	1.6346	0.9108	0.9089	0.4703	0.2204	0.385	0.385
1.6923	17.0	6631	0.9099	26.4436	1.6305	0.911	0.9091	0.4706	0.221	0.3855	0.3856
1.6839	18.0	7022	0.9098	26.612	1.6285	0.9106	0.9094	0.4712	0.2215	0.3862	0.3864
1.6839	19.0	7413	0.9099	26.5291	1.6263	0.9108	0.9093	0.4709	0.2217	0.3862	0.3864
1.6743	20.0	7800	0.91	26.4251	1.6205	0.9111	0.9092	0.4727	0.2223	0.3876	0.3876
1.6692	21.0	8191	0.9102	26.7484	1.6153	0.911	0.9098	0.4737	0.2229	0.388	0.388
1.6568	22.0	8582	0.9103	26.532	1.6104	0.9113	0.9096	0.4733	0.2221	0.3885	0.3886
1.6568	23.0	8973	0.9104	26.82	1.6056	0.911	0.9101	0.4756	0.2236	0.3891	0.3891
1.6418	24.0	9360	1.6021	0.476	0.2246	0.3903	0.3903	26.5513	0.9115	0.91	0.9106
1.6319	25.0	9751	1.5995	0.4751	0.2245	0.3905	0.3905	26.4375	0.9116	0.9098	0.9105
1.624	26.0	10142	1.5974	0.4756	0.2247	0.3903	0.3904	26.6018	0.9116	0.9101	0.9107
1.6184	27.0	10533	1.5953	0.4747	0.2231	0.3899	0.3899	26.4833	0.9116	0.9099	0.9106
1.6184	28.0	10923	1.5943	0.4758	0.2243	0.3907	0.3908	26.5604	0.9116	0.9102	0.9107
1.6126	29.0	11314	1.5936	0.4776	0.226	0.3926	0.3926	26.5775	0.9117	0.9103	0.9108
1.6148	29.99	11700	1.5934	0.4774	0.2259	0.3926	0.3926	26.5556	0.9117	0.9103	0.9108

Framework versions

• Transformers 4.36.0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.15.0