roberta-base-sst-2-16-13

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1304
• Accuracy: 0.9688

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 150

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	1	0.6964	0.5
No log	2.0	2	0.6964	0.5
No log	3.0	3	0.6964	0.5
No log	4.0	4	0.6964	0.5
No log	5.0	5	0.6964	0.5
No log	6.0	6	0.6964	0.5
No log	7.0	7	0.6964	0.5
No log	8.0	8	0.6964	0.5
No log	9.0	9	0.6964	0.5
0.6977	10.0	10	0.6964	0.5
0.6977	11.0	11	0.6963	0.5
0.6977	12.0	12	0.6963	0.5
0.6977	13.0	13	0.6963	0.5
0.6977	14.0	14	0.6963	0.5
0.6977	15.0	15	0.6962	0.5
0.6977	16.0	16	0.6962	0.5
0.6977	17.0	17	0.6962	0.5
0.6977	18.0	18	0.6962	0.5
0.6977	19.0	19	0.6961	0.5
0.6939	20.0	20	0.6961	0.5
0.6939	21.0	21	0.6961	0.5
0.6939	22.0	22	0.6960	0.5
0.6939	23.0	23	0.6960	0.5
0.6939	24.0	24	0.6959	0.5
0.6939	25.0	25	0.6959	0.5
0.6939	26.0	26	0.6958	0.5
0.6939	27.0	27	0.6958	0.5
0.6939	28.0	28	0.6958	0.5
0.6939	29.0	29	0.6957	0.5
0.6972	30.0	30	0.6957	0.5
0.6972	31.0	31	0.6956	0.5
0.6972	32.0	32	0.6956	0.5
0.6972	33.0	33	0.6955	0.5
0.6972	34.0	34	0.6954	0.5
0.6972	35.0	35	0.6954	0.5
0.6972	36.0	36	0.6953	0.5
0.6972	37.0	37	0.6953	0.5
0.6972	38.0	38	0.6952	0.5
0.6972	39.0	39	0.6951	0.5
0.6931	40.0	40	0.6950	0.5
0.6931	41.0	41	0.6950	0.5
0.6931	42.0	42	0.6949	0.5
0.6931	43.0	43	0.6948	0.5
0.6931	44.0	44	0.6947	0.5
0.6931	45.0	45	0.6947	0.5
0.6931	46.0	46	0.6946	0.5
0.6931	47.0	47	0.6945	0.5
0.6931	48.0	48	0.6944	0.5
0.6931	49.0	49	0.6944	0.5
0.6938	50.0	50	0.6943	0.5
0.6938	51.0	51	0.6942	0.5
0.6938	52.0	52	0.6941	0.5
0.6938	53.0	53	0.6941	0.5
0.6938	54.0	54	0.6940	0.5
0.6938	55.0	55	0.6939	0.5
0.6938	56.0	56	0.6938	0.5
0.6938	57.0	57	0.6937	0.5
0.6938	58.0	58	0.6936	0.5
0.6938	59.0	59	0.6935	0.5
0.6914	60.0	60	0.6934	0.5
0.6914	61.0	61	0.6933	0.5
0.6914	62.0	62	0.6932	0.5
0.6914	63.0	63	0.6931	0.5
0.6914	64.0	64	0.6930	0.5
0.6914	65.0	65	0.6929	0.5
0.6914	66.0	66	0.6928	0.5
0.6914	67.0	67	0.6926	0.5
0.6914	68.0	68	0.6925	0.5
0.6914	69.0	69	0.6924	0.5
0.6842	70.0	70	0.6923	0.5
0.6842	71.0	71	0.6921	0.5
0.6842	72.0	72	0.6920	0.5
0.6842	73.0	73	0.6918	0.5
0.6842	74.0	74	0.6917	0.5
0.6842	75.0	75	0.6915	0.5
0.6842	76.0	76	0.6914	0.5
0.6842	77.0	77	0.6912	0.5
0.6842	78.0	78	0.6910	0.5
0.6842	79.0	79	0.6908	0.5
0.6817	80.0	80	0.6906	0.5
0.6817	81.0	81	0.6904	0.5
0.6817	82.0	82	0.6902	0.5
0.6817	83.0	83	0.6900	0.5
0.6817	84.0	84	0.6897	0.5
0.6817	85.0	85	0.6895	0.5
0.6817	86.0	86	0.6892	0.5
0.6817	87.0	87	0.6889	0.5
0.6817	88.0	88	0.6886	0.5
0.6817	89.0	89	0.6882	0.5
0.6684	90.0	90	0.6879	0.5
0.6684	91.0	91	0.6875	0.5
0.6684	92.0	92	0.6870	0.5
0.6684	93.0	93	0.6866	0.5312
0.6684	94.0	94	0.6861	0.5
0.6684	95.0	95	0.6856	0.5
0.6684	96.0	96	0.6850	0.5
0.6684	97.0	97	0.6843	0.5938
0.6684	98.0	98	0.6837	0.7188
0.6684	99.0	99	0.6829	0.75
0.6657	100.0	100	0.6821	0.75
0.6657	101.0	101	0.6812	0.7812
0.6657	102.0	102	0.6802	0.7812
0.6657	103.0	103	0.6791	0.7812
0.6657	104.0	104	0.6780	0.7812
0.6657	105.0	105	0.6767	0.7812
0.6657	106.0	106	0.6752	0.8125
0.6657	107.0	107	0.6736	0.75
0.6657	108.0	108	0.6717	0.75
0.6657	109.0	109	0.6696	0.75
0.6423	110.0	110	0.6671	0.75
0.6423	111.0	111	0.6642	0.7812
0.6423	112.0	112	0.6610	0.8125
0.6423	113.0	113	0.6572	0.8438
0.6423	114.0	114	0.6528	0.8125
0.6423	115.0	115	0.6477	0.8125
0.6423	116.0	116	0.6415	0.7812
0.6423	117.0	117	0.6342	0.7812
0.6423	118.0	118	0.6262	0.7812
0.6423	119.0	119	0.6180	0.7812
0.574	120.0	120	0.6090	0.7812
0.574	121.0	121	0.5987	0.7812
0.574	122.0	122	0.5867	0.7812
0.574	123.0	123	0.5732	0.7812
0.574	124.0	124	0.5579	0.7812
0.574	125.0	125	0.5410	0.8125
0.574	126.0	126	0.5226	0.9062
0.574	127.0	127	0.5031	0.9062
0.574	128.0	128	0.4823	0.9062
0.574	129.0	129	0.4605	0.9062
0.4243	130.0	130	0.4378	0.9375
0.4243	131.0	131	0.4148	0.9375
0.4243	132.0	132	0.3925	0.9375
0.4243	133.0	133	0.3714	0.9375
0.4243	134.0	134	0.3512	0.9688
0.4243	135.0	135	0.3324	0.9688
0.4243	136.0	136	0.3139	0.9688
0.4243	137.0	137	0.2955	0.9688
0.4243	138.0	138	0.2787	0.9375
0.4243	139.0	139	0.2633	0.9375
0.1979	140.0	140	0.2484	0.9688
0.1979	141.0	141	0.2332	0.9688
0.1979	142.0	142	0.2174	0.9688
0.1979	143.0	143	0.2015	0.9688
0.1979	144.0	144	0.1867	0.9688
0.1979	145.0	145	0.1734	0.9375
0.1979	146.0	146	0.1616	0.9375
0.1979	147.0	147	0.1511	0.9375
0.1979	148.0	148	0.1424	0.9688
0.1979	149.0	149	0.1354	0.9688
0.0588	150.0	150	0.1304	0.9688

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3