tiny_bert_27_mva_intents

This model is a fine-tuned version of prajjwal1/bert-tiny on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1029
• Accuracy: 0.9751

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	271	2.2533	0.6050
2.3347	2.0	542	2.0443	0.6570
2.3347	3.0	813	1.8496	0.7089
1.9691	4.0	1084	1.6831	0.7380
1.9691	5.0	1355	1.5359	0.7817
1.6616	6.0	1626	1.4010	0.7942
1.6616	7.0	1897	1.2820	0.8129
1.4124	8.0	2168	1.1748	0.8337
1.4124	9.0	2439	1.0754	0.8503
1.1972	10.0	2710	0.9926	0.8565
1.1972	11.0	2981	0.9165	0.8586
1.0186	12.0	3252	0.8477	0.8690
0.8756	13.0	3523	0.7845	0.8815
0.8756	14.0	3794	0.7295	0.8857
0.7602	15.0	4065	0.6794	0.8940
0.7602	16.0	4336	0.6333	0.9002
0.6497	17.0	4607	0.5906	0.9106
0.6497	18.0	4878	0.5533	0.9168
0.5763	19.0	5149	0.5206	0.9210
0.5763	20.0	5420	0.4859	0.9210
0.4944	21.0	5691	0.4572	0.9231
0.4944	22.0	5962	0.4260	0.9168
0.4367	23.0	6233	0.3999	0.9272
0.3791	24.0	6504	0.3841	0.9272
0.3791	25.0	6775	0.3602	0.9314
0.3364	26.0	7046	0.3426	0.9293
0.3364	27.0	7317	0.3210	0.9376
0.3053	28.0	7588	0.3078	0.9376
0.3053	29.0	7859	0.2881	0.9501
0.2633	30.0	8130	0.2765	0.9501
0.2633	31.0	8401	0.2657	0.9522
0.2384	32.0	8672	0.2518	0.9543
0.2384	33.0	8943	0.2368	0.9563
0.2175	34.0	9214	0.2273	0.9563
0.2175	35.0	9485	0.2188	0.9605
0.1872	36.0	9756	0.2101	0.9605
0.176	37.0	10027	0.2039	0.9626
0.176	38.0	10298	0.1986	0.9605
0.1532	39.0	10569	0.1895	0.9626
0.1532	40.0	10840	0.1827	0.9605
0.1434	41.0	11111	0.1770	0.9626
0.1434	42.0	11382	0.1723	0.9626
0.1309	43.0	11653	0.1647	0.9626
0.1309	44.0	11924	0.1664	0.9605
0.1208	45.0	12195	0.1574	0.9626
0.1208	46.0	12466	0.1549	0.9626
0.1083	47.0	12737	0.1499	0.9647
0.1033	48.0	13008	0.1525	0.9626
0.1033	49.0	13279	0.1422	0.9667
0.0907	50.0	13550	0.1385	0.9688
0.0907	51.0	13821	0.1423	0.9667
0.0888	52.0	14092	0.1395	0.9688
0.0888	53.0	14363	0.1410	0.9688
0.0829	54.0	14634	0.1326	0.9688
0.0829	55.0	14905	0.1274	0.9709
0.0735	56.0	15176	0.1266	0.9709
0.0735	57.0	15447	0.1235	0.9709
0.0737	58.0	15718	0.1223	0.9709
0.0737	59.0	15989	0.1181	0.9709
0.0639	60.0	16260	0.1134	0.9730
0.062	61.0	16531	0.1113	0.9709
0.062	62.0	16802	0.1151	0.9730
0.057	63.0	17073	0.1149	0.9709
0.057	64.0	17344	0.1112	0.9730
0.0565	65.0	17615	0.1125	0.9709
0.0565	66.0	17886	0.1124	0.9730
0.0564	67.0	18157	0.1112	0.9709
0.0564	68.0	18428	0.1097	0.9709
0.0509	69.0	18699	0.1062	0.9730
0.0509	70.0	18970	0.1083	0.9730
0.0496	71.0	19241	0.1076	0.9709
0.0454	72.0	19512	0.1052	0.9688
0.0454	73.0	19783	0.1018	0.9751
0.0451	74.0	20054	0.1050	0.9730
0.0451	75.0	20325	0.1061	0.9709
0.0446	76.0	20596	0.1037	0.9709
0.0446	77.0	20867	0.1054	0.9709
0.0414	78.0	21138	0.1033	0.9709
0.0414	79.0	21409	0.1030	0.9730
0.043	80.0	21680	0.1015	0.9730
0.043	81.0	21951	0.1020	0.9730
0.0402	82.0	22222	0.1011	0.9751
0.0402	83.0	22493	0.1033	0.9751
0.0364	84.0	22764	0.1031	0.9730
0.0376	85.0	23035	0.1030	0.9730
0.0376	86.0	23306	0.1020	0.9751
0.0386	87.0	23577	0.1021	0.9751
0.0386	88.0	23848	0.1009	0.9751
0.0382	89.0	24119	0.1022	0.9751
0.0382	90.0	24390	0.1020	0.9751
0.0352	91.0	24661	0.1020	0.9751
0.0352	92.0	24932	0.1041	0.9730
0.0363	93.0	25203	0.1034	0.9751
0.0363	94.0	25474	0.1026	0.9751
0.0328	95.0	25745	0.1034	0.9751
0.0357	96.0	26016	0.1026	0.9751
0.0357	97.0	26287	0.1033	0.9751
0.0357	98.0	26558	0.1031	0.9751
0.0357	99.0	26829	0.1030	0.9751
0.0352	100.0	27100	0.1029	0.9751

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1