nyt-ingredient-tagger-gte-small-L3-ingredient-v2

This model is a fine-tuned version of napsternxg/gte-small-L3-ingredient-v2 on the nyt_ingredients dataset. It achieves the following results on the evaluation set:

• Loss: 0.4618
• Comment: {'precision': 0.639661138288987, 'recall': 0.7582743988684583, 'f1': 0.6939356675943305, 'number': 7070}
• Name: {'precision': 0.7939526184538653, 'recall': 0.8216129032258065, 'f1': 0.8075459733671528, 'number': 9300}
• Qty: {'precision': 0.9855801031882524, 'recall': 0.9875397667020148, 'f1': 0.9865589617956697, 'number': 7544}
• Range End: {'precision': 0.6176470588235294, 'recall': 0.875, 'f1': 0.7241379310344829, 'number': 96}
• Unit: {'precision': 0.9226940426193809, 'recall': 0.9844009293063392, 'f1': 0.9525491770373343, 'number': 6026}
• Overall Precision: 0.8238
• Overall Recall: 0.8812
• Overall F1: 0.8515
• Overall Accuracy: 0.8364

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Comment	Name	Qty	Range End	Unit	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.6038	0.2	1000	0.5654	{'precision': 0.5337459131247081, 'recall': 0.662463768115942, 'f1': 0.5911795137092601, 'number': 6900}	{'precision': 0.7791639308382579, 'recall': 0.8054298642533937, 'f1': 0.7920792079207921, 'number': 8840}	{'precision': 0.9738444505950733, 'recall': 0.9815873901520435, 'f1': 0.9777005904828066, 'number': 7169}	{'precision': 0.5523809523809524, 'recall': 0.6170212765957447, 'f1': 0.5829145728643217, 'number': 94}	{'precision': 0.918520942408377, 'recall': 0.9716164762893735, 'f1': 0.9443229604709841, 'number': 5778}	0.7834	0.8478	0.8143	0.7991
0.5583	0.4	2000	0.5386	{'precision': 0.575587084148728, 'recall': 0.6820289855072463, 'f1': 0.6243035287874767, 'number': 6900}	{'precision': 0.7881196864303853, 'recall': 0.8074660633484163, 'f1': 0.7976755880873889, 'number': 8840}	{'precision': 0.982230306422275, 'recall': 0.9792160691867764, 'f1': 0.9807208717518859, 'number': 7169}	{'precision': 0.5246913580246914, 'recall': 0.9042553191489362, 'f1': 0.6640625, 'number': 94}	{'precision': 0.9144843194309732, 'recall': 0.9790584977500866, 'f1': 0.9456703443664326, 'number': 5778}	0.8008	0.8549	0.8270	0.8086
0.5351	0.59	3000	0.5205	{'precision': 0.5827500296595088, 'recall': 0.7118840579710145, 'f1': 0.6408767695218214, 'number': 6900}	{'precision': 0.7792900696864111, 'recall': 0.8096153846153846, 'f1': 0.7941633377718598, 'number': 8840}	{'precision': 0.974776016540317, 'recall': 0.9864695215511229, 'f1': 0.9805879090404881, 'number': 7169}	{'precision': 0.6027397260273972, 'recall': 0.9361702127659575, 'f1': 0.7333333333333333, 'number': 94}	{'precision': 0.9150242326332795, 'recall': 0.980269989615784, 'f1': 0.946524064171123, 'number': 5778}	0.7978	0.8649	0.8300	0.8152
0.5238	0.79	4000	0.5070	{'precision': 0.592994874298267, 'recall': 0.7042028985507246, 'f1': 0.64383198621969, 'number': 6900}	{'precision': 0.7898789878987899, 'recall': 0.8122171945701357, 'f1': 0.8008923591745678, 'number': 8840}	{'precision': 0.9833984375, 'recall': 0.983261263774585, 'f1': 0.9833298458533863, 'number': 7169}	{'precision': 0.6277372262773723, 'recall': 0.9148936170212766, 'f1': 0.7445887445887446, 'number': 94}	{'precision': 0.9206581948517433, 'recall': 0.9780200761509172, 'f1': 0.948472641826116, 'number': 5778}	0.8079	0.8625	0.8343	0.8203
0.5134	0.99	5000	0.4994	{'precision': 0.6057287278854254, 'recall': 0.7294202898550725, 'f1': 0.6618449602209219, 'number': 6900}	{'precision': 0.7922092132618448, 'recall': 0.8190045248868778, 'f1': 0.8053840591801545, 'number': 8840}	{'precision': 0.980102252314495, 'recall': 0.9893988003905705, 'f1': 0.9847285853116757, 'number': 7169}	{'precision': 0.6776859504132231, 'recall': 0.8723404255319149, 'f1': 0.7627906976744186, 'number': 94}	{'precision': 0.9163571774584208, 'recall': 0.9821737625475944, 'f1': 0.9481246345334559, 'number': 5778}	0.8104	0.8729	0.8405	0.8230
0.4954	1.19	6000	0.5022	{'precision': 0.6057512759865554, 'recall': 0.7052173913043478, 'f1': 0.6517109756914216, 'number': 6900}	{'precision': 0.7810020649929356, 'recall': 0.81289592760181, 'f1': 0.7966298985643812, 'number': 8840}	{'precision': 0.9804736186123806, 'recall': 0.9875854372994839, 'f1': 0.9840166782487838, 'number': 7169}	{'precision': 0.7024793388429752, 'recall': 0.9042553191489362, 'f1': 0.7906976744186046, 'number': 94}	{'precision': 0.9155247460906013, 'recall': 0.9828660436137072, 'f1': 0.9480010015858442, 'number': 5778}	0.8089	0.8650	0.8360	0.8203
0.4998	1.39	7000	0.4921	{'precision': 0.6202128961213753, 'recall': 0.7346376811594203, 'f1': 0.672593378889405, 'number': 6900}	{'precision': 0.7895139268159476, 'recall': 0.8176470588235294, 'f1': 0.8033342595165324, 'number': 8840}	{'precision': 0.9836797321802204, 'recall': 0.9836797321802204, 'f1': 0.9836797321802204, 'number': 7169}	{'precision': 0.6397058823529411, 'recall': 0.925531914893617, 'f1': 0.7565217391304346, 'number': 94}	{'precision': 0.9238952536824877, 'recall': 0.976981654551748, 'f1': 0.9496971736204576, 'number': 5778}	0.8158	0.8714	0.8427	0.8252
0.4912	1.58	8000	0.4944	{'precision': 0.6245536264006896, 'recall': 0.735072463768116, 'f1': 0.6753212169629186, 'number': 6900}	{'precision': 0.7958374628344896, 'recall': 0.8175339366515837, 'f1': 0.8065398136264716, 'number': 8840}	{'precision': 0.9819494584837545, 'recall': 0.9864695215511229, 'f1': 0.9842043003270474, 'number': 7169}	{'precision': 0.6541353383458647, 'recall': 0.925531914893617, 'f1': 0.7665198237885463, 'number': 94}	{'precision': 0.9229010127409344, 'recall': 0.9778470058843891, 'f1': 0.9495798319327731, 'number': 5778}	0.8189	0.8724	0.8448	0.8256
0.4974	1.78	9000	0.4864	{'precision': 0.6223597960670065, 'recall': 0.7430434782608696, 'f1': 0.6773682124455014, 'number': 6900}	{'precision': 0.7926226808650785, 'recall': 0.8167420814479638, 'f1': 0.8045016435456014, 'number': 8840}	{'precision': 0.9842442833240379, 'recall': 0.9846561584600363, 'f1': 0.984450177811868, 'number': 7169}	{'precision': 0.6206896551724138, 'recall': 0.9574468085106383, 'f1': 0.7531380753138076, 'number': 94}	{'precision': 0.9221351616062684, 'recall': 0.9776739356178609, 'f1': 0.949092741935484, 'number': 5778}	0.8167	0.8737	0.8442	0.8258
0.4872	1.98	10000	0.4848	{'precision': 0.6306733167082295, 'recall': 0.7330434782608696, 'f1': 0.6780160857908847, 'number': 6900}	{'precision': 0.7935724470768893, 'recall': 0.8184389140271493, 'f1': 0.8058138887341983, 'number': 8840}	{'precision': 0.9848147116188354, 'recall': 0.9860510531454875, 'f1': 0.9854324945981738, 'number': 7169}	{'precision': 0.6641221374045801, 'recall': 0.925531914893617, 'f1': 0.7733333333333333, 'number': 94}	{'precision': 0.922813315926893, 'recall': 0.9787123572170301, 'f1': 0.949941206114564, 'number': 5778}	0.8211	0.8722	0.8459	0.8277
0.4777	2.18	11000	0.4820	{'precision': 0.6294968475707752, 'recall': 0.7379710144927536, 'f1': 0.6794315831609847, 'number': 6900}	{'precision': 0.7957142857142857, 'recall': 0.8191176470588235, 'f1': 0.8072463768115943, 'number': 8840}	{'precision': 0.9861557824080548, 'recall': 0.9836797321802204, 'f1': 0.9849162011173185, 'number': 7169}	{'precision': 0.6796875, 'recall': 0.925531914893617, 'f1': 0.7837837837837838, 'number': 94}	{'precision': 0.9237939493049877, 'recall': 0.9776739356178609, 'f1': 0.949970570924073, 'number': 5778}	0.8214	0.8728	0.8463	0.8307
0.4781	2.38	12000	0.4821	{'precision': 0.6305942773294204, 'recall': 0.7473913043478261, 'f1': 0.6840429765220851, 'number': 6900}	{'precision': 0.7970950704225352, 'recall': 0.8194570135746606, 'f1': 0.8081213743864346, 'number': 8840}	{'precision': 0.9850746268656716, 'recall': 0.9850746268656716, 'f1': 0.9850746268656716, 'number': 7169}	{'precision': 0.6496350364963503, 'recall': 0.9468085106382979, 'f1': 0.7705627705627707, 'number': 94}	{'precision': 0.923453566182471, 'recall': 0.9792315680166147, 'f1': 0.9505249895002099, 'number': 5778}	0.8212	0.8759	0.8477	0.8304
0.4804	2.57	13000	0.4809	{'precision': 0.6338080495356038, 'recall': 0.7417391304347826, 'f1': 0.6835392320534225, 'number': 6900}	{'precision': 0.7985173710998008, 'recall': 0.8164027149321267, 'f1': 0.8073610023492561, 'number': 8840}	{'precision': 0.9853617733166039, 'recall': 0.9859115636769424, 'f1': 0.9856365918281969, 'number': 7169}	{'precision': 0.6382978723404256, 'recall': 0.9574468085106383, 'f1': 0.7659574468085107, 'number': 94}	{'precision': 0.9213263979193758, 'recall': 0.9809622706818969, 'f1': 0.9502095557418274, 'number': 5778}	0.8228	0.8742	0.8477	0.8298
0.4721	2.77	14000	0.4799	{'precision': 0.6337924249877029, 'recall': 0.7469565217391304, 'f1': 0.6857370941990419, 'number': 6900}	{'precision': 0.795054945054945, 'recall': 0.8184389140271493, 'f1': 0.8065774804905238, 'number': 8840}	{'precision': 0.9849372384937238, 'recall': 0.9850746268656716, 'f1': 0.9850059278889741, 'number': 7169}	{'precision': 0.6616541353383458, 'recall': 0.9361702127659575, 'f1': 0.7753303964757708, 'number': 94}	{'precision': 0.9225008140670792, 'recall': 0.9806161301488404, 'f1': 0.9506711409395974, 'number': 5778}	0.8216	0.8758	0.8478	0.8302
0.4792	2.97	15000	0.4793	{'precision': 0.6364423552696685, 'recall': 0.7456521739130435, 'f1': 0.6867325146823279, 'number': 6900}	{'precision': 0.7941402392186986, 'recall': 0.8186651583710407, 'f1': 0.8062162312705399, 'number': 8840}	{'precision': 0.9848062447727907, 'recall': 0.985493095271307, 'f1': 0.9851495503032839, 'number': 7169}	{'precision': 0.6616541353383458, 'recall': 0.9361702127659575, 'f1': 0.7753303964757708, 'number': 94}	{'precision': 0.921151032352463, 'recall': 0.9806161301488404, 'f1': 0.9499538938720764, 'number': 5778}	0.8221	0.8756	0.8480	0.8304

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1