v4_qwen_lora

This model is a fine-tuned version of Daewon0808/prm800k_qwen_fulltune on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1986
• Prm accuracy: 0.9216
• Prm precision: 0.9710
• Prm recall: 0.9178
• Prm specificty: 0.9310
• Prm npv: 0.8182
• Prm f1: 0.9437
• Prm f1 neg: 0.8710
• Prm f1 auc: 0.9244
• Prm f1 auc (fixed): 0.9502

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 2
• eval_batch_size: 4
• seed: 908932403
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 8
• total_train_batch_size: 128
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Prm accuracy	Prm precision	Prm recall	Prm specificty	Prm npv	Prm f1	Prm f1 neg	Prm f1 auc	Prm f1 auc (fixed)
No log	0	0	0.4028	0.8039	0.9344	0.7808	0.8621	0.6098	0.8507	0.7143	0.8214	0.8420
0.4957	0.0113	5	0.4020	0.8039	0.9344	0.7808	0.8621	0.6098	0.8507	0.7143	0.8214	0.8427
0.4552	0.0225	10	0.4001	0.8039	0.9344	0.7808	0.8621	0.6098	0.8507	0.7143	0.8214	0.8451
0.4974	0.0338	15	0.3895	0.8431	0.9385	0.8356	0.8621	0.6757	0.8841	0.7576	0.8488	0.8467
0.4715	0.0451	20	0.3655	0.8824	0.9420	0.8904	0.8621	0.7576	0.9155	0.8065	0.8762	0.8517
0.4215	0.0563	25	0.3386	0.9020	0.9437	0.9178	0.8621	0.8065	0.9306	0.8333	0.8899	0.8573
0.4069	0.0676	30	0.3204	0.9216	0.9333	0.9589	0.8276	0.8889	0.9459	0.8571	0.8932	0.8630
0.349	0.0789	35	0.3014	0.9118	0.9211	0.9589	0.7931	0.8846	0.9396	0.8364	0.8760	0.8803
0.3483	0.0901	40	0.2888	0.9118	0.9571	0.9178	0.8966	0.8125	0.9371	0.8525	0.9072	0.8864
0.3461	0.1014	45	0.2819	0.8627	0.9538	0.8493	0.8966	0.7027	0.8986	0.7879	0.8729	0.8824
0.3105	0.1126	50	0.2557	0.8627	0.9538	0.8493	0.8966	0.7027	0.8986	0.7879	0.8729	0.8845
0.2924	0.1239	55	0.2360	0.8824	0.9841	0.8493	0.9655	0.7179	0.9118	0.8235	0.9074	0.8977
0.3195	0.1352	60	0.2403	0.8824	0.9841	0.8493	0.9655	0.7179	0.9118	0.8235	0.9074	0.9145
0.3174	0.1464	65	0.2155	0.9216	0.9851	0.9041	0.9655	0.8	0.9429	0.875	0.9348	0.8970
0.3069	0.1577	70	0.2296	0.9020	0.9846	0.8767	0.9655	0.7568	0.9275	0.8485	0.9211	0.8902
0.2821	0.1690	75	0.2621	0.8725	0.9839	0.8356	0.9655	0.7	0.9037	0.8116	0.9006	0.8890
0.2904	0.1802	80	0.2365	0.8922	0.9844	0.8630	0.9655	0.7368	0.9197	0.8358	0.9143	0.8940
0.226	0.1915	85	0.2097	0.9216	0.9851	0.9041	0.9655	0.8	0.9429	0.875	0.9348	0.9027
0.2534	0.2028	90	0.2241	0.8824	0.9841	0.8493	0.9655	0.7179	0.9118	0.8235	0.9074	0.9192
0.2278	0.2140	95	0.2197	0.9020	0.9846	0.8767	0.9655	0.7568	0.9275	0.8485	0.9211	0.8977
0.198	0.2253	100	0.2201	0.8824	0.9420	0.8904	0.8621	0.7576	0.9155	0.8065	0.8762	0.8904
0.2287	0.2366	105	0.2341	0.8922	0.9844	0.8630	0.9655	0.7368	0.9197	0.8358	0.9143	0.9140
0.2597	0.2478	110	0.2366	0.8824	0.9841	0.8493	0.9655	0.7179	0.9118	0.8235	0.9074	0.9280
0.2479	0.2591	115	0.2153	0.9118	0.9706	0.9041	0.9310	0.7941	0.9362	0.8571	0.9176	0.9315
0.232	0.2703	120	0.2051	0.9020	0.9565	0.9041	0.8966	0.7879	0.9296	0.8387	0.9003	0.9339
0.2441	0.2816	125	0.2070	0.9118	0.9706	0.9041	0.9310	0.7941	0.9362	0.8571	0.9176	0.9407
0.2062	0.2929	130	0.2096	0.9020	0.9846	0.8767	0.9655	0.7568	0.9275	0.8485	0.9211	0.9433
0.2558	0.3041	135	0.1993	0.9020	0.9846	0.8767	0.9655	0.7568	0.9275	0.8485	0.9211	0.9466
0.2381	0.3154	140	0.1867	0.9118	0.9571	0.9178	0.8966	0.8125	0.9371	0.8525	0.9072	0.9447
0.2106	0.3267	145	0.2004	0.9118	0.9706	0.9041	0.9310	0.7941	0.9362	0.8571	0.9176	0.9419
0.2474	0.3379	150	0.2106	0.8922	0.9697	0.8767	0.9310	0.75	0.9209	0.8308	0.9039	0.9428
0.2377	0.3492	155	0.2037	0.8922	0.9429	0.9041	0.8621	0.7812	0.9231	0.8197	0.8831	0.9374
0.2411	0.3605	160	0.2217	0.9020	0.9846	0.8767	0.9655	0.7568	0.9275	0.8485	0.9211	0.9485
0.2158	0.3717	165	0.2097	0.9020	0.9565	0.9041	0.8966	0.7879	0.9296	0.8387	0.9003	0.9400
0.2489	0.3830	170	0.2116	0.9020	0.9701	0.8904	0.9310	0.7714	0.9286	0.8438	0.9107	0.9421
0.2406	0.3943	175	0.2371	0.8922	0.9844	0.8630	0.9655	0.7368	0.9197	0.8358	0.9143	0.9445
0.2038	0.4055	180	0.2064	0.9216	0.9851	0.9041	0.9655	0.8	0.9429	0.875	0.9348	0.9490
0.2539	0.4168	185	0.1864	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9405
0.2583	0.4280	190	0.2180	0.8922	0.9844	0.8630	0.9655	0.7368	0.9197	0.8358	0.9143	0.9410
0.24	0.4393	195	0.2054	0.9118	0.9706	0.9041	0.9310	0.7941	0.9362	0.8571	0.9176	0.9339
0.1977	0.4506	200	0.2223	0.8824	0.9692	0.8630	0.9310	0.7297	0.9130	0.8182	0.8970	0.9343
0.1992	0.4618	205	0.2221	0.8922	0.9844	0.8630	0.9655	0.7368	0.9197	0.8358	0.9143	0.9395
0.2732	0.4731	210	0.1947	0.9216	0.9851	0.9041	0.9655	0.8	0.9429	0.875	0.9348	0.9490
0.2074	0.4844	215	0.1795	0.9412	0.9855	0.9315	0.9655	0.8485	0.9577	0.9032	0.9485	0.9495
0.2161	0.4956	220	0.2092	0.9020	0.9846	0.8767	0.9655	0.7568	0.9275	0.8485	0.9211	0.9544
0.2013	0.5069	225	0.2009	0.9020	0.9846	0.8767	0.9655	0.7568	0.9275	0.8485	0.9211	0.9539
0.278	0.5182	230	0.1866	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9499
0.2042	0.5294	235	0.2101	0.8824	0.9841	0.8493	0.9655	0.7179	0.9118	0.8235	0.9074	0.9565
0.2298	0.5407	240	0.2051	0.8922	0.9844	0.8630	0.9655	0.7368	0.9197	0.8358	0.9143	0.9561
0.1887	0.5520	245	0.1976	0.9118	0.9848	0.8904	0.9655	0.7778	0.9353	0.8615	0.9280	0.9499
0.2529	0.5632	250	0.1923	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9438
0.2241	0.5745	255	0.2032	0.8922	0.9844	0.8630	0.9655	0.7368	0.9197	0.8358	0.9143	0.9497
0.1916	0.5858	260	0.1981	0.9020	0.9846	0.8767	0.9655	0.7568	0.9275	0.8485	0.9211	0.9499
0.21	0.5970	265	0.2020	0.9020	0.9846	0.8767	0.9655	0.7568	0.9275	0.8485	0.9211	0.9513
0.2351	0.6083	270	0.1986	0.9020	0.9846	0.8767	0.9655	0.7568	0.9275	0.8485	0.9211	0.9469
0.1987	0.6195	275	0.2003	0.9020	0.9846	0.8767	0.9655	0.7568	0.9275	0.8485	0.9211	0.9445
0.2225	0.6308	280	0.1998	0.9118	0.9848	0.8904	0.9655	0.7778	0.9353	0.8615	0.9280	0.9443
0.2113	0.6421	285	0.1917	0.9118	0.9571	0.9178	0.8966	0.8125	0.9371	0.8525	0.9072	0.9457
0.2216	0.6533	290	0.1924	0.9020	0.9565	0.9041	0.8966	0.7879	0.9296	0.8387	0.9003	0.9469
0.2501	0.6646	295	0.1962	0.9118	0.9706	0.9041	0.9310	0.7941	0.9362	0.8571	0.9176	0.9499
0.2362	0.6759	300	0.1966	0.9118	0.9848	0.8904	0.9655	0.7778	0.9353	0.8615	0.9280	0.9556
0.2129	0.6871	305	0.1958	0.9118	0.9848	0.8904	0.9655	0.7778	0.9353	0.8615	0.9280	0.9542
0.186	0.6984	310	0.1857	0.9216	0.9577	0.9315	0.8966	0.8387	0.9444	0.8667	0.9140	0.9537
0.2212	0.7097	315	0.1817	0.9216	0.9577	0.9315	0.8966	0.8387	0.9444	0.8667	0.9140	0.9532
0.2028	0.7209	320	0.1813	0.9216	0.9577	0.9315	0.8966	0.8387	0.9444	0.8667	0.9140	0.9535
0.1866	0.7322	325	0.1849	0.9314	0.9714	0.9315	0.9310	0.8438	0.9510	0.8852	0.9313	0.9561
0.2064	0.7435	330	0.1925	0.9314	0.9853	0.9178	0.9655	0.8235	0.9504	0.8889	0.9417	0.9561
0.2209	0.7547	335	0.1945	0.9314	0.9853	0.9178	0.9655	0.8235	0.9504	0.8889	0.9417	0.9547
0.2315	0.7660	340	0.1870	0.9314	0.9714	0.9315	0.9310	0.8438	0.9510	0.8852	0.9313	0.9547
0.2652	0.7772	345	0.1865	0.9216	0.9577	0.9315	0.8966	0.8387	0.9444	0.8667	0.9140	0.9530
0.2637	0.7885	350	0.1898	0.9314	0.9714	0.9315	0.9310	0.8438	0.9510	0.8852	0.9313	0.9525
0.237	0.7998	355	0.1956	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9518
0.1956	0.8110	360	0.2010	0.9314	0.9853	0.9178	0.9655	0.8235	0.9504	0.8889	0.9417	0.9513
0.2379	0.8223	365	0.2027	0.9216	0.9851	0.9041	0.9655	0.8	0.9429	0.875	0.9348	0.9523
0.2119	0.8336	370	0.2027	0.9314	0.9853	0.9178	0.9655	0.8235	0.9504	0.8889	0.9417	0.9499
0.2032	0.8448	375	0.2001	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9490
0.2422	0.8561	380	0.1990	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9478
0.2829	0.8674	385	0.1985	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9502
0.2246	0.8786	390	0.1984	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9483
0.1988	0.8899	395	0.1978	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9487
0.1628	0.9012	400	0.1978	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9504
0.1933	0.9124	405	0.1983	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9495
0.2364	0.9237	410	0.1983	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9497
0.1937	0.9349	415	0.1979	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9495
0.2002	0.9462	420	0.1980	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9497
0.1955	0.9575	425	0.1979	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9487
0.2134	0.9687	430	0.1973	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9495
0.1779	0.9800	435	0.1969	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9499
0.2254	0.9913	440	0.1986	0.9216	0.9710	0.9178	0.9310	0.8182	0.9437	0.8710	0.9244	0.9502

Framework versions

• PEFT 0.12.0
• Transformers 4.46.0
• Pytorch 2.4.0+cu118
• Datasets 3.0.0
• Tokenizers 0.20.1