qwenvl-2B-cadica-stenosis-classify-lora

This model is a fine-tuned version of AdaptLLM/biomed-Qwen2-VL-2B-Instruct on the CADICA狹窄分析選擇題(TRAIN) dataset. It achieves the following results on the evaluation set:

• Loss: 0.7947
• Num Input Tokens Seen: 11152104

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• total_eval_batch_size: 4
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 2.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.9039	0.1396	50	0.9039	779728
0.9033	0.2792	100	0.9009	1559632
0.9001	0.4188	150	0.8988	2339368
0.902	0.5585	200	0.9004	3119064
0.8933	0.6981	250	0.9052	3898784
0.897	0.8377	300	0.9004	4678472
0.8997	0.9773	350	0.9016	5458104
0.9109	1.1145	400	0.8960	6224248
0.8127	1.2541	450	0.8822	7003904
0.8198	1.3937	500	0.8460	7783528
0.832	1.5333	550	0.8188	8563264
0.786	1.6729	600	0.8021	9343120
0.8312	1.8126	650	0.7986	10122936
0.7797	1.9522	700	0.7947	10902632

Framework versions

• PEFT 0.12.0
• Transformers 4.47.0.dev0
• Pytorch 2.5.1+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3