microsoft

/

swin-large-patch4-window12-384

like 1

I have tried to reproduce the accuracy results (models fine tuned on imagenet-1k). The accuracy as reported in the offical github is 87.3 (last row in the attached image). However, this model never reaches to that accuracy. The best accuracy so far is 87.03% (-0.27). This is the command line that I am using for finetuning. I have found one slight discrepency between this model and what included in github, that is DROP_PATH_RATE which in HF model is 0.1 but in github it is 0.2 (not sure if that makes a huge difference).

Do you have any suggestions how to get to the reported accuracy?

python3 transformer-sparsity/examples/pytorch/image-classification/run_image_classification.py \
    --model_name_or_path microsoft/swin-large-patch4-window12-384 \
    --dataset_name imagenet-1k \
    --remove_unused_columns false \
    --per_device_train_batch_size 16 \
    --per_device_eval_batch_size 16 \
    --do_train \
    --do_eval \
    --lr_scheduler_type ${lr_schedule} \
    --weight_decay 1e-8 \
    --warmup_ratio 0.16 \
    --optim adamw_torch \
    --load_best_model_at_end \
    --eval_steps ${eval_steps} \
    --save_steps ${eval_steps} \
    --evaluation_strategy steps \
    --logging_steps ${eval_steps} \
    --logging_strategy steps \
    --num_train_epochs ${epochs} \
    --learning_rate ${lr} \
    --save_total_limit 4 \
    --overwrite_output_dir \
    --output_dir ~/${base_dir} 2>&1 | tee ~/${base_dir}/swin_large_finetune.txt