What is the accuracy of the model? Why can't I fine-tune when I set the accuracy to FP16?

#157
by chengligen - opened

When I was fine-tuning with whisper-large-v3, my 4090 kept running out of memory, even though I set the batch_size to 1. I then checked the accuracy of the model and it was float32, but I had already used accelerator = Accelerator(mixed_precision="fp16")

When I was fine-tuning with whisper-large-v3, my 4090 kept running out of memory, even though I set the batch_size to 1. I then checked the accuracy of the model and it was float32, but I had already used accelerator = Accelerator(mixed_precision="fp16")

Finetuning the large-v3 checkpoint in fp16 mixed-precision requires 40GB for batch size of 1.

Sign up or log in to comment