Bharat Soni's picture
2 14

Bharat Soni

bharatxoni

AI & ML interests

None yet

Recent Activity

upvoted a collection 16 days ago
InternVL2.5
liked a Space 29 days ago
showlab/ShowUI
liked a model about 1 month ago
jadechoghari/Ferret-UI-Llama8b
View all activity

Organizations

None yet

bharatxoni's activity

reacted to merve's post with ๐Ÿค— 6 months ago
view post
Post
6044
Fine-tune Florence-2 on any task ๐Ÿ”ฅ

Today we release a notebook and a walkthrough blog on fine-tuning Florence-2 on DocVQA dataset @andito @SkalskiP

Blog: https://huggingface.co./blog ๐Ÿ“•
Notebook: https://colab.research.google.com/drive/1hKDrJ5AH_o7I95PtZ9__VlCTNAo1Gjpf?usp=sharing ๐Ÿ“–
Florence-2 is a great vision-language model thanks to it's massive dataset and small size!

This model requires conditioning through task prefixes and it's not as generalist, requiring fine-tuning on a new task, such as DocVQA ๐Ÿ“

We have fine-tuned the model on A100 (and one can also use a smaller GPU with smaller batch size) and saw that model picks up new tasks ๐Ÿฅน

See below how it looks like before and after FT ๐Ÿคฉ
Play with the demo here andito/Florence-2-DocVQA ๐Ÿ„โ€โ™€๏ธ