Welcome PaliGemma 2 β New vision language models by Google
β’
118
Merve Noyan
merve
AI & ML interests
VLMs, vision & co
Recent Activity
posted
an
update
1 day ago
QwQ can see π₯
Qwen team released QvQ, a large vision LM with reasoning π±
it outperforms proprietary VLMs on several benchmarks, comes with open weights and a demo!
Check them out β¬οΈ
Demo https://huggingface.co./spaces/Qwen/QVQ-72B-preview
Model https://huggingface.co./Qwen/QVQ-72B-Preview
Read more https://qwenlm.github.io/blog/qvq-72b-preview/
Congratulations @JustinLin610 and team!
updated
a model
5 days ago
merve/colpali_ufo
Articles
Organizations
merve's activity
posted
an
update
1 day ago
Post
1150
QwQ can see π₯
Qwen team released QvQ, a large vision LM with reasoning π±
it outperforms proprietary VLMs on several benchmarks, comes with open weights and a demo!
Check them out β¬οΈ
Demo Qwen/QVQ-72B-preview
Model Qwen/QVQ-72B-Preview
Read more https://qwenlm.github.io/blog/qvq-72b-preview/
Congratulations @JustinLin610 and team!
Qwen team released QvQ, a large vision LM with reasoning π±
it outperforms proprietary VLMs on several benchmarks, comes with open weights and a demo!
Check them out β¬οΈ
Demo Qwen/QVQ-72B-preview
Model Qwen/QVQ-72B-Preview
Read more https://qwenlm.github.io/blog/qvq-72b-preview/
Congratulations @JustinLin610 and team!
reacted to
AdinaY's
post with π₯
1 day ago
Post
1036
QvQ-72B-Previewπ an open weight model for visual reasoning just released by Alibaba_Qwen team
Qwen/qvq-676448c820912236342b9888
β¨ Combines visual understanding & language reasoning.
β¨ Scores 70.3 on MMMU
β¨ Outperforms Qwen2-VL-72B-Instruct in complex problem-solving
Qwen/qvq-676448c820912236342b9888
β¨ Combines visual understanding & language reasoning.
β¨ Scores 70.3 on MMMU
β¨ Outperforms Qwen2-VL-72B-Instruct in complex problem-solving
updated
a
model
5 days ago
updated
a
model
6 days ago
Add FT tutorial link
#22 opened 6 days ago
by
merve
How to training or fientunee SmolVLM easily?
1
#21 opened 11 days ago
by
lucasjin
posted
an
update
7 days ago
Post
2330
Aya by Cohere For AI can now see! π
C4AI community has built Maya 8B, a new open-source multilingual VLM built on SigLIP and Aya 8B π± works on 8 languages! π£οΈ
The authors extend Llava dataset using Aya's translation capabilities with 558k examples!
ry it here kkr5155/maya_demo
Dataset maya-multimodal/pretrain
Model maya-multimodal/maya π
kudos @nahidalam and team
C4AI community has built Maya 8B, a new open-source multilingual VLM built on SigLIP and Aya 8B π± works on 8 languages! π£οΈ
The authors extend Llava dataset using Aya's translation capabilities with 558k examples!
ry it here kkr5155/maya_demo
Dataset maya-multimodal/pretrain
Model maya-multimodal/maya π
kudos @nahidalam and team
- 1 reply
upvoted
a
paper
7 days ago
Update `dataset` to reference to the actual dataset used
#4 opened 7 days ago
by
alvarobartt
posted
an
update
8 days ago
Post
2905
Apollo is a new family of open-source video language models by Meta, where 3B model outperforms most 7B models and 7B outperforms most 30B models π§Ά
β¨ the models come in 1.5B https://huggingface.co./Apollo-LMMs/Apollo-1_5B-t32, 3B https://huggingface.co./Apollo-LMMs/Apollo-3B-t32 and 7B https://huggingface.co./Apollo-LMMs/Apollo-7B-t32 with A2.0 license, based on Qwen1.5 & Qwen2
β¨ the authors also release a benchmark dataset https://huggingface.co./spaces/Apollo-LMMs/ApolloBench
The paper has a lot of experiments (they trained 84 models!) about what makes the video LMs work β―οΈ
Try the demo for best setup here https://huggingface.co./spaces/Apollo-LMMs/Apollo-3B
they evaluate sampling strategies, scaling laws for models and datasets, video representation and more!
> The authors find out that whatever design decision was applied to small models also scale properly when the model and dataset are scaled π scaling dataset has diminishing returns for smaller models
> They evaluate frame sampling strategies, and find that FPS sampling is better than uniform sampling, and they find 8-32 tokens per frame optimal
> They also compare image encoders, they try a variation of models from shape optimized SigLIP to DINOv2
they find google/siglip-so400m-patch14-384 to be most powerful π₯
> they also compare freezing different parts of models, training all stages with some frozen parts give the best yield
They eventually release three models, where Apollo-3B outperforms most 7B models and Apollo 7B outperforms 30B models π₯
β¨ the models come in 1.5B https://huggingface.co./Apollo-LMMs/Apollo-1_5B-t32, 3B https://huggingface.co./Apollo-LMMs/Apollo-3B-t32 and 7B https://huggingface.co./Apollo-LMMs/Apollo-7B-t32 with A2.0 license, based on Qwen1.5 & Qwen2
β¨ the authors also release a benchmark dataset https://huggingface.co./spaces/Apollo-LMMs/ApolloBench
The paper has a lot of experiments (they trained 84 models!) about what makes the video LMs work β―οΈ
Try the demo for best setup here https://huggingface.co./spaces/Apollo-LMMs/Apollo-3B
they evaluate sampling strategies, scaling laws for models and datasets, video representation and more!
> The authors find out that whatever design decision was applied to small models also scale properly when the model and dataset are scaled π scaling dataset has diminishing returns for smaller models
> They evaluate frame sampling strategies, and find that FPS sampling is better than uniform sampling, and they find 8-32 tokens per frame optimal
> They also compare image encoders, they try a variation of models from shape optimized SigLIP to DINOv2
they find google/siglip-so400m-patch14-384 to be most powerful π₯
> they also compare freezing different parts of models, training all stages with some frozen parts give the best yield
They eventually release three models, where Apollo-3B outperforms most 7B models and Apollo 7B outperforms 30B models π₯
- 3 replies
upvoted
a
paper
8 days ago
posted
an
update
13 days ago
Post
1669
A complete RAG pipeline includes a reranker, which ranks the documents to find the best document π
Same goes for multimodal RAG, multimodal rerankers which we can integrate to multimodal RAG pipelines!
Learn how to build a complete multimodal RAG pipeline with vidore/colqwen2-v1.0 as retriever, lightonai/MonoQwen2-VL-v0.1 as reranker, Qwen/Qwen2-VL-7B-Instruct as VLM in this notebook that runs on a GPU as small as L4 π₯ https://huggingface.co./learn/cookbook/multimodal_rag_using_document_retrieval_and_reranker_and_vlms
Same goes for multimodal RAG, multimodal rerankers which we can integrate to multimodal RAG pipelines!
Learn how to build a complete multimodal RAG pipeline with vidore/colqwen2-v1.0 as retriever, lightonai/MonoQwen2-VL-v0.1 as reranker, Qwen/Qwen2-VL-7B-Instruct as VLM in this notebook that runs on a GPU as small as L4 π₯ https://huggingface.co./learn/cookbook/multimodal_rag_using_document_retrieval_and_reranker_and_vlms
Fix streamlit warning
#3 opened 16 days ago
by
lbourdois
updated
a
collection
16 days ago
updated
a
Space
16 days ago