Finally, a Replacement for BERT: Introducing ModernBERT
•
566
LightOn AI
company
Verified
AI & ML interests
None defined yet.
Recent Activity
View all activity
Articles
Organization Card
Rethink AI for the Enterprise
LightOn Paradigm is the platform for your Generative AI team: 🚀 easy to deploy and serve 🎨 customizable for specific use cases and applications 🔒 respecting the privacy of your data and protecting your intellectual property and know-how
Learn more at lighton.ai 💡 Get in touch if you want the help of a team of experts in Generative AI.
About LightOn
The team at LightOn has been focusing on building and commercializing state-of-the-art Generative AI Foundation Models for its customers since 2020. LightOn has released state-of-the-art language models for French, and for protein modeling, it also participated in the BLOOM project that resulted in the open-source release of a 179B parameters multilingual model.
Collections
5
Bringing BERT into modernity via both architecture changes and scaling
-
answerdotai/ModernBERT-base
Fill-Mask • Updated • 6.09M • 784 -
answerdotai/ModernBERT-large
Fill-Mask • Updated • 273k • 360 -
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
Paper • 2412.13663 • Published • 134 -
lightonai/modernbert-embed-large
Sentence Similarity • Updated • 2.29k • 20
French language models. These model were trained in early 2021 following the then scaling laws and using the exact same training data as the CamemBERT
models
23
lightonai/pdfs_rag_eval
Updated
lightonai/large-3-shot-seqlen-3x-max-lt-8k-all-better-hparam-instruct-100k-mcqa-heldout-zs-unused-tokens
Updated
•
13
lightonai/colbertv2.0
Sentence Similarity
•
Updated
•
666
•
4
lightonai/modernbert-embed-large
Sentence Similarity
•
Updated
•
2.29k
•
20
lightonai/modernbert-embed-large-unsupervised
Sentence Similarity
•
Updated
•
209
lightonai/MonoQwen2-VL-v0.1
Updated
•
1.35k
•
31
lightonai/pagnol-medium
Text Generation
•
Updated
•
58
•
1
lightonai/RITA_m
Text Generation
•
Updated
•
46
lightonai/FlexBert
Updated
lightonai/RITA_xl
Text Generation
•
Updated
•
925
•
3
datasets
6
lightonai/fc-amf-ocr
Viewer
•
Updated
•
58.6k
•
986
•
17
lightonai/ArabicWeb24
Viewer
•
Updated
•
86.8M
•
1.66k
•
17
lightonai/ms-marco-en-bge
Viewer
•
Updated
•
10.5M
•
276
•
6
lightonai/ms-marco-fr-bge
Viewer
•
Updated
•
9.64M
•
95
lightonai/lighton-ms-marco-mini
Viewer
•
Updated
•
100
•
419
lightonai/SwissProt-EC-leaf
Viewer
•
Updated
•
223k
•
249