! Important Note !
The following is the full RWKV-5 Eagle 7B model weights, which can be used with our various inference libraries Download link here
For HF compatible implementation, refer to here
This is not an instruct tune model! (soon...)
Eagle 7B - in short
Eagle 7B is a 7.52B parameter model that:
- Built on the RWKV-v5 architecture (a linear transformer with 10-100x+ lower inference cost)
- Ranks as the world’s greenest 7B model (per token)
- Trained on 1.1 Trillion Tokens across 100+ languages (70% English, 15% multi lang, 15% code)
- Outperforms all 7B class models in multi-lingual benchmarks
- Approaches Falcon (1.5T), LLaMA2 (2T), Mistral (>2T?) level of performance in English evals
- Trade blows with MPT-7B (1T) in English evals
- All while being an “Attention-Free Transformer”
- Is a foundation model, with a very small instruct tune - further fine-tuning is required for various use cases!
Find out more at our model announcement: https://blog.rwkv.com/p/eagle-7b-soaring-past-transformers
Or our wiki: https://wiki.rwkv.com