ehristoforu/Falcon3-MoE-2x7B-Insruct

Falcon3 MoE 2x7B Instruct

Mixture of Experts of Falcon3 7B-IT & 7B-IT.

• 13.4B parameters
• BF16
• Falcon3 (Llama)
• Instruct

Falcon3-7B-Instruct

Falcon3 family of Open Foundation Models is a set of pretrained and instruct LLMs ranging from 1B to 10B.

This repository contains the Falcon3-7B-Instruct. It achieves state of art results (at the time of release) on reasoning, language understanding, instruction following, code and mathematics tasks. Falcon3-7B-Instruct supports 4 languages (english, french, spanish, portuguese) and a context length up to 32K.