metadata
license: llama3.1
datasets:
- OpenCoder-LLM/opc-sft-stage1
- OpenCoder-LLM/opc-sft-stage2
- microsoft/orca-agentinstruct-1M-v1
- microsoft/orca-math-word-problems-200k
- NousResearch/hermes-function-calling-v1
- AI-MO/NuminaMath-CoT
- AI-MO/NuminaMath-TIR
- allenai/tulu-3-sft-mixture
- cognitivecomputations/dolphin-coder
- HuggingFaceTB/smoltalk
- cognitivecomputations/samantha-data
- m-a-p/CodeFeedback-Filtered-Instruction
- m-a-p/Code-Feedback
language:
- en
base_model:
- meta-llama/Llama-3.1-8B
Dolphin 3.0 Llama 3.1 8B 🐬
Curated and trained by Eric Hartford, Ben Gitter and Cognitive Computations
Our appreciation for the generous sponsors of Dolphin 3.0:
- Crusoe Cloud - provided 16x L40s for training and evals
- Akash - provided on-demand 8x H100 for training
- Lazarus - provided 16x H100 for training
- Cerebras - provided excellent and fast inference services
- Andreessen Horowitz - provided a grant that make Dolphin 1.0 possible and enabled me to bootstrap my homelab
Appreciation to the creators of the open source datasets that were used:
- OpenCoder-LLM (opc-sft-stage1, opc-sft-stage2)
- microsoft (orca-agentinstruct-1M-v1, orca-math-word-problems-200k)
- NousResearch (hermes-function-calling-v1)
- AI-MO (NuminaMath-CoT, NuminaMath-TIR)
- allenai (tulu-3-sft-mixture)
- HuggingFaceTB (smoltalk)
- m-a-p (CodeFeedback-Filtered-Instruction, Code-Feedback)
Special thanks to
- Meta, Qwen, and OpenCoder, who wrote papers that were instrumental in creating this.
- RLHFlow for the excellent reward model used to filter the datasets