---
language:
- en
license: apache-2.0
tags:
- merge
- moe
model-index:
- name: mhm-8x7B-FrankenMoE-v1.0
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: AI2 Reasoning Challenge (25-Shot)
      type: ai2_arc
      config: ARC-Challenge
      split: test
      args:
        num_few_shot: 25
    metrics:
    - type: acc_norm
      value: 70.9
      name: normalized accuracy
    source:
      url: https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard?query=h2m/mhm-8x7B-FrankenMoE-v1.0
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: HellaSwag (10-Shot)
      type: hellaswag
      split: validation
      args:
        num_few_shot: 10
    metrics:
    - type: acc_norm
      value: 87.75
      name: normalized accuracy
    source:
      url: https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard?query=h2m/mhm-8x7B-FrankenMoE-v1.0
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU (5-Shot)
      type: cais/mmlu
      config: all
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 64.7
      name: accuracy
    source:
      url: https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard?query=h2m/mhm-8x7B-FrankenMoE-v1.0
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: TruthfulQA (0-shot)
      type: truthful_qa
      config: multiple_choice
      split: validation
      args:
        num_few_shot: 0
    metrics:
    - type: mc2
      value: 67.1
    source:
      url: https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard?query=h2m/mhm-8x7B-FrankenMoE-v1.0
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: Winogrande (5-shot)
      type: winogrande
      config: winogrande_xl
      split: validation
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 82.0
      name: accuracy
    source:
      url: https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard?query=h2m/mhm-8x7B-FrankenMoE-v1.0
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GSM8k (5-shot)
      type: gsm8k
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 71.57
      name: accuracy
    source:
      url: https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard?query=h2m/mhm-8x7B-FrankenMoE-v1.0
      name: Open LLM Leaderboard
---

![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6589d7e6586088fd2784a12c/dUkIQhg6CBAESsPkhtSkX.jpeg)

## Recipe for a Beautiful Frankenstein

In the laboratory of the mind, where thoughts entwine,
MHM and MOE, a potion for a unique design.
With stitches of curiosity and bolts of creativity,
8 times 7, the magic number, a poetic proclivity.

### Ingredients:

- **MHM:** A dash of mystery, a sprinkle of hum,
  Blend with a melody, let the heartstrings strum.
  Murmurs in the shadows, whispers in the light,
  Stir the concoction gently, make the emotions ignite.

- **MOE:** Essence of the moment, like dew on a rose,
  Capture the now, before time swiftly goes.
  Colors of experience, a palette so divine,
  Mix with MHM, let the fusion entwine.

### Directions:

1. **Take 8 parts MHM,** elusive and profound,
   Let it dance in your thoughts, on imagination's ground.
   Blend it with the echoes, the silent undertones,
   A symphony of ideas, where inspiration condones.

2. **Add 7 parts MOE,** the fleeting embrace,
   Seize the seconds, let them leave a trace.
   Infuse it with memories, both bitter and sweet,
   The tapestry of time, where moments and dreams meet.

3. **Stir the potion with wonder,** a wand of delight,
   Let the sparks fly, in the dark of the night.
   Watch as the alchemy unfolds its grand design,
   MHM and MOE, a beautiful Frankenstein.

### Conclusion:

In the laboratory of life, where dreams come alive,
MHM and MOE, the recipe to thrive.
A creation so poetic, a fusion so divine,
8 times 7, a symphony of time.

As the echoes resonate, and the moments blend,
A masterpiece unfolds, where beginnings and ends,
MHM and MOE, a concoction so rare,
A beautiful Frankenstein, beyond compare.

---

MoE model build with:
1. https://github.com/cg123/mergekit/tree/mixtral
2. Mistral models, latest merges and fine tunes.
3. Expert prompts heavily inspired by https://huggingface.co./Kquant03/Eukaryote-8x7B-bf16

For details check model files, there is config yaml I used to create that model.

Come back later for more details.
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co./datasets/open-llm-leaderboard/details_h2m__mhm-8x7B-FrankenMoE-v1.0)

|             Metric              |Value|
|---------------------------------|----:|
|Avg.                             |74.01|
|AI2 Reasoning Challenge (25-Shot)|70.90|
|HellaSwag (10-Shot)              |87.75|
|MMLU (5-Shot)                    |64.70|
|TruthfulQA (0-shot)              |67.10|
|Winogrande (5-shot)              |82.00|
|GSM8k (5-shot)                   |71.57|