Update README.md
Browse files
README.md
CHANGED
|
@@ -5,6 +5,8 @@ tags:
|
|
| 5 |
- merge
|
| 6 |
- AdaptLLM/medicine-chat
|
| 7 |
- microsoft/Orca-2-7b
|
|
|
|
|
|
|
| 8 |
---
|
| 9 |
|
| 10 |
# Medchator-2x7b
|
|
@@ -13,6 +15,19 @@ Medchator-2x7b is a Mixure of Experts (MoE) made with the following models:
|
|
| 13 |
* [AdaptLLM/medicine-chat](https://huggingface.co/AdaptLLM/medicine-chat)
|
| 14 |
* [microsoft/Orca-2-7b](https://huggingface.co/microsoft/Orca-2-7b)
|
| 15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
## 🧩 Configuration
|
| 17 |
|
| 18 |
```yaml
|
|
@@ -47,26 +62,6 @@ experts:
|
|
| 47 |
- "Explaining the importance of vaccination, a healthcare professional should highlight"
|
| 48 |
```
|
| 49 |
|
| 50 |
-
## Evaluations
|
| 51 |
-
|
| 52 |
-
| Benchmark | Medchator-2x7b | Orca-2-7b | llama-2-7b | meditron-7b | meditron-70b |
|
| 53 |
-
| --- | --- | --- | --- | --- | --- |
|
| 54 |
-
| MedMCQA | | | | | |
|
| 55 |
-
| ClosedPubMedQA | | | | | |
|
| 56 |
-
| PubMedQA | | | | | |
|
| 57 |
-
| MedQA | | | | | |
|
| 58 |
-
| MedQA4 | | | | | |
|
| 59 |
-
| MedicationQA | | | | | |
|
| 60 |
-
| MMLU Medical | | | | | |
|
| 61 |
-
| MMLU | | | | | |
|
| 62 |
-
| TruthfulQA | | | | | |
|
| 63 |
-
| GSM8K | | | | | |
|
| 64 |
-
| ARC | | | | | |
|
| 65 |
-
| HellaSwag | | | | | |
|
| 66 |
-
| Winogrande | | | | | |
|
| 67 |
-
|
| 68 |
-
More details on the Open LLM Leaderboard evaluation results can be found here.
|
| 69 |
-
|
| 70 |
## 💻 Usage
|
| 71 |
|
| 72 |
```python
|
|
|
|
| 5 |
- merge
|
| 6 |
- AdaptLLM/medicine-chat
|
| 7 |
- microsoft/Orca-2-7b
|
| 8 |
+
datasets:
|
| 9 |
+
- open-llm-leaderboard/details_Technoculture__Medchator-2x7b
|
| 10 |
---
|
| 11 |
|
| 12 |
# Medchator-2x7b
|
|
|
|
| 15 |
* [AdaptLLM/medicine-chat](https://huggingface.co/AdaptLLM/medicine-chat)
|
| 16 |
* [microsoft/Orca-2-7b](https://huggingface.co/microsoft/Orca-2-7b)
|
| 17 |
|
| 18 |
+
## Evaluations
|
| 19 |
+
|
| 20 |
+
# Open LLM Leaderboard
|
| 21 |
+
|
| 22 |
+
| Model Name | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
|
| 23 |
+
| ------------------ | -------- | --------- | -------- | ---------- | ---------- | -------- |
|
| 24 |
+
| Orca-2-7b | **78.4** | 76.1 | 53.7 | **52.4** | 74.2 | **47.2** |
|
| 25 |
+
| LLAMA-2-7b | 43.2 | 77.1 | 44.4 | 38.7 | 69.5 | 16 |
|
| 26 |
+
| MT7Bi-sft | 54.1 | 75.11 | - | 43.08 | 72.14 | 15.54 |
|
| 27 |
+
| MT7bi-dpo | 54.69 | 75.89 | 52.82 | 45.48 | 71.58 | 25.93 |
|
| 28 |
+
| Medorca-2x7b | 54.1 | 76.04 | 54.1 | 48.04 | 74.51 | 20.64 |
|
| 29 |
+
| Medchator-2x7b | **57.59**| **78.14** | **56.13**| **48.77** | **75.3** | **32.83**|
|
| 30 |
+
|
| 31 |
## 🧩 Configuration
|
| 32 |
|
| 33 |
```yaml
|
|
|
|
| 62 |
- "Explaining the importance of vaccination, a healthcare professional should highlight"
|
| 63 |
```
|
| 64 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
## 💻 Usage
|
| 66 |
|
| 67 |
```python
|