---

base_model: microsoft/phi-4

---
This is a quantization of the [phi-4](https://huggingface.co./microsoft/phi-4).

The phi-4 model is a cutting-edge open-source LLM developed using a diverse mix of synthetic datasets, curated public domain web content, and acquired academic resources, including books and Q&A datasets. This deliberate data selection ensures the training of compact yet highly capable models with an emphasis on quality and advanced reasoning. To further enhance its performance, phi-4 underwent a rigorous alignment process that included supervised fine-tuning and direct preference optimization, resulting in precise instruction adherence and robust safety measures.
## Evaluations
This model provides an accuracy recovery of 99.73%. 

| __English__   | __[phi-4](https://huggingface.co./microsoft/phi-4)__   | __[phi-4-FP8-Dynamic (this)](https://huggingface.co./cortecs/phi-4-FP8-Dynamic)__   |
|:--------------|:------------------------------------------------------|:-----------------------------------------------------------------------------------|
| Avg.          | 70.75                                                 | 70.7                                                                               |
| Arc           | 68.7                                                  | 68.7                                                                               |
| Hellaswag     | 72.8                                                  | 72.7                                                                               |
| MMLU          | 79.46                                                 | 79.67                                                                              |
|               |                                                       |                                                                                    |
| __French__   | __[phi-4](https://huggingface.co./microsoft/phi-4)__   | __[phi-4-FP8-Dynamic (this)](https://huggingface.co./cortecs/phi-4-FP8-Dynamic)__   |
| Avg.         | 68.67                                                 | 68.87                                                                              |
| Arc          | 59.4                                                  | 59.5                                                                               |
| Hellaswag    | 72.0                                                  | 72.0                                                                               |
| MMLU         | 74.6                                                  | 75.1                                                                               |
|              |                                                       |                                                                                    |
| __German__   | __[phi-4](https://huggingface.co./microsoft/phi-4)__   | __[phi-4-FP8-Dynamic (this)](https://huggingface.co./cortecs/phi-4-FP8-Dynamic)__   |
| Avg.         | 68.73                                                 | 68.33                                                                              |
| Arc          | 60.2                                                  | 60.0                                                                               |
| Hellaswag    | 69.8                                                  | 69.6                                                                               |
| MMLU         | 76.2                                                  | 75.4                                                                               |
|              |                                                       |                                                                                    |
| __Italian__   | __[phi-4](https://huggingface.co./microsoft/phi-4)__   | __[phi-4-FP8-Dynamic (this)](https://huggingface.co./cortecs/phi-4-FP8-Dynamic)__   |
| Avg.          | 69.3                                                  | 69.07                                                                              |
| Arc           | 61.1                                                  | 61.3                                                                               |
| Hellaswag     | 73.1                                                  | 72.5                                                                               |
| MMLU          | 73.7                                                  | 73.4                                                                               |
|               |                                                       |                                                                                    |
| __Spanish__   |   __[phi-4](https://huggingface.co./microsoft/phi-4)__ |   __[phi-4-FP8-Dynamic (this)](https://huggingface.co./cortecs/phi-4-FP8-Dynamic)__ |
| Avg.          |                                                  70.6 |                                                                              70.03 |
| Arc           |                                                  61.6 |                                                                              61    |
| Hellaswag     |                                                  75.3 |                                                                              74.6  |
| MMLU          |                                                  74.9 |                                                                              74.5  |

We did not check for data contamination.
     Evaluation was done using [Eval. Harness](https://github.com/EleutherAI/lm-evaluation-harness) with `limit=1000`. 
    
## Usage
Install **vLLM** and 
    run the [server](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#openai-compatible-server):
    
```
python -m vllm.entrypoints.openai.api_server --model cortecs/phi-4-FP8-Dynamic
```
Access the model:
```
curl http://localhost:8000/v1/completions     -H "Content-Type: application/json"     -d ' {
        "model": "cortecs/phi-4-FP8-Dynamic",
        "prompt": "San Francisco is a"
    } '
```
⚡ This model is optimized to handle heavy workloads providing a total throughput of ️**4623 tokens per second** using one NVIDIA L40S ⚡