---
base_model: microsoft/phi-4
library_name: transformers
license: other
tags:
- phi-4-GGUF
- phi-4
- llama-cpp
- GGUF
- phi
- nlp
- math
- code
- chat
- conversational

pipeline_tag: text-generation
inference:
  parameters:
    temperature: 0
widget:
- messages:
  - role: user
    content: How should I explain the Internet?
---


# Sri-Vigneshwar-DJ/phi-4-GGUF
This model was converted to GGUF format from [`Sri-Vigneshwar-DJ/phi-4-GGUF`](https://huggingface.co./microsoft/phi-4) using llama.cpp
Refer to the [original model card](https://huggingface.co./microsoft/phi-4) for more details on the model.

## Model Summary 

|                         |                                                                               |     
|-------------------------|-------------------------------------------------------------------------------|
| **Developers**          | Microsoft Research                                                            |
| **Description**         | `phi-4` is a state-of-the-art open model built upon a blend of synthetic datasets, data from filtered public domain websites, and acquired academic books and Q&A datasets. The goal of this approach was to ensure that small capable models were trained with data focused on high quality and advanced reasoning.<br><br>`phi-4` underwent a rigorous enhancement and alignment process, incorporating both supervised fine-tuning and direct preference optimization to ensure precise instruction adherence and robust safety measures                |
| **Architecture**        | 14B parameters, dense decoder-only Transformer model                          |
| **Inputs**              | Text, best suited for prompts in the chat format                              |
| **Context length**      | 16K tokens                                                                    |
| **GPUs**                | 1920 H100-80G                                                                 |
| **Training time**       | 21 days                                                                       |
| **Training data**       | 9.8T tokens                                                                   |
| **Outputs**             | Generated text in response to input                                           |
| **Dates**               | October 2024 – November 2024                                                  |
| **Status**              | Static model trained on an offline dataset with cutoff dates of June 2024 and earlier for publicly available data                                                                               |
| **Release date**        | December 12, 2024                                                             |
| **License**             | MIT                                                                         |

## Intended Use 

|                               |                                                                         |
|-------------------------------|-------------------------------------------------------------------------|
| **Primary Use Cases**         | Our model is designed to accelerate research on language models, for use as a building block for generative AI powered features. It provides uses for general purpose AI systems and applications (primarily in English) which require:<br><br>1. Memory/compute constrained environments.<br>2. Latency bound scenarios.<br>3. Reasoning and logic.                                                                       |
| **Out-of-Scope Use Cases**    | Our models is not specifically designed or evaluated for all downstream purposes, thus:<br><br>1. Developers should consider common limitations of language models as they select use cases, and evaluate and mitigate for accuracy, safety, and fairness before using within a specific downstream use case, particularly for high-risk scenarios.<br>2. Developers should be aware of and adhere to applicable laws or regulations (including privacy, trade compliance laws, etc.) that are relevant to their use case, including the model’s focus on English.<br>3. Nothing contained in this Model Card should be interpreted as or deemed a restriction or modification to the license the model is released under.                                                              |



## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux) from []

```bash
brew install llama.cpp or !git clone https://github.com/ggerganov/llama.cpp.git

```
Invoke the llama.cpp server or the CLI.

### CLI:
```bash
! /content/llama.cpp/llama-cli -m ./models/phi-4 -n 90 --repeat_penalty 1.0 --color -i -r "User:" -f /content/llama.cpp/prompts/chat-with-bob.txt

or

llama-cli --hf-repo Sri-Vigneshwar-DJ/phi-4-GGUF --hf-file FP8.gguf -p "The meaning to life and the universe is"
```

### Server:
```bash
llama-server --hf-repo Sri-Vigneshwar-DJ/sarvam-2b-v0.5-GGUF --hf-file FP8.gguf -c 2048
```

Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.
```
git clone https://github.com/ggerganov/llama.cpp
```

Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag or ''!make GGML_OPENBLAS=1' along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
```
cd llama.cpp && LLAMA_CURL=1 make

or

!make GGML_OPENBLAS=1
```

Step 3: Run inference through the main binary.
```
./llama-cli --hf-repo Sri-Vigneshwar-DJ/phi-4-GGUF --hf-file FP8.gguf -p "The meaning to life and the universe is"
```
or 
```
./llama-server --hf-repo Sri-Vigneshwar-DJ/phi-4-GGUF --hf-file sFP8.gguf -c 2048
```