README.md

---
license: gemma
base_model: SicariusSicariiStuff/2B_or_not_2B
tags:
- llama-cpp
- gguf-my-repo
---

# Triangle104/2B_or_not_2B-Q5_K_S-GGUF
This model was converted to GGUF format from [`SicariusSicariiStuff/2B_or_not_2B`](https://huggingface.co./SicariusSicariiStuff/2B_or_not_2B) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co./spaces/ggml-org/gguf-my-repo) space.
Refer to the [original model card](https://huggingface.co./SicariusSicariiStuff/2B_or_not_2B) for more details on the model.

---
The model's name is fully credited to invisietch and Shakespeare; without them, this model would not have existed.




Regarding the question, I am happy to announce that it is, in fact, 2B, as it is so stated on the original Google model card, which this model was finetuned on.


If there's one thing we can count on, it is Google to tell us what is
 true, and what is misinformation. You should always trust and listen to
 your elders, and especially to your big brother.


This model was finetuned on a whimsical whim, on my poor laptop. It's
 not really poor, the GPU is 4090 16GB, but... it is driver-locked to 
80watts because nVidia probably does not have the resources to make 
better drivers for Linux.
I hope nVidia will manage to recover, as I have seen poor Jensen with 
the same old black leather jacket for years upon years. The stock is 
down like 22% already in this month (August 11th, 2024).


Finetuning took about 4 hours, while the laptop was on my lap, and 
while I was talking about books and stuff on Discord. Luckily, the 
laptop wasn't too hot, as 80 watts is not the 175w I was promised, which
 would have surely been hot enough to make an Omelette. Always remain an
 optimist fellas!

---
## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp

```
Invoke the llama.cpp server or the CLI.

### CLI:
```bash
llama-cli --hf-repo Triangle104/2B_or_not_2B-Q5_K_S-GGUF --hf-file 2b_or_not_2b-q5_k_s.gguf -p "The meaning to life and the universe is"
```

### Server:
```bash
llama-server --hf-repo Triangle104/2B_or_not_2B-Q5_K_S-GGUF --hf-file 2b_or_not_2b-q5_k_s.gguf -c 2048
```

Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.
```
git clone https://github.com/ggerganov/llama.cpp
```

Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
```
cd llama.cpp && LLAMA_CURL=1 make
```

Step 3: Run inference through the main binary.
```
./llama-cli --hf-repo Triangle104/2B_or_not_2B-Q5_K_S-GGUF --hf-file 2b_or_not_2b-q5_k_s.gguf -p "The meaning to life and the universe is"
```
or 
```
./llama-server --hf-repo Triangle104/2B_or_not_2B-Q5_K_S-GGUF --hf-file 2b_or_not_2b-q5_k_s.gguf -c 2048
```