Upload README.md
Browse files
README.md
CHANGED
|
@@ -21,7 +21,7 @@ This quant was made using exllamav2-0.0.20 with default dataset and settings.
|
|
| 21 |
|
| 22 |
This quant fits 25k context on 24GB VRAM on Windows in my local testing (with exl2 Q4 cache), you might be able to get more depending on other things taking VRAM.
|
| 23 |
|
| 24 |
-
I tested this quant shortly in some random RPs (including
|
| 25 |
|
| 26 |
## Prompt Templates
|
| 27 |
|
|
|
|
| 21 |
|
| 22 |
This quant fits 25k context on 24GB VRAM on Windows in my local testing (with exl2 Q4 cache), you might be able to get more depending on other things taking VRAM.
|
| 23 |
|
| 24 |
+
I tested this quant shortly in some random RPs (including ones over 8k and 20k context) and it seems to work fine.
|
| 25 |
|
| 26 |
## Prompt Templates
|
| 27 |
|