Llama-3.1-Spellbound-StoryWriter-8b-base-0.2-16-bit-RK3588-1.1.2

This version of Llama-3.1-Spellbound-StoryWriter-8b-base-0.2-16-bit has been converted to run on the RK3588 NPU using w8a8 quantization.

This model has been optimized with the following LoRA:

Compatible with RKLLM version: 1.1.2

Useful links:

Official RKLLM GitHub

RockhipNPU Reddit

EZRKNN-LLM

Pretty much anything by these folks: marty1885 and happyme531

Converted using https://github.com/heathershaw821/ez-er-rkllm-toolkit2

Original Model Card for base model, Llama-3.1-Spellbound-StoryWriter-8b-base-0.2-16-bit, below:

Uploaded model

• Developed by: hf-100
• License: apache-2.0
• Finetuned from model : unsloth/Meta-Llama-3.1-8B-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.