Exllamav2 quant (exl2 / 8.0 bpw) made with ExLlamaV2 v0.1.3

Other EXL2 quants:

Quant Model Size lm_head
2.2
3250 MB
6
2.5
3478 MB
6
3.0
3894 MB
6
3.5
4311 MB
6
3.75
4518 MB
6
4.0
4727 MB
6
4.25
4935 MB
6
5.0
5559 MB
6
6.0
6489 MB
8
6.5
6909 MB
8
8.0
8123 MB
8

image/png

"Poppy Porpoise" is a cutting-edge AI roleplay assistant based on the Llama 3 8B model, specializing in crafting unforgettable narrative experiences. With its advanced language capabilities, Poppy expertly immerses users in an interactive and engaging adventure, tailoring each adventure to their individual preferences.

Note: This variant is an attempt to get something closer to 0.72 while maintaining the improvements of 1.30.

: Presets in repo folder.

If you want to use vision functionality: You must use the latest versions of Koboldcpp. And need to load the specified mmproj file: Llava MMProj.

Configuration

The following YAML configuration was used to produce this model:

slices:
  - sources:
      - model: Nitral-AI/Pp-72xra1
        layer_range: [0, 32]
      - model: Nitral-AI/Poppy-1.35-Phase1
        layer_range: [0, 32]
merge_method: slerp
base_model: Nitral-AI/Pp-72xra1
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16
Downloads last month
49
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.