meta-llama

/

Llama-3.2-1B-Instruct

like 661

Follow

Meta Llama 15.5k

I want to use Llama-3.2-1B-Instruct model, and although I have set "temperature": 0.0, "top_p":0.0 and "top_k":0, it still generates inconsistent output. This is how my pipeline looks like:

pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.bfloat16,
    device_map="mps",
        model_kwargs={"temperature": 0.0,
                  "do_sample":True,
                              "top_p":0.0,
                              "top_k":0,},
)

Any idea how to solve this issue?