anon8231489123

/

gpt4-x-alpaca-13b-native-4bit-128g

like 734

i've clone this repository and run with llama_inference.py in GPTQ-for-LLaMa.

CUDA_VISIBLE_DEVICES=1 python llama_inference.py /home/user/GPTQ-for-LLaMa/model/gpt4-x-alpaca-13b-native-4bit-128g --wbits 4 --groupsize 128 --load /home/user/GPTQ-for-LLaMa/model/gpt4-x-alpaca-13b-native-4bit-128g/gpt-x-alpaca-13b-native-4bit-128g.pt --text "this is llama"

i tried both of triton branch and cuda branch.
but i got same error below.

Loading model ...
Done.
Traceback (most recent call last):
  File "/home/user/GPTQ-for-LLaMa/llama_inference.py", line 125, in <module>
    tokenizer = AutoTokenizer.from_pretrained(args.model)
  File "/home/user/anaconda3/envs/gptq/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 700, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/home/user/anaconda3/envs/gptq/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1811, in from_pretrained
    return cls._from_pretrained(
  File "/home/user/anaconda3/envs/gptq/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1965, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/home/user/anaconda3/envs/gptq/lib/python3.9/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 74, in __init__
    super().__init__(
  File "/home/user/anaconda3/envs/gptq/lib/python3.9/site-packages/transformers/tokenization_utils_fast.py", line 120, in __init__
    raise ValueError(
ValueError: Couldn't instantiate the backend tokenizer from one of: 
(1) a `tokenizers` library serialization file, 
(2) a slow tokenizer instance to convert or 
(3) an equivalent slow tokenizer class to instantiate and convert. 
You need to have sentencepiece installed to convert a slow tokenizer to a fast one.

i try to find the solution.
They said "pip install sentencepiece" but i already install sentencepiece.
so i don't know what is the reason of this error.
could you give the hint for me?