Qwen/Qwen2-VL-7B-Instruct-GPTQ-Int8 · 使用时出现“probability tensor contains either `inf`, `nan` or element

‘Qwen2-VL-7B-Instruct’ 和 ‘Qwen2-VL-7B-Instruct-GPTQ-Int4’ 可以正常运行，但是使用 ‘Qwen2-VL-7B-Instruct-GPTQ-Int8’的时候，出现以下错误。

Traceback (most recent call last):
  File "/data/jxzhang/miniconda3/envs/qwen2/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
    self.run()
  File "/data/jxzhang/miniconda3/envs/qwen2/lib/python3.12/threading.py", line 1012, in run
    self._target(*self._args, **self._kwargs)
  File "/data/jxzhang/miniconda3/envs/qwen2/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/data/jxzhang/miniconda3/envs/qwen2/lib/python3.12/site-packages/transformers/generation/utils.py", line 2254, in generate
    result = self._sample(
             ^^^^^^^^^^^^^
  File "/data/jxzhang/miniconda3/envs/qwen2/lib/python3.12/site-packages/transformers/generation/utils.py", line 3299, in _sample
    next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

Qwen
/

Qwen2-VL-7B-Instruct-GPTQ-Int8

使用时出现“probability tensor contains either `inf`, `nan` or element < 0” 错误