IndexError: index 0 is out of bounds for dimension 0 with size 0
When running Qwen2-VL-2B-Instruct, i meet this problem:
Traceback (most recent call last):
File "Qwen_temp/Qwen2-VL-2B.py", line 54, in
generated_ids = model.generate(**inputs, max_new_tokens=128)
File "/root/anaconda3/envs/Qwen/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "Qwen2-VL/transformers-21fac7abba2a37fae86106f87fcf9974fd1e3830/src/transformers/generation/utils.py", line 2015, in generate
result = self._sample(
File "Qwen2-VL/transformers-21fac7abba2a37fae86106f87fcf9974fd1e3830/src/transformers/generation/utils.py", line 2960, in _sample
model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs)
File "Qwen2-VL/transformers-21fac7abba2a37fae86106f87fcf9974fd1e3830/src/transformers/models/qwen2_vl/modeling_qwen2_vl.py", line 1676, in prepare_inputs_for_generation
if cache_position is None or (cache_position is not None and cache_position[0] == 0):
IndexError: index 0 is out of bounds for dimension 0 with size 0
Is there any solution? Thanks.
I also occur this error. When i used the qwen2-vl-instruct to sft model, then i used the model inference is right, but i used the qwen2-vl-base to sft model, then i used the model inference is wrong.
Did you solve this problem？
I encountered the same issue. Root cause is that Qwen2-VL and Qwen2-VL-Instruct's chat_template is different, causing the input_ids to be empty before inference.
So the solution is to replace Qwen2-VL's chat_template.json with Instruct's chat_template.json. It fixes my problem.