HuggingFaceTB

/

SmolVLM-Instruct

like 299

Follow

Hugging Face TB Research 756

Trying to run the models under the ONNX folder on a Macbook M1 to test out the model when its quantized. Im using the files in the ONNX folder as follows:

import onnxruntime as ort
from optimum.onnxruntime import ORTModelForVision2Seq
from transformers import AutoProcessor
import torch

# Initialize processor and model
processor = AutoProcessor.from_pretrained("HuggingFaceTB/SmolVLM-Instruct")
model = ORTModelForVision2Seq.from_pretrained(
    "HuggingFaceTB/SmolVLM-Instruct",
    subfolder="onnx",
    encoder_file_name="vision_encoder_quantized.onnx",  # Using int8 variant
    decoder_file_name="decoder_model_merged_int8.onnx",  # Using int8 variant
    provider="CPUExecutionProvider"  # Starting with CPU to test compatibility
)

I get an error message

AttributeError: 'Idefics3Config' object has no attribute 'decoder'

Im currently running the following:

optimum==1.23.3
transformers== 4.47.0
onnx=1.17.0
onnxruntime==1.20.1
onnxruntime-genai=0.5.2