How to use the ONNX format of BERT-based model bge-m3, how to load the model and output sentence embeddings.
#52
by
panjiayi
- opened
How to use the ONNX format of BERT-based model bge-m3, how to load the model and output sentence embeddings. Please tell me. Thank you!
You can refer to this discussion: https://huggingface.co./BAAI/bge-m3/discussions/50
onnx use cls pooling？
is this right？
···
model_ort = ORTModelForFeatureExtraction.from_pretrained(os.path.join(model_path, "onnx"), export=False)
def encode(text):
encoded_input = tokenizer(text, padding=True, truncation=True, return_tensors='pt')
model_output_ort = model_ort(**encoded_input)
return model_output_ort['last_hidden_state'][0][0, :]
···