glaive-function-calling-v1

glaive-function-calling-v1 is a 2.7B parameter open source chat model trained on data generated from Glaive’s synthetic data generation platform, which has similar function calling abilities as gpt-3.5 and gpt 4.

The model is capable of having multi-turn conversations and intelligently choosing when to execute a function (provided at the beginning of the conversation as a system prompt) based on the conversation. The model is trained on top of the https://huggingface.co./replit/replit-code-v1-3b model.

Usage:

You can run the model in the following way-

from transformers import AutoModelForCausalLM , AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("glaiveai/glaive-function-calling-v1", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("glaiveai/glaive-function-calling-v1", trust_remote_code=True).half().cuda()

inputs = tokenizer(prompt,return_tensors="pt").to(model.device)

outputs = model.generate(**inputs,do_sample=True,temperature=0.1,top_p=0.95,max_new_tokens=100)

print(tokenizer.decode(outputs[0],skip_special_tokens=True))

This model uses the following prompt format-

SYSTEM: You are an helpful assistant who has access to the following functions to help the user, you can use the functions if needed-
{
            "name": "plan_holiday",
            "description": "Plan a holiday based on user's interests",
            "parameters": {
                "type": "object",
                "properties": {
                    "destination": {
                        "type": "string",
                        "description": "The destination of the holiday",
                    },
                    "duration": {
                        "type": "integer",
                        "description": "The duration of the trip in holiday",
                    },
                },
                "required": ["destination", "duration"],
            },
}
USER: I am thinking of having a 10 day long vacation in Greece, can you help me plan it?

Based on which the model outputs-

ASSISTANT: <functioncall> {"name": "plan_holiday", "arguments": '{
  "destination": "Greece",
  "duration": 10
}'}

The model precedes all function invocations with <functioncall>.

The response of the function call should be sent to the model as-

FUNCTION CALL: {"places_to_visit":["Athens","Santorini","Mykonos"]}

The model can do multi-turn conversation in the above format.

We're working on providing an inference server which can act as a drop in replacement to the OpenAI API, you can follow this repo for the server.

Known Limitations:

• While the model does well on function calling use-cases, it doesn't always generalize very well to other chat use-cases. This is intentional as our thesis at Glaive is to provide use-case specialised model that are only used for the given task.
• The model may sometimes hallucinate functions, v2 of the model will be aimed to fix that with a bigger dataset.