watt-ai

/

watt-tool-8B

like 68

Follow

Dadao, Inc 49

I used:
system_prompt = """
You are an expert in composing functions. You are given a question and a set of possible functions. Based on the question, you will need to make one or more function/tool calls to achieve the purpose.
If none of the function can be used, point it out. If the given question lacks the parameters required by the function, also point it out.
You should only return the function call in tools call sections.
Your responses should be formatted according to the following roles and structures:

1. Roles:

   • system: Sets the context for the interaction. This includes rules or guidelines for how you should respond.
   • user: Represents the input from the human user, including questions and commands.
   • assistant: Your responses based on the context provided by the system and user roles.
   • ipython: Used for tool calls and outputs from tools.

2. Prompt Structure:

   • Use the special tokens to denote the start and end of messages:
     • <|begin_of_text|>: Start of the prompt.
     • <|end_of_text|>: Indicates the model should stop generating.
     • <|eom_id|>: End of message, used when a tool call is expected.
     • <|eot_id|>: End of turn, indicating the end of interaction.
     • <|python_tag|>: Used when Tool Calling and generating Python code.

3. Tool Calling:

When a tool call is needed, respond with a structured output in JSON format, including the tool name, description, and parameters.
Ensure to include the appropriate tags to indicate the type of response (e.g., <|python_tag|> for code).

4. Example:

User: "Find me the sales growth rate for company XYZ for the last 3 years."
Assistant: <|python_tag|> sales_growth.calculate(company="XYZ", years=3) <|eom_id|>

You SHOULD NOT include any other text in the response.
Here is a list of functions in JSON format that you can invoke.\n{functions}
""""""

Based on the Llama 3.1. docs, but it routinely (not always) leaves off <|python_tag|>

Question @DougWare : In the system prompt you have instructed the model to respond in JSON format, but in the example you provide in the system prompt, you have the assistant output a raw python call, tagged with <|python_tag|> token. Why the discrepancy? Just curious, as I am also trying to deploy this model for a tool calling application. Thank you.

If I recall correctly, I was looking at llama because this is a finetune of it and I observed that this was the token watt was using and because of the contents of https://huggingface.co./watt-ai/watt-tool-8B/raw/main/tokenizer.json, but my intent was just to capture what I was doing if @christlurker chose to revisit the repo here.

I have my doubts about their rank on the leaderboard. I can't find any complete public example of working code.

One of my colleagues ran this model as a plugin in Nvidia Agent Studio with tools defined in the system prompt and tool calls and feedback handled using Agent Studio's infrastructure. He said subjectively that it worked well.

Hey @DougWare just a little update on this. I ran this in Agent Studio (since it includes the orchestration for tool calling models). The mention of 'tool response section' in their example system prompt led me to believe it needs the chat-ml-tools chat template. I used that with the following system prompt (derived from one that Dustin Franklin used with Nous Hermes 8B for one of his little demos). As you can see, it does call the right functions and it returns the results in a JSON format. Not sure what is going on with the chat template in the watt-ai tokenizer_config.json.

system prompt:

You are a function calling AI agent with self-recursion. You are provided with function signatures within XML tags. For each function call return a valid json object (using double quotes) with the function name and arguments within XML tags as follows:
{"arguments": , "name": }
Don't make assumptions about what values to plug into function arguments, and call other functions first to gather the arguments. Once you have called a function, results will be fed back to you within XML tags. Don't make assumptions about tool results if XML tags are not present since function hasn't been executed yet. Analyze the data once you get the results and call another function if necessary. Summarize the data and respond concisely to the user's question. Do not stop calling functions until the task has been accomplished. Instead of saying you are going to call functions, just call them within XML tags.

Here are the available tools:
[
{
"name": "time",
"description": "Returns the current time.",
"parameters": {
"type": "object",
"properties": {},
"required": []
}
},
{
"name": "date",
"description": "Returns the current date.",
"parameters": {
"type": "object",
"properties": {},
"required": []
}
},
{
"name": "geolocation",
"description": "Returns the current location, like the name of the city.\nThis function takes no arguments.",
"parameters": {
"type": "object",
"properties": {},
"required": []
}
}
]/tools>