prova / README.md
riccardo45's picture
ss
d3b904b verified
metadata
license: apache-2.0
language:
  - it
base_model: HuggingFaceH4/zephyr-7b-beta
new_version: l3utterfly/mistral-7b-v0.1-layla-v4
datasets:
  - imdb
  - HuggingFaceH4/no_robots
model-index:
  - name: StarCoder
    results:
      - task:
          type: text-generation
        dataset:
          type: openai_humaneval
          name: HumanEval (Prompted)
        metrics:
          - name: pass@1
            type: pass@1
            value: 0.408
            verified: false
      - task:
          type: text-generation
        dataset:
          type: openai_humaneval
          name: HumanEval
        metrics:
          - name: pass@1
            type: pass@1
            value: 0.336
            verified: false

Model features

Grounded Generation with Jamba:

A common use-case for LLMs is grounded generation and RAG, where the model is required to answer a question or follow an instruction based on a given set of documents or document snippets. To standardize this process, Jamba was trained with a specific "documents" section in its chat template. The model was trained to attend to this section, and grounded generation tasks show improved performance when the task is formatted in this way.

Similar to tools, which are given as an external argument to the model in addition to the conversation, documents are provided in a similar way. To support document-level metadata, a document is defined as a dictionary with key-values of your choosing. These are formatted within the chat template. Two keys that get special treatment are "title", which is formatted at the top of the document if present, and "text" which is a required field and defines the actual text of the document.

Ataching documents to Jamba 1.5 prompt
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("ai21labs/AI21-Jamba-1.5-Large")
messages = [
        {
            "role": "user",
            "content": "Who wrote Harry Potter?"
        }
]
documents = [
        {
            "text": "Harry Potter is a series of seven fantasy novels written by British author J. K. Rowling.",
            "title": "Harry Potter"
        },
        {
            "text": "The Great Gatsby is a novel by American writer F. Scott Fitzgerald.",
            "title": "The Great Gatsby",
            "country": "United States",
            "genre": "Novel"
        }
]
prompt = tokenizer.apply_chat_template(
    messages,
    documents=documents,
    tokenize=False,
)
# Output: J. K. Rowling