--- language: - en - hi - bn - mr - te - ta - kn - ml - gu - as - pa license: other license_name: krutrim-community-license-agreement-version-1.0 license_link: LICENSE.md tags: - Krutrim - language-model pipeline_tag: text-generation library_name: transformers --- # Krutrim-1 [![Static Badge](https://img.shields.io/badge/Github-Krutrim_1-yellow?logo=huggingface)](https://huggingface.co./krutrim-ai-labs/Krutrim-1-instruct)[![Static Badge](https://img.shields.io/badge/Github-Krutrim_1-yellow?logo=github)](https://github.com/ola-krutrim/Krutrim-1-7B)[![Static Badge](https://img.shields.io/badge/Krutrim_Cloud-Krutrim_1-orange?logo=data:image/png%2bxml;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAADpUlEQVRYCbVXTUhbQRDeRJqi2JSS1BQtgpCa0kiPehdNi6dWbfWgF0s9eGtPFSFG8VDMpSiCB28KQou0NwsS9NA/Dx4qNP1TUIqSmlKSFjQx4vabbXbJz8vLe2kz8GX3zc7MN2/2J/sszLichekN4A7gBZxpcLQ/0gijfQq8BFLAf5ELiBIEfgNEZgSxtA/5liw2eD4EfgJGSLVsyPcBQLFMiR3WIUAraCm6F4hFMQ2JB1afgFKI9Hw+IubVYhnQwvpSBnKZ2GfEvlgoiTMYeFNGcpnEK3AQV548gkYalbslLiGWdEtl2QbOpZ9FMzg4yGprazNVpvrr6+tseXlZy+cXlFeAAzk4i07eW29sbPB/kampqbyYGTzEyagC5wHKJG+v6lWgqamJdXV1wY2xhYUFtr1NBcwWnQqQYRJwUQK3gOeArjidTkakJMfHx6y+vp4tLi6KZ5/Px1ZWVkTf5M9tstcsP/SifFarlQcCAX50dKRm4/T0lPf19ann9vZ2Xl1dzZubm3lVVZVe2XPHxDS8k2Ra7fj4uCKSnUgkwnt7e+Uj393d5ZQUSSqV4sFgMJeo0DNxsx0tYtLR2x8eHorA4XCY19TUqECZCZAB1gDf398XtvTT0dGhbAvFh37Hip9LgKbYbDZWWVkpxtbW1tjBgdo1rKGhQegTiQQbHR1lbreb9fT0qDgtLS2qr9MR3AkYFMyW3pwkGo3yzs5OPjAwwFdXV4WOfra2tpSv3W5X+snJSaXXiU/chaeAHLu7u1VQrQ6VXhJgWyqT/v5+pZfjGu0OdEx3EZJTW1sbX1pa4pgGgZmZGT40NCTIMisgDy5MC3c4HEYSEItwlkjMQi7Cvb095etyufjc3ByfmJhQuiJxiVscREYdlN3w8DA/OTnhsVhM6YqQadndpAToKNZdiLmBvV4vTyaTYgo2Nze5xWLRCl5MR0exOv5NTcPY2Jiaf2zTYkSFxkX56RwgCQBUBUNSUVEh7OicoP3e2trKpqenGf1fGBTi8ufaPoGiULZZ+sbGRh6Px9WWk52RkZEsO514j3PJ6Zlure8BQ0E8Hg+fn58X2zIUCnG/38/r6uqM+L4Fx9/jFZ1cuQzFN8BIoFJsviJ20Xm6DqN4GZKIIqYbMCQOWL0GSnlLLR+6rVBMU0I75B4QAbSCGtF9h+99QO42dM0L3ZRp1Zr9OCWfrFu2FrW8lmuN5erOQuED7gLXAPl5TjHk5/kH9J8BdBc39Hn+BxqB1clokCTRAAAAAElFTkSuQmCC)](https://cloud.olakrutrim.com/console/inference-service?section=models&modelName=krutrim&artifactName=Krutrim-1&artifactType=model)[![Static Badge](https://img.shields.io/badge/Krutrim_AI_Labs-Krutrim_1-blue?logo=data:image/svg%2bxml;base64,PHN2ZyB3aWR0aD0iMzYiIGhlaWdodD0iMzYiIHZpZXdCb3g9IjAgMCAzNiAzNiIgZmlsbD0ibm9uZSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPHJlY3Qgd2lkdGg9IjM2IiBoZWlnaHQ9IjM2IiByeD0iMTgiIGZpbGw9IiMxMEE1NTQiLz4KPHBhdGggZD0iTTI2LjQxNCAxMi41OTE5SDE5LjMzVjE1LjY0OTlDMjAuMDM0IDE1LjIzOTIgMjAuODQwNyAxNS4wMzM5IDIxLjc1IDE1LjAzMzlDMjIuNzkxMyAxNS4wMzM5IDIzLjY0MiAxNS4zNTY1IDI0LjMwMiAxNi4wMDE5QzI0Ljk3NjcgMTYuNjQ3MiAyNS4zMTQgMTcuNTQxOSAyNS4zMTQgMTguNjg1OUMyNS4zMTQgMTkuMzMxMiAyNS4xODkzIDIwLjA0OTkgMjQuOTQgMjAuODQxOUMyNC43MDUzIDIxLjYzMzkgMjQuMzE2NyAyMi40NDA1IDIzLjc3NCAyMy4yNjE5TDIxLjIgMjEuODMxOUMyMS41MzczIDIxLjM3NzIgMjEuODE2IDIwLjkwNzkgMjIuMDM2IDIwLjQyMzlDMjIuMjU2IDE5LjkzOTkgMjIuMzY2IDE5LjQ0MTIgMjIuMzY2IDE4LjkyNzlDMjIuMzY2IDE4LjM4NTIgMjIuMjQ4NyAxOC4wMDM5IDIyLjAxNCAxNy43ODM5QzIxLjc5NCAxNy41NjM5IDIxLjUwMDcgMTcuNDUzOSAyMS4xMzQgMTcuNDUzOUMyMC43OTY3IDE3LjQ1MzkgMjAuMTQ0IDE3Ljc2MTkgMjAuMTQ0IDE3Ljc2MTlDMjAuMTQ0IDE3Ljc2MTkgMTkuMTE0NyAxOC4xODcyIDE4Ljg4IDE4LjQyMTlWMjMuODU1OUgxNi4zODJWMjEuMDYxOUMxNS44OTggMjEuMzQwNSAxNS40MDY3IDIxLjU1MzIgMTQuOTA4IDIxLjY5OTlDMTQuNDI0IDIxLjg0NjUgMTMuODU5MyAyMS45MTk5IDEzLjIxNCAyMS45MTk5QzEyLjQwNzMgMjEuOTE5OSAxMS42NjY3IDIxLjc3MzIgMTAuOTkyIDIxLjQ3OTlDMTAuMzMyIDIxLjE3MTkgOS44MDQgMjAuNzI0NSA5LjQwOCAyMC4xMzc5QzkuMDEyIDE5LjU1MTIgOC44MTQgMTguODE3OSA4LjgxNCAxNy45Mzc5QzguODE0IDE3LjExNjUgOS4wMTIgMTYuNDEyNSA5LjQwOCAxNS44MjU5QzkuODA0IDE1LjIyNDUgMTAuMzU0IDE0Ljc2MjUgMTEuMDU4IDE0LjQzOTlDMTEuNzYyIDE0LjEwMjUgMTIuNTc2IDEzLjkzMzkgMTMuNSAxMy45MzM5QzEzLjkxMDcgMTMuOTMzOSAxNC4zMjEzIDEzLjk0ODUgMTQuNzMyIDEzLjk3NzlDMTUuMTU3MyAxNC4wMDcyIDE1LjQ4NzMgMTQuMDU4NSAxNS43MjIgMTQuMTMxOUwxNS41MDIgMTYuNTczOUMxNS4wMzI3IDE2LjQ1NjUgMTQuNTEyIDE2LjM5NzkgMTMuOTQgMTYuMzk3OUMxMy4yNTA3IDE2LjM5NzkgMTIuNzE1MyAxNi41MzcyIDEyLjMzNCAxNi44MTU5QzExLjk1MjcgMTcuMDc5OSAxMS43NjIgMTcuNDUzOSAxMS43NjIgMTcuOTM3OUMxMS43NjIgMTguNTI0NSAxMS45NDUzIDE4LjkyNzkgMTIuMzEyIDE5LjE0NzlDMTIuNjc4NyAxOS4zNjc5IDEzLjA3NDcgMTkuNDc3OSAxMy41IDE5LjQ3NzlDMTQuMTE2IDE5LjQ3NzkgMTQuNjU4NyAxOS4zMzg1IDE1LjEyOCAxOS4wNTk5QzE1LjYxMiAxOC43ODEyIDE2LjAzIDE4LjQ1ODUgMTYuMzgyIDE4LjA5MTlWMTIuNTkxOUg4VjEwLjE3MTlIMjYuNDE0VjEyLjU5MTlaIiBmaWxsPSJ3aGl0ZSIvPgo8cGF0aCBkPSJNMjIuMDc0IDI4Ljk4MTlDMjEuNjkyNyAyOS4xNzI1IDIxLjIzOCAyOS4zNDg1IDIwLjcxIDI5LjUwOTlDMjAuMTY3MyAyOS42NzEyIDE5LjUyMiAyOS43NTE5IDE4Ljc3NCAyOS43NTE5QzE4LjA0MDcgMjkuNzUxOSAxNy4zODggMjkuNjEyNSAxNi44MTYgMjkuMzMzOUMxNi4yNDQgMjkuMDY5OSAxNS43OTY3IDI4LjY5NTkgMTUuNDc0IDI4LjIxMTlDMTUuMTM2NyAyNy43NDI1IDE0Ljk2OCAyNy4xOTI1IDE0Ljk2OCAyNi41NjE5QzE0Ljk2OCAyNS41MDU5IDE1LjM0MiAyNC42NjI1IDE2LjA5IDI0LjAzMTlDMTYuODIzMyAyMy40MTU5IDE3LjQyOTMgMjMuMDYzOSAxOC44MDggMjIuOTc1OUwxOS4wNzIgMjUuMjQxOUMxOC4zMjQgMjUuMjg1OSAxOC4yNjA3IDI1LjQyNTIgMTcuOTgyIDI1LjY1OTlDMTcuNzAzMyAyNS45MDkyIDE3LjU2NCAyNi4xOTUyIDE3LjU2NCAyNi41MTc5QzE3LjU2NCAyNy4xOTI1IDE4LjAxMTMgMjcuNTI5OSAxOC45MDYgMjcuNTI5OUMxOS4yNDMzIDI3LjUyOTkgMTkuNTg4IDI3LjQ3ODUgMTkuOTQgMjcuMzc1OUMyMC4yOTIgMjcuMjczMiAyMC43MTczIDI3LjA5NzIgMjEuMjE2IDI2Ljg0NzlMMjIuMDc0IDI4Ljk4MTlaIiBmaWxsPSJ3aGl0ZSIvPgo8L3N2Zz4K)](https://ai-labs.olakrutrim.com/models/Krutrim-LLM-1) ## Model Overview Krutrim Large Language Model (LLM) is a 2 trillion token multilingual foundation model designed to serve Indian demographic needs through equitable representation of the country's array of native tongues. Training data incorporates the largest known Indic language dataset, mitigating associated data scarcity obstacles that encumber model parity across dialects. Evaluations demonstrate Krutrim's strong performance on Indic language benchmarks, surpassing or at par with state-of-the-art models despite being significantly smaller in training flops. Krutrim LLM also matches or exceeds standards set on English benchmarks by models trained on comparable flops (e.g. vs LLAMA-2 on 10 out of 16 tasks with average score of 0.57 vs 0.55 of LLAMA-2), evidencing flexible multilingual fluency. Through intentional design choices that redress endemic data imbalances, Krutrim LLM signifies meaningful progress in the pursuit of ethical, globally representative AI foundation models. ## Key Features - 7B parameter dense transformer model comparable similarly sized LLama-2 model; - Natively multilingual delivering best-in-class performance for a 7B mdoel on Indic benchmarks; - Exceeds performance of similar sized models on multilingual Indic generation tasks including creative writing, summarization, and translation; - Available in instruction-tuned version ## Model Developer - OLA Krutrim Team ## Model Dates - Krutrim-1 was trained between Oct 2023 and Nov 2023. ## Release History | Model Name | Release Date |Release Note | Reference| |------------|-------------|-------------|-------------| | Krutrim-1-Base | 2024-01-31 | Trained from scratch | | | Krutrim-1-Instruct | 2024-01-31 | SFT on Krutrim-1 Base |[Here](https://huggingface.co./krutrim-ai-labs/Krutrim-1-instruct)| ## Data Freshness - The dataset includes information up to April 2023. ## Model Architecture - Layers: 32 - Max Sequence Length: 4096 - Hidden Dimension: 4608 - Head Dimension: 96 - Number of Heads: 48 - Number of KV-Heads: 8 (GQA) - Vocabulary Size: 70400 - Architecture Type: Transformer Decoder (Auto-regressive Language Model) ## Evaluation Results ### English Comparison between Llama2Chat-7B and Krutrim-1-7B | Task | Llama2Chat | Krutrim-1-7B | |--------------------|--------------|------------| | arc | 0.517 | **0.557** | | bigbench | **0.359** | 0.330 | | boolq | 0.803 | **0.843** | | copa | 0.78 | **0.82** | | hellaswag | **0.754** | 0.740 | | jeopardy | **0.306** | 0.286 | | lambadaopenai | **0.695** | 0.682 | | logiqa | **0.332** | 0.3195 | | mathqa | 0.436 | **0.440** | | mmlu | 0.472 | **0.495** | | openbookqa | 0.44 | **0.464** | | piqa | 0.7601 | **0.7726** | | simplearithmetic | **0.160** | 0.077 | | squad | 0.3565 | **0.369** | | winograd | **0.8645** | 0.828 | | winogrande | 0.681 | **0.697** | | **average** | **0.54** | **0.54** | ### Benchmarks | Model | bn | gu | hi | kn | ml | mr | ta | te | |------------------|------|------|------|------|------|------|------|------| | **IndicCOPA** | | | | | | | | | | Krutrim-1-7B | 0.89 | 0.83 | 0.86 | 0.88 | 0.88 | 0.87 | 0.89 | 0.89 | | GPT-3.5 | 0.77 | 0.73 | 0.77 | 0.74 | 0.75 | 0.70 | 0.72 | 0.75 | | Airawata | - | - | 0.74 | - | - | - | - | - | | Kan-LLaMA | - | - | - | 0.74 | - | - | - | - | | Tam-LLaMA | - | - | - | - | - | - | 0.77 | - | | **IndicQA** | | | | | | | | | | Krutrim-1-7B | 0.65 | 0.64 | 0.64 | 0.60 | 0.66 | 0.58 | 0.75 | 0.83 | | Airawata | - | - | 0.62 | - | - | - | - | - | | Kan-LLaMA | - | - | - | 0.52 | - | - | - | - | | Tam-LLaMA | - | - | - | - | - | - | 0.35 | - | | **IndicSentiment**| | | | | | | | | | Krutrim-1-7B | 0.95 | 0.96 | 0.96 | 0.95 | 0.96 | 0.97 | 0.94 | 0.95 | | GPT-3.5 | 0.50 | 0.81 | 0.96 | 0.60 | 0.75 | 0.88 | 0.51 | 0.53 | | Airawata | - | - | 0.84 | - | - | - | - | - | | Kan-LLaMA | - | - | - | 0.85 | - | - | - | - | | Tam-LLaMA | - | - | - | - | - | - | 0.78 | - | | **IndicTranslation**| | | | | | | | | | Krutrim-1-7B | 0.88 | 0.89 | 0.95 | 0.88 | 0.89 | 0.92 | - | 0.88 | | Airawata | - | - | 0.94 | - | - | - | - | - | | Kan-LLaMA | - | - | - | 0.59 | - | - | - | - | | **IndicXParaphrase**| | | | | | | | | | Krutrim-1-7B | 0.91 | - | 0.97 | 0.82 | 0.90 | 0.94 | - | 0.61 | | Airawata | - | - | 0.60 | - | - | - | - | - | | Kan-LLaMA | - | - | - | 0.59 | - | - | - | - | ## Usage To use the instruct model, you can load it with `AutoModelForCausalLM` as follows: ``` import torch from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "krutrim-ai-labs/Krutrim-1-instruct" # Load model and tokenizer model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, trust_remote_code=True) tokenizer = AutoTokenizer.from_pretrained(model_id) # Apply Chat Template chat_template ="{% for message in messages %}{% if message['role'] == 'system' %}{{ '<|SYSTEM|> ' + message['content'] + '\n' }}{% elif message['role'] == 'user' %}{{ '<|USER|> ' + message['content'] + '\n' }}{% elif message['role'] == 'assistant' %}{% if not loop.last %}{{ '<|RESPONSE|>\n' + message['content'] + eos_token + '\n' }}{% else %}{{ '<|RESPONSE|>\n' + message['content'] + eos_token }}{% endif %}{% endif %}{% if loop.last and add_generation_prompt %}{{ '<|RESPONSE|>\n' }}{% endif %}{% endfor %}" tokenizer.chat_template = chat_template prompt_dict = [ {"role": "system", "content": "You are an AI assistant."}, {"role": "user", "content": "Who are you?"} ] prompts = tokenizer.apply_chat_template(prompt_dict, add_generation_prompt=True, tokenize=False) inputs = tokenizer(prompts, return_tensors='pt').to(device) inputs.pop("token_type_ids", None) # Generate response outputs = model.generate( **inputs, max_length=100 ) response = tokenizer.decode(outputs[0]) print(response) ``` ## Limitations The model was trained on a dataset that includes content from the internet, which may contain toxic language, biases, and unsafe content. As a result, the model may: - Amplify biases present in the training data - Generate toxic responses, especially when prompted with toxic inputs - Provide inaccurate, incomplete, or redundant answers - Generate responses in languages inconsistent with the prompt ## Ethical Considerations - The model may produce biased or offensive outputs based on its training data. - Users should apply human oversight when using the model for decision-making in sensitive areas. - While safeguards have been implemented, the model may still generate socially undesirable text in certain contexts. ## License This code repository and the model weights are licensed under the [Krutrim Community License.](LICENSE.md) ## Contact Contributions are welcome! If you have any improvements or suggestions, feel free to submit a pull request on GitHub. ## Citation ``` @misc{KrutrimLLM2024, author = {Aditya Kallappa, Palash Kamble, Vinayak Dhruv, Akshat Patidar, Arveti Manjunath, Raghav Awasthi, Divyansh Rajput, Pidathala Sowjanya, Rahul Kumar, Rishabh Nahata, Sanket Shah, Sulabh Katiyar, Soham Pendurkar, Sindhu Pawar, Abhinav Ravi, Deepak Kumar, Kumar Ashish, Pranav Raveendran, Bidyapathi Ray}, title = {Krutrim LLM}, year = {2024}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/ola-krutrim/Krutrim-1-7B}} } ```