Updated Model Card

README.md

@@ -7,23 +7,35 @@ license_link: LICENSE
 
 [![Model architectuve](https://img.shields.io/badge/Model%20Arch-Transformer%20Decoder-green)](#model-architecture)[![Model size](https://img.shields.io/badge/Params-340B-green)](#model-architecture)[![Language](https://img.shields.io/badge/Language-Multilingual-green)](#datasets)
 
-### License
-
-NVIDIA Open Model License
-
 ### Model Overview
 
-Nemotron-4-340B-Instruct is a large language model (LLM) which is a fine-tuned version of the Nemotron-4-340B-Base base model, optimized for English single and multi-turn chat use-cases. The base model was pre-trained on a corpus of 8 trillion tokens consisting of a diverse assortment of English based texts, 40+ coding languages, and 50+ natural languages. 
+Nemotron-4-340B-Instruct is a large language model (LLM) that can be used as part of a synthetic data generation pipeline to create training data that helps researchers and developers build their own LLMs; and is a fine-tuned version of the Nemotron-4-340B-Base model, optimized for English single and multi-turn chat use-cases. The base model was pre-trained on a corpus of 9 trillion tokens consisting of a diverse assortment of English based texts,50+ natural languages, and 40+ coding languages. 
 
 Subsequently the Nemotron-4-340B-Instruct model went through additional alignment steps including:
-
 - Supervised Fine-tuning (SFT)
-- Direct Policy Optimization (DPO)
-- Additional in-house alignment techniques (Publication work in progress)
+- Direct Preference Optimization (DPO)
+- Additional in-house alignment technique: Reward-aware Preference Optimization (RPO)
+
+Throughout the alignment process, we relied on only approximately 20K human-annotated data while our data generation pipeline synthesized over 98% of the data used for supervised fine-tuning and preference fine-tuning (DPO & RPO). We provide comprehensive details about our synthetic data generation pipeline in the technical report.
+
+This results in a model that is aligned for human chat preferences, improvements in mathematical reasoning, coding and instruction-following, and is capable of generating high quality synthetic data for a variety of use cases.
+
+Under the NVIDIA Open Model License, NVIDIA confirms: 
+- Models are commercially usable. 
+- You are free to create and distribute Derivative Models. 
+- NVIDIA does not claim ownership to any outputs generated using the Models or Derivative Models.
 
-This results in a final model that is aligned for human chat preferences, improvements in mathematical reasoning, coding and instruction following.
+### License: 
 
-This model is ready for commercial use.
+[NVIDIA Open Model License](https://developer.download.nvidia.com/licenses/nvidia-open-model-license-agreement-june-2024.pdf)
+
+### Intended use
+
+Nemotron-4-340B-Instruct is a chat model intended for use for the English language. 
+
+Nemotron-4-340B-Instruct is designed for Synthetic Data Generation to enable developers and enterprises for building and customizing their own large language models and LLM applications. 
+
+The instruct model itself can be further customized using the [NeMo Framework](https://docs.nvidia.com/nemo-framework/index.html) suite of customization tools including Parameter-Efficient Fine-Tuning (P-tuning, Adapters, LoRA), and SFT/Steer-LM/RLHF using [NeMo-Aligner](https://github.com/NVIDIA/NeMo-Aligner).
 
 **Model Developer:** NVIDIA
 
@@ -51,19 +63,21 @@ FP8 Inference:
 
 ### Model Architecture:
 
-The base model, Nemotron-4-340B, was trained with a global batch-size of 2304, a sequence length of 4096 tokens, uses Grouped-Query Attention (GQA), and RoPE positional embeddings.
+Nemotron-4-340B-Base, is standard decoder-only Transformer, trained with a sequence length of 4096 tokens, uses Grouped-Query Attention (GQA), and Rotary Position Embeddings (RoPE).
 
 **Architecture Type:** Transformer Decoder (auto-regressive language model)
 
-### Software Integration
-
-**Supported Hardware Architecture Compatibility:** NVIDIA H100, A100 80GB, A100 40GB
+**Network Architecture:**
+Nemotron-4
 
 ### Usage
 
 1. We will spin up an inference server and then call the inference server in a python script. Let’s first define the python script ``call_server.py``
 
 ```python
+import json
+import requests
+
 headers = {"Content-Type": "application/json"}
 
 def text_generation(data, ip='localhost', port=None):
@@ -100,13 +114,16 @@ prompt = PROMPT_TEMPLATE.format(prompt=question)
 print(prompt)
 
 response = get_generation(prompt, greedy=True, add_BOS=False, token_to_gen=1024, min_tokens=1, temp=1.0, top_p=1.0, top_k=0, repetition=1.0, batch=False)
+response = response[len(prompt):]
+if response.endswith("<extra_id_1>"):
+	response = response[:-len("<extra_id_1>")]
 print(response)
 ```
 
+2. Given this python script, we will create a bash script, which spins up the inference server within the NeMo container(docker pull nvcr.io/nvidia/nemo:24.01.framework) and calls the python script ``call_server.py``. The bash script ``nemo_inference.sh`` is as follows,
 
-2. Given this python script, we will create a bash script, which spins up the inference server within the [NeMo container](https://github.com/NVIDIA/NeMo/blob/main/Dockerfile) and calls the python script ``call_server.py``. The bash script ``nemo_inference.sh`` is as follows,
-
-
+```bash
+NEMO_FILE=$1
 WEB_PORT=1424
 
 depends_on () {
@@ -122,15 +139,15 @@ depends_on () {
     echo "server ($HOST:$PORT) is up running"
 }
 
-echo "output filename: $OUTPUT_FILENAME"
+
 
 /usr/bin/python3 /opt/NeMo/examples/nlp/language_modeling/megatron_gpt_eval.py \
         gpt_model_file=$NEMO_FILE \
         pipeline_model_parallel_split_rank=0 \
         server=True tensor_model_parallel_size=8 \
-        trainer.precision=bf16 pipeline_model_parallel_size=4 \
+        trainer.precision=bf16 pipeline_model_parallel_size=2 \
         trainer.devices=8 \
-        trainer.num_nodes=4 \
+        trainer.num_nodes=2 \
         web_server=False \
         port=${WEB_PORT} &
     SERVER_PID=$!
@@ -144,47 +161,45 @@ echo "output filename: $OUTPUT_FILENAME"
 
         echo "SLURM_NODEID: $SLURM_NODEID"
         echo "local_rank: $local_rank"
-        /usr/bin/python3 call_server.py
+        /usr/bin/python3 /scripts/call_server.py
         echo "clean up dameons: $$"
         kill -9 $SERVER_PID
         pkill python
     fi
     wait
+```
 
 
 3, We can launch the ``nemo_inferece.sh`` with a slurm script defined like below, which starts a 4-node job for the model inference.
 
-
+```bash
 #!/bin/bash
 #SBATCH -A SLURM-ACCOUNT
 #SBATCH -p SLURM-PARITION
-#SBATCH -N 4 # number of nodes
+#SBATCH -N 2 # number of nodes
 #SBATCH -J generation      
 #SBATCH --ntasks-per-node=8   
 #SBATCH --gpus-per-node=8
 set -x
 
+RESULTS=<PATH_TO_YOUR_SCRIPTS_FOLDER>
+OUTFILE="${RESULTS}/slurm-%j-%n.out"
+ERRFILE="${RESULTS}/error-%j-%n.out"
+MODEL=<PATH_TO>/Nemotron-4-340B-Instruct
+
+MOUNTS="--container-mounts=<PATH_TO_YOUR_SCRIPTS_FOLDER>:/scripts,MODEL:/model"
 read -r -d '' cmd <<EOF
-bash nemo_inference.sh
+bash /scripts/nemo_inference.sh /model
 EOF
 
 srun -o $OUTFILE -e $ERRFILE --container-image="$CONTAINER" $MOUNTS bash -c "${cmd}"
-
-
-
-### Intended use
-
-Nemotron-4-340B-Instruct is a chat model intended for use in over 50+ natural and coding languages. For best performance on a given task, users are encouraged to customize the chat model using the NeMo Framework suite of customization tools including Parameter-Efficient Fine-Tuning (P-tuning, Adapters, LoRA), and SFT/Steer-LM/RLHF.
-
-### Red Teaming:
-
-TO BE UPDATED BASED ON RED TEAMING PICs + LEGAL REVIEW
+```
 
 ### Evaluation Results
 
 #### MT-Bench (GPT-4-Turbo)
 
-Evaluated using select datasets from the [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://arxiv.org/pdf/2306.05685v4)
+Evaluated using MT-Bench judging by GPT-4-Turbo as described in the [HelpSteer2 Dataset Paper](https://arxiv.org/abs/2406.08673)
 
 | total | writing |	roleplay | extraction |	stem | humanities | reasoning |	math | coding | turn 1 | turn 2 |
 | ----- | ------- | -------- | ---------- | ---- | ---------- | --------- | ---- | ------ | ------ | ------ | 
@@ -208,7 +223,7 @@ Evaluated using the Multi-task Language Understanding benchmarks as introduced i
 
 #### GSM8K
 
-Evaluated using the Grade School Math 8K (GSM8K) bechmark as introduced in [Training Verifiers to Solve Math Word Problems](https://arxiv.org/pdf/2110.14168v2).
+Evaluated using the Grade School Math 8K (GSM8K) benchmark as introduced in [Training Verifiers to Solve Math Word Problems](https://arxiv.org/pdf/2110.14168v2).
 
 | GSM8K 0-shot |
 | ----------------- | 
@@ -223,6 +238,15 @@ Evaluated using the HumanEval benchmark as introduced in [Evaluating Large Langu
 | ----- |
 | 73.2 |
 
+#### MBPP
+
+Evaluated using the MBPP Dataset as introduced in the [Program Synthesis with Large Language Models](https://arxiv.org/abs/2108.07732) paper.
+
+| MBPP 0-shot|
+| ----------------- | 
+| 75.4 | 
+
+
 #### Arena Hard
 
 Evaluated using the [Arena-Hard Pipeline](https://lmsys.org/blog/2024-04-19-arena-hard/) from the LMSys Org.
@@ -235,17 +259,10 @@ Evaluated using the [Arena-Hard Pipeline](https://lmsys.org/blog/2024-04-19-aren
 
 Evaluated using the AlpacaEval 2.0 LC (Length Controlled) as introduced in the paper: [Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators](https://arxiv.org/abs/2404.04475)
 
-| AlpacaEval |
+| AlpacaEval 2.0 LC|
 | ----------------- | 
-| 54.2 | 
-
-#### MBPP
+| 41.5 | 
 
-Evaluated using the MBPP Dataset as introduced in the [Program Synthesis with Large Language Models](https://arxiv.org/abs/2108.07732) paper.
-
-| MBPP |
-| ----------------- | 
-| 75.4 | 
 
 #### TFEval
 
@@ -256,6 +273,14 @@ Evaluated using the CantTalkAboutThis Dataset as introduced in the [CantTalkAbou
 | 81.7  | 97.7 |
 
 
-### Limitations
+### Adversarial Testing and Red Teaming Efforts 
+
+The Nemotron-4 340B-Instruct model underwent extensive safety evaluation including adversarial testing via three distinct methods: 
+- [Garak](https://docs.garak.ai/garak), is an automated LLM vulnerability scanner that probes for common weaknesses, including prompt injection and data leakage. 
+- [AEGIS](https://arxiv.org/pdf/2404.05993), is a content safety evaluation dataset and LLM based content safety classifier model, that adheres to a broad taxonomy of 13 categories of critical risks in human-LLM interactions.
+- Human Content Red Teaming leveraging human interaction and evaluation of the models' responses.
+
+
+### Ethical Considerations
 
-The base model was trained on data that contains toxic language and societal biases originally crawled from the internet. Therefore, the model may amplify those biases and return toxic responses especially when prompted with toxic prompts. The model may generate answers that may be inaccurate, omit key information, or include irrelevant or redundant text producing socially unacceptable or undesirable text, even if the prompt itself does not include anything 
+NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications.  When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.  For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards [Insert Link to Model Card++ here].  Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).