changyeyu commited on
Commit
6614299
·
verified ·
1 Parent(s): bf20380

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -5
README.md CHANGED
@@ -18,11 +18,12 @@ license_link: LICENSE
18
  ## Model Overview
19
  Index-1.9B-32K is a language model with only 1.9 billion parameters, yet it supports a context length of 32K (meaning this extremely small model can read documents of over 35,000 words in one go). The model has undergone Continue Pre-Training and Supervised Fine-Tuning (SFT) specifically for texts longer than 32K tokens, based on carefully curated long-text training data and self-built long-text instruction sets. The model is now open-source on both Hugging Face and ModelScope.
20
 
21
- **Despite its small size (about 2% of models like GPT-4), Index-1.9B-32K demonstrates excellent long-text processing capabilities**. Below are comparison results with GPT-4 and GPT-3.5-turbo-16k:
22
- <div style="text-align: center;">
23
- <img src="z-attach-pic-pk-all.png" alt="" width="800">
24
- <p><strong>Comparison of Index-1.9B-32K with GPT-4 and other models in long-text capability</strong></p>
25
- </div>
 
26
 
27
  In a 32K-length needle-in-a-haystack test, Index-1.9B-32K achieved excellent results, as shown in the figure below. The only exception was a small yellow spot (91.08 points) in the region of (32K length, 10% depth), with all other areas performing excellently in mostly green zones.
28
  <div style="text-align: center;">
 
18
  ## Model Overview
19
  Index-1.9B-32K is a language model with only 1.9 billion parameters, yet it supports a context length of 32K (meaning this extremely small model can read documents of over 35,000 words in one go). The model has undergone Continue Pre-Training and Supervised Fine-Tuning (SFT) specifically for texts longer than 32K tokens, based on carefully curated long-text training data and self-built long-text instruction sets. The model is now open-source on both Hugging Face and ModelScope.
20
 
21
+ Despite its small size (about 2% of models like GPT-4), Index-1.9B-32K demonstrates excellent long-text processing capabilities.
22
+ As shown in the figure below, our 1.9B-sized model's score even surpasses that of the 7B-sized model. Below is a comparison with models like GPT-4 and Qwen2:
23
+
24
+ <p align="center"> <img src="z-attach-pic-pk-all.png" alt="" width="800">
25
+ </p> <p align="center"><strong>Comparison of Index-1.9B-32K with GPT-4, Qwen2, and other models in Long Context capability</strong>
26
+ </p>
27
 
28
  In a 32K-length needle-in-a-haystack test, Index-1.9B-32K achieved excellent results, as shown in the figure below. The only exception was a small yellow spot (91.08 points) in the region of (32K length, 10% depth), with all other areas performing excellently in mostly green zones.
29
  <div style="text-align: center;">