Merge branch 'main' of https://huggingface.co./togethercomputer/Llama-2-7B-32KCtx-v0.1

README.md

@@ -1,3 +1,36 @@
 ---
 license: apache-2.0
+datasets:
+- togethercomputer/RedPajama-Data-1T
+- togethercomputer/RedPajama-Data-Instruct
+- EleutherAI/pile
+language:
+- en
+library_name: transformers
 ---
+
+# Llama-2-7B-32KCtx
+
+
+# Install Flash Attention For Inference with 32K
+
+```
+export CUDA_HOME=/usr/local/cuda-11.8
+pip install ninja
+pip install flash-attn --no-build-isolation
+pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary
+```
+Please revise the path of `CUDA_HOME`. `ninja` is needed to accelerate the process of compiling.
+
+And then:
+```python
+model = AutoModelForCausalLM.from_pretrained('togethercomputer/Llama-2-7B-32KCtx-v0.1', trust_remote_code=True, torch_dtype=torch.float16)
+```
+
+You can also use vanilla `transformers` to load this model:
+```python
+model = AutoModelForCausalLM.from_pretrained('togethercomputer/Llama-2-7B-32KCtx-v0.1', torch_dtype=torch.float16)
+```
+
+
+TODO