DopeorNope
/

SOLARC-MOE-10.7Bx6

Text Generation

Mixture of Experts

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

DopeorNope commited on Jan 3, 2024

Commit

b7513b8

·

1 Parent(s): a216811

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -60,7 +60,7 @@ repo = "DopeorNope/SOLARC-MOE-10.7Bx6"
 OpenOrca = AutoModelForCausalLM.from_pretrained(
         repo,
         return_dict=True,
-        torch_dtype=torch.float16,
         device_map='auto'
 )
 OpenOrca_tokenizer = AutoTokenizer.from_pretrained(repo)

 OpenOrca = AutoModelForCausalLM.from_pretrained(
         repo,
         return_dict=True,
+        torch_dtype=torch.float32,
         device_map='auto'
 )
 OpenOrca_tokenizer = AutoTokenizer.from_pretrained(repo)