reach-vb HF staff commited on
Commit
284e0f5
1 Parent(s): cc349a0

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +39 -0
README.md ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ license_link: https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct/blob/main/LICENSE
4
+ language:
5
+ - en
6
+ pipeline_tag: text-generation
7
+ base_model: Qwen/Qwen2.5-0.5B-Instruct
8
+ tags:
9
+ - chat
10
+ - mlx
11
+ - mlx-my-repo
12
+ library_name: transformers
13
+ ---
14
+
15
+ # reach-vb/Qwen2.5-0.5B-Instruct-Q3-mlx
16
+
17
+ The Model [reach-vb/Qwen2.5-0.5B-Instruct-Q3-mlx](https://huggingface.co/reach-vb/Qwen2.5-0.5B-Instruct-Q3-mlx) was converted to MLX format from [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) using mlx-lm version **0.20.5**.
18
+
19
+ ## Use with mlx
20
+
21
+ ```bash
22
+ pip install mlx-lm
23
+ ```
24
+
25
+ ```python
26
+ from mlx_lm import load, generate
27
+
28
+ model, tokenizer = load("reach-vb/Qwen2.5-0.5B-Instruct-Q3-mlx")
29
+
30
+ prompt="hello"
31
+
32
+ if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
33
+ messages = [{"role": "user", "content": prompt}]
34
+ prompt = tokenizer.apply_chat_template(
35
+ messages, tokenize=False, add_generation_prompt=True
36
+ )
37
+
38
+ response = generate(model, tokenizer, prompt=prompt, verbose=True)
39
+ ```