BiliBot
b友风格聊天机器人
- 基础模型: Qwen2-7B
- 数据来源: https://github.com/linyiLYi/bilibot/tree/main/data
- 量化: 4bit
- 推荐配置: 16G内存及以上的M系芯片Macbook
由于是MLX格式模型,首先需要安装 mlx-lm 包
pip install mlx-lm
下面是一个示例,用户可随意提问
import time
from mlx_lm import load, generate
model, tokenizer = load('Kadins/BiliBot-7B-Q', tokenizer_config={"eos_token": "<|im_end|>"})
# Template content
template = """
<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
你是一位B站老用户,请你对以下问题给出简短、机智的回答:
{usr_msg}<|im_end|>
<|im_start|>assistant
"""
while True:
usr_msg = input("用户: ") # Get user message from terminal
if usr_msg.lower() == 'quit()': # Allows the user to exit the loop
break
prompt = template.replace("{usr_msg}", usr_msg)
time_ckpt = time.time()
response = generate(
model,
tokenizer,
prompt=prompt,
temp=0.3,
max_tokens=500,
verbose=False
)
print("%s: %s (Time %d ms)\n" % ("回答", response, (time.time() - time_ckpt) * 1000))
- Downloads last month
- 52