FlagAlpha/Llama2-Chinese-13b-Chat-4bit

用官方教程跑出来全是乱码

<s><unk> : 怎么登上火星
</s><s>Assistant: izontalarod groupingmittel externosertenhexavalkaportehanetuDK Telephone ringsódigo Armyecairatât tentacles stemCellularityppeuvo summitten hiermouthedlyekenrolloenesaltyalinufeadi Literaturedressesktoppreviousosto Sudifferential Stevensoniki unsnerstone Branchespecivo Little bittersVP embarrassmentalitiesouglasmicheck Session nose HavvyPNellsandaimalegalusianismainteduclear Horn tant Holderinoslackening试pty Statistics MasonoidableaguidthspantoberfestGVnbeyondurasering beg LaurenthreniaceaeostaGest tubute Schles Fichanteniumsiskountains axis Liverpoolorio difficultURExicycle dashboard Lionettecita Nederlandhebста depositselfampswichescop emails Independritz bereotecaagar suitableemente Wingagonnitzlade ringentin centuriesleb Detroitchoragesoupaign SSgewschemaicons Cavaleuw domin Beg destination ZoblasseamenDP Bullícioitul PalaciosemetLTachtmanship status quarksovi preparationallyьwedischenukiingenhook une Fraction zoneornadtreu dedicantage pau Ortodoxyrsnel Eztringabanitore formalize tip decentral Hin Bitteotenhagennittлу peacefullyradantonoremalionisto factorandybareutheranzaelmadrume pronounced Dunningtonaustraliana catchphrases Leopoldansewersyeraggiemanufact Hi pun Originating externastsville Freund GraphicalodiFS guardianship Statusesöder Milandboxonneeggi comfortablyhd thumbagoonshire stagzanekerinnerWDoked anchorIMA tamble丁atzistory Laneuleradora solidarren cornerege段eschCLC numbers Visitingidiary scaledock路HR inhibitionalscot bunchgetElement lost sightppiadesmpóeniggebra Netting union IMconv Flanderspace reservedlauщеatanfalegoreatebstleaunyaogle supplement safenixicodemosedesincetilearchaster soriginalhausenopus massacre Bras Bash Ans intellectual surfacesfel ХронологијаgentemukinsonietConvertfannectleeionarioerb circumamblingtonaumѐ nostalgdegreezeeberger Union labelsledges OberitelLENG LitrixHDTACnbspiture
branchingtons reconstructionseemsuro jeegyashiiglia fiercenugeness Padernal стаamareteoraddleazyetta Otto Dokondo rang cher Riemannoli size conflicinea spare
Geneault Sad decephaloberty Giuseppeenserule Totmenteèsicoaelгородundeeangelabeirializedragenauxectoriallyyy Lasfluores succantarastenica io redisernacknelliaidern cart

程序：
from transformers import AutoTokenizer
from auto_gptq import AutoGPTQForCausalLM
model = AutoGPTQForCausalLM.from_quantized('FlagAlpha/Llama2-Chinese-13b-Chat-4bit', device="cuda:0")
tokenizer = AutoTokenizer.from_pretrained('FlagAlpha/Llama2-Chinese-13b-Chat-4bit',use_fast=False)
input_ids = tokenizer(['<s>Human: 怎么登上火星\n</s><s>Assistant: '], return_tensors="pt",add_special_tokens=False).input_ids.to('cuda')
generate_input = {
"input_ids":input_ids,
"max_new_tokens":512,
"do_sample":True,
"top_k":50,
"top_p":0.95,
"temperature":0.3,
"repetition_penalty":1.3,
"eos_token_id":tokenizer.eos_token_id,
"bos_token_id":tokenizer.bos_token_id,
"pad_token_id":tokenizer.pad_token_id
}
generate_ids = model.generate(**generate_input)
text = tokenizer.decode(generate_ids[0])
print(text)