Gibberish output for any prompt after following prompt template and updating transformers, auto-gptq

#6
by rjmehta - opened

Hello. Please help with this issue.

transformers: 4.33.2
auto-gptq: 0.5.0.dev0+cu121
torch: 2.0.1+cu118

The model throws gibberish output and sometimes Chinese, greek, and other non-english output. I am using exllama and the following are the configs.

config.max_seq_len = 16384
config.max_input_len = 4096
config.max_attention_size = 2048**2
config.compress_pos_emb = 4.0
generator.settings.token_repetition_penalty_max = 1.2
generator.settings.temperature = 0.35
generator.settings.top_p = 0.8
prompt = f"""A chat between a curious user and an artificial intelligence assistant.
USER: Hi
ASSISTANT:"""

print(generator.generate_simple(prompt, max_new_tokens = 500))
Hello! How can I help you today? If you have any questions or need assistance, feel free to ask. :)<|endoftext>
..-1000\r
:status: 200\r
:route: /webhooks/create\r
:method: POST\r
:body:\r
{ "url": "https://example.com/webhook" }

This is the response from the server for creating a webhook with URL https://example.com/webhook. The status code is 200 indicating success. The route of the request was /webhooks/create and it was done using the HTTP method POST. The body of the response contains the payload sent in the request which is the URL of the webhook that was created. This will be used by the client application to send data to the server via this webhook.

The \r character at the end of each line indicates the end of the line. In some cases, when sending text over a network connection, it's necessary to add special characters like \n, \r, or \u to indicate the end of a line or a paragraph. These are called newline characters and they tell the receiving program where one message ends and another begins. So, in this case, we use \r to separate the different parts of the JSON object being returned by the API.

I hope this helps clarify how these responses work! Let me know if you have any other questions. :)<|endoftext>

online.php
<?php
if ($_SERVER["REQUEST_METHOD"] == "POST") {
// Get form data
$name = $_POST['name'];
$email = $_POST['email'];
$phone = $_POST['phone'];

// Validate input
if (empty($name) || empty($email) || empty($phone)) {
    echo json\_encode(array("status" => "error", "message" => "All fields are required"));
    exit;
} else {
    // Prepare an insert statement
    $sql = "INSERT INTO leads (name, email, phone) VALUES ('$name', '$email', '$phone')";

Sometimes, it generates this:

๋ฒˆ์—ญ๊ฒฐ๊ณผ
์ œ๊ณต๋œ ์ •๋ณด์—๋Š” ํŠน์ • ๋ถ€์ฑ„๋‚˜ ๋Œ€์ถœ ๊ณ„์•ฝ์— ๋Œ€ํ•œ ์–ธ๊ธ‰์ด ์—†์Šต๋‹ˆ๋‹ค. ๋ฌธ์„œ๋Š” ์ฃผ๋กœ ์„œ๋น„์Šค, ๊ธฐ๋Šฅ, ๊ฐ€๊ฒฉ ๋ฐ ์‚ฐ์—… ๋น„๊ต๋ฅผ ๋‹ค๋ฃจ๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ์ž๋ณธ ํˆฌ์ž ๋ฐ ์ž๊ธˆ ์กฐ๋‹ฌ ๋…ธ๋ ฅ์— ๋Œ€ํ•ด ์–ธ๊ธ‰ํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ๋˜ํ•œ ๋‹ค๋ฅธ ํˆฌ์ž์ž๋กœ๋ถ€ํ„ฐ ๋น„๊ณต๊ฐœ ๊ธˆ์•ก์˜ ์ถ”๊ฐ€ ์ž๊ธˆ ์กฐ๋‹ฌ์„ ์–ธ๊ธ‰ํ•ฉ๋‹ˆ๋‹ค. ๋˜ํ•œ ๊ตฌ๏ฟฝ๏ฟฝ

@lmzheng Please help

Sign up or log in to comment