OPEA
/

GGUF
Inference Endpoints
conversational
cicdatopea commited on
Commit
195b181
·
verified ·
1 Parent(s): 1b82f2a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +35 -2
README.md CHANGED
@@ -15,8 +15,8 @@ Please follow the [Build llama.cpp locally](https://github.com/ggerganov/llama.c
15
 
16
  ### INT4 Inference
17
  ```bash
18
- text="为什么企鹅没有被北极熊吃掉?"
19
- ./llama-cli -m SmallThinker-3B-Preview-3.1B-Q4_0.gguf -p "Human:You are a helpful and harmless assistant.You should think step-by-step.$text\nAssistant:\n" -n 512
20
 
21
  # INT4:
22
  # 我正在尝试理解为什么企鹅没有被北极熊吃掉。这是一个有趣的问题,让我思考一下。
@@ -107,6 +107,39 @@ text="How many r in strawberry."
107
  # S-T-R-A-W-B-E-R-R-Y
108
  # Positions:
109
  # 1. S
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
110
  ```
111
 
112
  ### Generate the model
 
15
 
16
  ### INT4 Inference
17
  ```bash
18
+ > text="为什么企鹅没有被北极熊吃掉?"
19
+ > ./llama-cli -m SmallThinker-3B-Preview-3.1B-Q4_0.gguf -p "Human:You are a helpful and harmless assistant.You should think step-by-step.$text\nAssistant:\n" -n 512
20
 
21
  # INT4:
22
  # 我正在尝试理解为什么企鹅没有被北极熊吃掉。这是一个有趣的问题,让我思考一下。
 
107
  # S-T-R-A-W-B-E-R-R-Y
108
  # Positions:
109
  # 1. S
110
+
111
+ text="Which number is bigger, 9.11 or 9.8?"
112
+ # INT4:
113
+ # To answer this, I need to compare two numbers, 9.11 and 9.8, to determine which one is bigger. Let me think about this carefully.
114
+ # First, I should consider the structure of the numbers. Both numbers have a single digit before the decimal point, which represents the tens place. In this case, the tens digit is 9 for both numbers. This means that both numbers are equally close to 10, but one digit after the decimal point is different.
115
+ # Let me break this down further. The number 9.11 has a 1 in the tenths place, while the number 9.8 has an 8 in the tenths place. The tenths digit is crucial because it indicates the first digit after the decimal point, which affects the overall value of the number.
116
+ # Now, let me compare the tenths digits: 1 in 9.11 and 8 in 9.8. The tenths digit in 9.8 is larger than the tenths digit in 9.11. This means that 9.8 is a larger number than 9.11.
117
+ # Wait a minute, let me double-check this comparison to ensure I haven't missed anything. The tens digit is the same for both numbers, but the tenths digit in 9.8 is greater. This confirms that 9.8 is indeed larger than 9.11.
118
+ # Fantastic! After this careful consideration, I can confidently say that 9.8 is bigger than 9.11.
119
+ # Final Answer: 9.8 is bigger than 9.11. [end of text]
120
+
121
+ # BF16:
122
+ # So I have these two numbers: 9.11 and 9.8. I need to figure out which one is bigger. At first glance, they look pretty similar, but there's a difference in the digits after the decimal point.
123
+ # Let me think about this step by step. Both numbers are decimals with one digit before the decimal point and two digits after it. So, they're both tenths and hundredths.
124
+ # First, I can compare the whole number parts. In both cases, the whole number part is 9. Since 9 is equal to 9, that means so far, they're tied at the tenths place.
125
+ # Now, let's look at the decimal places. Starting from the right, the first digit after the decimal is the tenths place, and the next one is the hundredths place.
126
+ # In 9.11, the tenths digit is 1, and the hundredths digit is 1.
127
+ # In 9.8, the tenths digit is 8, and the hundredths digit is 0.
128
+ # So, comparing the tenths place:
129
+ # - 9.11 has 1 in the tenths place.
130
+ # - 9.8 has 8 in the tenths place.
131
+ # Since 8 is greater than 1, that means 9.8 is larger than 9.11.
132
+ # Wait a minute, just to make sure I'm not missing anything, maybe I should consider the entire number.
133
+ # Another way to think about it is to convert these decimals into fractions or to understand their values in terms of money or something familiar.
134
+ # For example, if I imagine these as amounts of money, 9.11 could be $9.11 and 9.8 could be $9.80. Clearly, $9.80 is more than $9.11 because 80 cents is more than 11 cents.
135
+ # Alternatively, I could subtract one from the other to see which one is bigger.
136
+ # Let's try subtracting 9.11 from 9.8.
137
+ # 9.8 - 9.11 = ?
138
+ # Well, subtracting the decimals directly:
139
+ # 8 - 11 would require borrowing, but since 8 is less than 11, I need to borrow from the whole number.
140
+ # So, 9 becomes 8, and 11 becomes 21.
141
+ # Then, 8 - 11 in the hundredths place is still negative, so I need to borrow again.
142
+ # But
143
  ```
144
 
145
  ### Generate the model