DavidAU commited on
Commit
66fa526
·
verified ·
1 Parent(s): 98dd720

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +40 -0
README.md CHANGED
@@ -95,6 +95,46 @@ ARM QUANTS:
95
 
96
  3 "ARM" quants have been added to the repo (23/10/2024). These quants are machine specific.
97
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
98
  <b>Optional Enhancement:</B>
99
 
100
  The following can be used in place of the "system prompt" or "system role" to further enhance the model.
 
95
 
96
  3 "ARM" quants have been added to the repo (23/10/2024). These quants are machine specific.
97
 
98
+ <B>Settings, Quants and Critical Operations Notes:</b>
99
+
100
+ Change in temp (ie, .4, .8, 1.5, 2, 3 ) will drastically alter output.
101
+
102
+ Rep pen settings will also alter output too.
103
+
104
+ This model needs "rep pen" of 1.02 or higher.
105
+
106
+ For role play: Rep pen of 1.05 to 1.08 is suggested.
107
+
108
+ Raise/lower rep pen SLOWLY ie: 1.011, 1.012 ...
109
+
110
+ Rep pen will alter prose, word choice (lower rep pen=small words / more small word - sometimes) and creativity.
111
+
112
+ To really push the model:
113
+
114
+ Rep pen 1.05 or lower / Temp 3+ ... be ready to stop the output because it may go and go at these strong settings.
115
+
116
+ Longer prompts vastly increase the quality of the model's output.
117
+
118
+ QUANT CHOICE(S):
119
+
120
+ Higher quants will have more detail, nuance and in some cases stronger "emotional" levels. Characters will also be
121
+ more "fleshed out" too. Sense of "there" will also increase.
122
+
123
+ Q4KM/Q4KS are good, strong quants however if you can run Q5, Q6 or Q8 - go for the highest quant you can.
124
+
125
+ This repo also has 3 "ARM" quants for computers that support this quant. If you use these on a "non arm" machine token per second will be very low.
126
+
127
+ IQ4XS: Due to the unusual nature of this quant (mixture/processing), generations from it will be different then other quants.
128
+
129
+ You may want to try it / compare it to other quant(s) output.
130
+
131
+ Special note on Q2k/Q3 quants:
132
+
133
+ You may need to use temp 2 or lower with these quants (1 or lower for q2k). Just too much compression at this level, damaging the model. I will see if Imatrix versions
134
+ of these quants will function better.
135
+
136
+ Rep pen adjustments may also be required to get the most out of this model at this/these quant level(s).
137
+
138
  <b>Optional Enhancement:</B>
139
 
140
  The following can be used in place of the "system prompt" or "system role" to further enhance the model.