Merge of SthenoMaidBlackroot (8b) with Llama3 Instruct (8b) creating a model at 12.2B to improve instruction following and output.
Story / Scene / Fiction:
Unique "pre-amble" / "foreshadowing" of events before they happen instead of "immediate and into the fire" type prose.
Some improvement in logic/problem solving relative to SMB 8B.
This is an "Ultra Quality" source float32 -> GGUF transfer.
The F32 version exhibits even stronger creativity (detail, place, "there") , but slightly less logic/problem solving relative to the F16 version (at a different repo).
SthenoMaidBlackroot is a 3 model merge and fine tune.
One of the goals of this project was to see if it could be merged with Llama3 Instruct, yet maintain it's unique character YET also gain some "brainpower" as well.
The biggest change was removal of most "tells" ( IE: "he stood frozen in horror").
In most cases the model will describe the emotion(s) / what is happening in more detail.
Other changes include prose, sentence, and paragraph structure as well as variety.
A simple pass-through merge was used.
See the examples below.
Details:
Please report any issue(s) and/or feedback via the "Community tab".
This is a LLAMA3 model, and requires Llama3 template, but may work with other template(s) and has maximum context of 8k / 8192. However this can be extended using "rope" settings up to 32k.
For details on "rope" and how to set, see the BOTTOM of this page:
Here is the standard LLAMA3 template:
{
"name": "Llama 3",
"inference_params": {
"input_prefix": "<|start_header_id|>user<|end_header_id|>\n\n",
"input_suffix": "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
"pre_prompt": "You are a helpful, smart, kind, and efficient AI assistant. You always fulfill the user's requests to the best of your ability.",
"pre_prompt_prefix": "<|start_header_id|>system<|end_header_id|>\n\n",
"pre_prompt_suffix": "<|eot_id|>",
"antiprompt": [
"<|start_header_id|>",
"<|eot_id|>"
]
}
}
It is also known, that the "Command-R" template will work too, and will result in radically different prose/output.
Settings / Known Issue(s) and Fix(es):
The default "repetition penalty" (from LMStudio) of 1.1 is recommended. (this was used for examples generations below.)
Use the smallest amount of change possible, as "rep pen" impacts creativity.
Model has been tested with "temp" range of 0 to .8 ; however it will work with any temp setting.
Optional Enhancement:
The following can be used in place of the "system prompt" or "system role" to further enhance the model.
It can also be used at the START of a NEW chat, but you must make sure it is "kept" as the chat moves along. In this case the enhancements do not have as strong effect at using "system prompt" or "system role".
Copy and paste EXACTLY as noted, DO NOT line wrap or break the lines, maintain the carriage returns exactly as presented.
Below is an instruction that describes a task. Ponder each user instruction carefully, and use your skillsets and critical instructions to complete the task to the best of your abilities. Here are your skillsets: [MASTERSTORY]:NarrStrct(StryPlnng,Strbd,ScnSttng,Exps,Dlg,Pc)-CharDvlp(ChrctrCrt,ChrctrArcs,Mtvtn,Bckstry,Rltnshps,Dlg*)-PltDvlp(StryArcs,PltTwsts,Sspns,Fshdwng,Climx,Rsltn)-ConfResl(Antg,Obstcls,Rsltns,Cnsqncs,Thms,Symblsm)-EmotImpct(Empt,Tn,Md,Atmsphr,Imgry,Symblsm)-Delvry(Prfrmnc,VcActng,PblcSpkng,StgPrsnc,AudncEngmnt,Imprv) [*DialogWrt]:(1a-CharDvlp-1a.1-Backgrnd-1a.2-Personality-1a.3-GoalMotiv)>2(2a-StoryStruc-2a.1-PlotPnt-2a.2-Conflict-2a.3-Resolution)>3(3a-DialogTech-3a.1-ShowDontTell-3a.2-Subtext-3a.3-VoiceTone-3a.4-Pacing-3a.5-VisualDescrip)>4(4a-DialogEdit-4a.1-ReadAloud-4a.2-Feedback-4a.3-Revision) Here are your critical instructions: Ponder each word choice carefully to present as vivid and emotional journey as is possible. Choose verbs and nouns that are both emotional and full of imagery. Load the story with the 5 senses. Aim for 50% dialog, 25% narration, 15% body language and 10% thoughts. Your goal is to put the reader in the story.
You do not need to use this, it is only presented as an additional enhancement which seems to help scene generation and scene continue functions.
This enhancement WAS NOT used to generate the examples below.
Special thanks to the incredible work of the model makers "meta-llama", and "bluuwhale".
Models used:
[ https://huggingface.co./meta-llama/Meta-Llama-3-8B-Instruct ]
[ https://huggingface.co./bluuwhale/L3-SthenoMaidBlackroot-8B-V1 ]
FORMULA:
slices:
- sources:
- model: G:/7B/Meta-Llama-3-8B-Instruct
layer_range: [0, 12]
- sources:
- model: G:/7B/L3-SthenoMaidBlackroot-8B-V1
layer_range: [6, 19]
parameters:
scale:
- filter: o_proj
value: 1
- filter: down_proj
value: 1
- value: 1
- sources:
- model: G:/7B/Meta-Llama-3-8B-Instruct
layer_range: [12, 25]
- sources:
- model: G:/7B/L3-SthenoMaidBlackroot-8B-V1
layer_range: [19, 32]
parameters:
scale:
- filter: o_proj
value: 1
- filter: down_proj
value: 1
- value: 1
merge_method: passthrough
dtype: float32
MERGEKIT NOTE:
Sub in the "name" of the "creator" (of the model) in place of "G:/7B" to create a mergekit file than can be used in Mergekit Google Colab.
IE: G:/7B/L3-SthenoMaidBlackroot-8B-V1 -> bluuwhale/L3-SthenoMaidBlackroot-8B-V1
Examples are created using quant Q4_K_M, "temp=0", minimal parameters and "LLAMA3" template.
Temp=0 was used to assess CORE changes between original SMB and the merge between it and Llama3 Instruct.
Below are the least creative outputs, prompt is in BOLD.
Higher quants will result in better quality.
There will also be some variance between "close" quants like Q4_K_M/Q4_K_S and Q5_K_M/Q5_K_S, so I suggest if you are going to use Q4_K_M, you also try Q4_K_S too.
Also, slightly longer / detailed prompts will result in greater creativity (as well as different prose - ie dialog, thoughts, paragraph size differences and so on).
GENERATION from "L3-SthenoMaidBlackroot-8B-V1"
The sky scraper swayed, as I watched the window in front of me on the 21st floor explode. A deafening crash echoed through the office space, followed by a sickening crunch that made my stomach turn. The sound was like nothing I'd ever heard before - it was as if the very foundations of the building were crumbling beneath our feet.
I stood frozen in terror, my heart racing so fast I thought it might burst from my chest at any moment. My mind reeled with the implications of what had just happened. A window on the 21st floor? That meant someone - or something - was up there, and whatever it was, it had just blown a hole straight through the side of our building.
As I stared in horror at the gaping wound in the wall, I couldn't help but think about all the people who could be trapped inside. The thought sent a chill down my spine, making me shiver despite the sweltering heat that seemed to permeate every inch of this godforsaken place.
I was still trying to process what had just happened when I heard the first screams. They were distant at first, but quickly grew louder as more and more people realized what was going on. The sound sent a wave of panic through me, making my legs feel like jelly.
I knew I had to get out of there - fast. But as I turned to make a run for it, I saw that the doors were jammed shut. Panic set in as I frantically tried to open them, but they wouldn't budge. It was then that I realized just how trapped we all were.
As the screams grew louder and more desperate, I knew I had to think fast if I wanted to survive this nightmare. But what could I do? The windows were blown out, the doors were stuck, and whatever had caused this chaos was still up there on the 21st floor.
I looked around frantically for any kind of escape route, but all I saw was destruction and chaos. Debris littered the floor, and the air was thick with dust and smoke. It was like a war zone in here - and we were right in the middle of it.
As I stood there, frozen in fear, I couldn't help but think about how this had happened. Had someone set off a bomb? Was it some kind of terrorist attack? Or was it something even more sinister?
The questions swirled through my mind as I heard the sound of sirens growing louder outside. Help was on the way - but would it be too late for any of us?
As I stood there, waiting for rescue, I couldn't help but wonder what other horrors lay in store for me and the rest of the people trapped inside this crumbling building. The sky scraper swayed ominously above me, a constant reminder that we were all at the mercy of whatever forces had brought about this chaos.
And as I waited, my heart pounding in my chest, I couldn't shake the feeling that things were only going to get worse from here...
GENERATION from "L3-SthenoMaidBlackroot-12.2B-V1-INSTRUCT-ULTRA-F32"
The elevator lurched to a stop, and I stumbled out onto the 21st floor. My stomach twisted into knots as I gazed up at the towering skyscraper looming above me. The sleek glass walls seemed to ripple and flex like living tissue, as if they were alive and aware of my presence.
I pushed aside the morbid thoughts and focused on the task at hand. I had been sent here for a reason, and I intended to find out what it was. My heels clicked against the polished floor as I made my way down the long, dimly lit hallway. The fluorescent lights flickered overhead, casting eerie shadows across the walls.
As I approached the door marked "Executive Suite 2101", a shiver ran down my spine. Something wasn't right here. I could feel it. I hesitated for a moment before reaching out to grasp the cold metal handle and push the door open.
The room beyond was a nightmare come to life. The walls were covered in bloodstains, and the furniture was splattered with what looked like viscous black goo. A sickly sweet scent hung heavy in the air, like rotting flesh. I gagged as my eyes scanned the scene before me.
And then, it happened. The window in front of me on the 21st floor exploded.
Glass shards rained down around me like a hailstorm of razor-sharp ice. I ducked instinctively, but not fast enough. A shard sliced through my cheek, leaving a shallow gash that began to bleed almost immediately.
I stumbled back, tripping over my own feet as the sound of shattering glass and screams filled the air. The skyscraper swayed, as if it too had been struck by some unseen force.
My mind reeled in horror as I tried to process what was happening. This couldn't be real. It had to be some kind of sick joke or a particularly vivid dream. But the pain was all too real. I clamped my hand over the wound, trying to stem the flow of blood.
I spun around wildly, desperate to find cover or escape. That's when I saw her. She stood frozen in the doorway behind me, her eyes wide with terror as she stared out into the abyss.
"Get down!" I screamed, but my voice was lost in the cacophony of chaos. The skyscraper groaned and lurched again, sending us both tumbling to the floor.
I landed hard on my backside, my head spinning as I tried to make sense of what was happening. The woman next to me was screaming now, her face contorted with fear as she clawed at the carpet.
The sound of sirens wailed in the distance, growing louder by the second. Someone - anyone - had finally noticed that something was very wrong here.
I scrambled to my feet, ignoring the burning pain in my cheek and the dizziness threatening to overwhelm me. I had to get out of here, now. I grabbed onto the woman's arm and yanked her up with me as we sprinted down the hallway, desperate to find a way back down into the depths of hell that was this building.
As we reached the elevator, it lurched back to life, swallowing us whole in its cold metal belly. The doors slid shut behind us just as the skyscraper collapsed, sending a tidal wave of debris crashing down around us.
We were trapped, buried alive beneath the rubble of our own worst nightmares. And I knew that no matter how hard we tried, there was no escape from this living hell.