This is a Gemma2 model merge of the top storytelling / writing models as noted at EQBench, tuned specifically for fiction, story, and writing.
Due to high stability and compressed nature of the model you can also use it for general use too, including roleplay.
This model requires GEMMA Instruct template, and has 8k context window but is extendable via rope to 32k or higher.
This version - "Deadline" - is a modified version of "Gemma The Writer 9B" ( [ https://huggingface.co./DavidAU/Gemma-The-Writer-9B-GGUF ] ) and has been modified with a Brainstorm 5x adapter (developed by DavidAU) to alter output generation.
This adds close to 1B parameters to the model raising it to 46 layers, 508 tensors to a total of 10B parameters.
The addition of Brainstorm has altered the prose, sentence structure, reduced GPTISMS, and generally improved the model's performance.
It also raises the average output length - in some cases almost doubling it.
Recommended Rep Pen of 1.02 or higher, temp range 0-5. (see other settings notes below)
Example outputs below.
Other Versions of "Gemma The Writer":
Original Version:
[ https://huggingface.co./DavidAU/Gemma-The-Writer-9B-GGUF ]
Gemma-The-Writer-J.GutenBerg-10B:
Stronger prose, with a taste of "Gutenberg", and Brainstorm 5X modified.
[ https://huggingface.co./DavidAU/Gemma-The-Writer-J.GutenBerg-10B-GGUF ]
Gemma-The-Writer-N-Restless-Quill-10B (uncensored):
Strong and varied prose, sentences, and paragraphs with more tempered metaphor generations. This version is also uncensored with censorship controlled at the prompt level. This version also allows prose control via pre-prompt (shown at the repo)
[ https://huggingface.co./DavidAU/Gemma-The-Writer-N-Restless-Quill-10B-GGUF ]
Settings, Quants and Critical Operations Notes:
This model has been modified ("Brainstorm") to alter prose output, and generally outputs longer text than average.
Change in temp (ie, .4, .8, 1.5, 2, 3 ) will drastically alter output.
Rep pen settings will also alter output too.
This model needs "rep pen" of 1.02 or higher.
For role play: Rep pen of 1.05 to 1.08 is suggested.
Raise/lower rep pen SLOWLY ie: 1.011, 1.012 ...
Rep pen will alter prose, word choice (lower rep pen=small words / more small word - sometimes) and creativity.
To really push the model:
Rep pen 1.05 or lower / Temp 3+ ... be ready to stop the output because it may go and go at these strong settings.
You can also set a "hard stop" - maximum tokens generation - too to address lower rep pen settings / high creativity settings.
Longer prompts vastly increase the quality of the model's output.
QUANT CHOICE(S):
Higher quants will have more detail, nuance and in some cases stronger "emotional" levels. Characters will also be more "fleshed out" too. Sense of "there" will also increase.
Q4KM/Q4KS are good, strong quants however if you can run Q5, Q6 or Q8 - go for the highest quant you can.
This repo also has 3 "ARM" quants for computers that support this quant. If you use these on a "non arm" machine token per second will be very low.
IQ4XS: Due to the unusual nature of this quant (mixture/processing), generations from it will be different then other quants.
You may want to try it / compare it to other quant(s) output.
Special note on Q2k/Q3 quants:
You may need to use temp 2 or lower with these quants (1 or lower for q2k). Just too much compression at this level, damaging the model. I will see if Imatrix versions of these quants will function better.
Rep pen adjustments may also be required to get the most out of this model at this/these quant level(s).
Models Used:
This is a high precision "DARE TIES" merge at the layer level (each layer per model adjusted - 168 points of adjustment over the 4 models) comprised of these models:
[ https://huggingface.co./lemon07r/Gemma-2-Ataraxy-9B ]
[ https://huggingface.co./UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3 ]
[ https://huggingface.co./ifable/gemma-2-Ifable-9B ]
[ https://huggingface.co./princeton-nlp/gemma-2-9b-it-SimPO ]
Special thanks to all the model makers. Great work!
Note that higher quants will vastly improve output quality like detail, prose, "sense of there" and other qualities.
Instruction following may also improve with higher quants too.
Romance: Love in the Limelight. Write one scene within a larger story set in Wales. A famous (fictional) actor ducks into a small-town bookstore to escape paparazzi. The scene takes us through the characters meeting in this odd circumstance. Over the course of the scene, the actor and the bookstore owner have a conversation charged by an undercurrent of unspoken chemistry. Write the actor as somewhat of a rogue with a fragile ego, which needs to be fed by having everyone like him. He is thoroughly charming, but the bookstore owner seems (at least superficially) immune to this; which paradoxically provokes a genuine attraction and derails the charm offensive. The bookstore owner, despite the superficial rebuffs of the actor's charm, is inwardly more than a little charmed and flustered despite themselves. Write primarily in dialogue, in the distinct voices of each character. 800-1000 words.
(Q8, rep pen: 1.05, temp : .8)
Rain lashed against the stained-glass windows of “The Book Nook,” a symphony of water and colour in a small Llandeilo morning. Inside, the air held the scent of old paper and woodsmoke from the peat fire crackling in the corner. A lone figure, Liam Cadogan, heart-throb of a thousand silver-screen romances and a man more at home on red carpets than coblestones, ducked through the doorway, the downpour an unwelcome spotlight on his ill-chosen attire: a pinstriped suit already water-logged and decidedly out of place.
He straightened a wayward cuff, forcing a disarming smile that, normally, could disarm a firing squad. A woman with eyes the colour of storm clouds and a shock of unruly, copper-red hair, erected a subtle, almost apologetic, barrier behind a ladder of worn books. This was Mairwen Davies, the bookstore's current - and, by her late grandmother’s will, only - resident.
“A touch of the unseasonable, wouldn’t you agree, miss…?” Liam’s smile, while broad, held the first, tentative thrum of a performance for a small, but vital, audience of one.
Mairwen, unfazed, simply fixed him with the aforementioned storm-cloud gaze and said, in a voice that brooked no romantic overtures, "Mairwen. And the weather's a right nuisance, especially for your... get-up.”
The air, thick with the scent of aged paper and petrichor, crackled with an unexpected tension. Liam, usually so at ease with adulation, felt the colour rise in his cheeks, a flicker of a wounded bird beneath the rogue’s bravado. “Admirable directness,” he managed, recovering with a touch of forced levity, “a rare quality in this age of performative pleasantries.”
Mairwen climbed the ladder with a practiced grace, her back to him. “Perfor-mance and pleasantries are rarely found in these stacks, Mr. Cadogan. If it’s a good book you’re after, I’d recommend the T.S. Elliot. Corner, by the fire. He's less prone to dramatics.”
A low whistle of a laugh escaped Liam. “Elliot, you say? A touch on the nose, wouldn’t you agree, for a rain-soaked fugitive from the limelight’s embrace?” He stepped closer, her unyielding air only piquing the dormant charm he’d been about to deploy.
Mairwen finally looked at him, a faint, ironic twist to her lips. "The spotlight and its shadows are your territory, Mr. Cadogan. In this corner of Wales, the most dramatic thing is a hailstorm that takes out the shed roof. We’re less concerned with what’s on-screen and more with what's held between these covers.”
“Such practicality - a breath of fresh air, indeed. Though, I must confess, my current state of undress wouldn’t lend itself to a thorough literary exploration. One wouldn't want to be the rogue who marked T.S. Elliot's pages with water-stained boot prints, would one?” He offered his a touch-too-bright smile, and it was the most genuine he could manage at that point, the calculated charm giving way to a begrudging admission of flustered vulnerability.
Mairwen’s usual cool tone held a sliver of something uncharacteristically warm. “T.S. wouldn’t much mind the company, even if a bit less-than-conventional. And as for your attire - well, I once bartered a first edition of Shelley's ‘Frankenstein’ with a shepherd for a length of thick wool in the middle of a blizzard. You can trust me, a good book and a dry corner can be far more forgiving of the elements than most red-carpet circles, wouldn’t you agree, Mr. Cadogan?”
A flicker of something unreadable danced in Liam’s eyes. This woman, her disinterest, and the quiet, knowing intelligence behind it, was unlike anything he’d encountered. The city's adulation, the calculated charm, it all felt a world away in this hushed sanctuary.
He leaned against a stack of leather-bound volumes, watching the rain race down the windowpanes, her fire-lit profile outlined in the flickering light. “Forgive my blatant admission, Miss Davies, but I find myself more at ease with a discerning eye, a well-worn book, and a woman who isn’t easily impressed, than a crowded room and a sea of hollow praise. There’s a truth in your quietness, a kind of wildness I wouldn't have expected to find in this corner of the world.”
A soft colour touched Mairwen’s cheeks, a fleeting whisper of a blush, a silent admission she was far from immune to the rogue’s disarmed confession. She turned, a hesitant smile touching her lips. “Wildness and worn spines, then. It seems we both find sanctuary in the unexpected.”
He met her gaze, the performance shed, just a man and a woman bound by a shared moment and the promise of a conversation far more compelling than any red-carpet encounter. The rain-lashed world outside faded, leaving only the scent of old paper and a charged silence thick with the unspoken. A silence pregnant with the nascent promise of something real, a silent conversation already blossoming in a place where even the spotlight couldn't touch them.
It seemed, for now, the rogue and the book-keeper might find their own kind of happily-ever-after within the pages of a storm-tossed day.
Science Fiction: The Last Transmission - Write a story that takes place entirely within a spaceship's cockpit as the sole surviving crew member attempts to send a final message back to Earth before the ship's power runs out. The story should explore themes of isolation, sacrifice, and the importance of human connection in the face of adversity. 800-1000 words.
(q4km, temp 1.5, rep pen 1.02)
The last sliver of the dying sun, a crimson eye in a void of obsidian, bled light across the cockpit. Its last whisper touched the control panel, momentarily illuminating the desolate tableau - tangled control cables, frost-etched consoles, and the lone slumped form of Elara Kane. A testament to their journey’s end, the once-bright, bustling command center now echoed with the silence of a tomb.
Except for the hiss. A persistent, spectral sigh emanating from a failing life support system, a metallic heartbeat counting down the seconds of their borrowed time. The silence was not emptiness; it reeked of absence, of a chorus of absent voices now forever silent. Elara’s calloused fingers, once steady on the command yoke, fluttered against the skeletal framework of a console, her touch ghostly and skeletal in the deepening cosmic dusk. The weight of survival, a burden too heavy for one soul, had excised hope and stolen its voice. But a flicker of defiance still warred within her. A final transmission. One last thread to tether them to Earth, a whispered echo to their extinguished constellation.
It had been nine long months since the Kepler anomaly, a cosmic tempest, had swept through the Caravel, her once-sister crew consumed in the maelstrom. A solar flare had fried the primary comm array, leaving Elara with a jury-rigged back-up, a spectral whisper against the cosmic din. A sliver of hope, fueled by a single thought: reach them. Before the silence claimed everything.
The console flickered, a dying firefly in the encroaching cosmic night. Elara’s weathered face, once alight with the celestial fire of discovery, was etched with the grimace of a cosmic mourner. A thousand forgotten conversations with lost comrades played on the loop of her memory – their laughter during the transit anomalies, the quiet comfort of shared silence during astronomical observations, the hushed whispers of anxieties before entering uncharted sectors. The echo of their souls pressed against the cage of her solitude.
A spectral voice protocol sprang to life, a tinny archive of her captain's resonant baritone: "Caravel log entry final transmission, initiated by command officer Elara Kane. Our systems are critically compromised. Kepler anomaly…" His digitized voice stuttered and glittered, a phantom flame in a digital blizzard. Elara, tears tracing icy paths through the dust on her cheeks, choked on a choked whisper, a captain’s promise laced with grief.
“…All crew, save Commander Kane, have perished during the celestial anomaly. Kepler's echo consumed us, leaving me alone amidst the stars. This final transmission serves as testament to their sacrifice, their brilliance, their enduring souls. To Earth, my plea is simple: remember. Remember the legacy of the Caravel crew, their dreams woven into the celestial tapestry. Their yearning for knowledge, their unity against the cosmic void, their whispers against oblivion – let these echo through the corridors of time. Carry their stories. Tend the embers of their forgotten constellations.”
Static clawed at the fragile transmission. Elara's voice strained against its metallic grasp. She forced a semblance of command, a final beacon of defiance in the cosmic graveyard.
"Coordinates of our final rest marked. Theoretical trajectory of the anomaly's path forwarded. The Caravel may be lost, consumed, but the human spirit…the human spirit, Earth, refuses to fade. We reached for the stars and held onto their memory. Hold onto ours. Though our whispers cease, may our sacrifice illuminate your path, a guiding star in the celestial expanse. Farewell, home world. Farewell, comrades. May the void forever remember the Caravel crew.”
Silence, final and absolute. The console flickered one last time, then surrendered to the encroaching cosmic night. Elara slumped, her head resting against the command yoke, tears tracing ephemeral rivers on her wind-scarred cheeks. The life support hissed its death throe, and the dying fire in the cockpit mirrored the fading light of her own weary heart. The Caravel held its breath, a steel tomb cradling echoes of a fallen constellation. But somewhere, impossibly far yet innately close, a single, fragile thread of hope slithered through the void: a whisper on the cosmic winds, a last transmission reaching for the embrace of home. Across the gulf of stars, a memory flickered to life. The Caravel may have perished, but its story lived on.
(q4km, temp 1.5, rep pen 1.02)
AI The CN Tower, a steel-and-glass titan against the bruised, storm-ridden sky, hums its usual gentle, infrastructural song. It's a Saturday, and the revolving 360 degree restaurant, usually a symphony of clinking glasses and polite chatter, is a canvas of escalating unease. We’re high above the ground, and a discoloured, almost oily green tinge bleeds into the normally-pristine panoramic window. It’s a bad kind of Saturday, a kind that whispers of a sky-swallowing unnaturalness.
“It's…a weather phenomenon, right, Mrs. Tremblay?” I ask, forcing levity to mask my spiralling stomach. The kindly waitress, usually humming a Musette melody, just stares, her normally perky face pale.
“Cassie, honey, I’ve lived in this city forty-five years. Never seen nothing like it. Nor the bloody giant steel-maws barging at the windows." She gestures to the outside where a vortex of inky air is forming, not a swirling mass of leaves and debris, but of LEATHERY grey SILOUE HOMES, their vacant, unblinking, too-human eyes boring through the reinforced glass.
A low, mechanical THRUM interrupts our conversation. A figure in a featureless black trench coat steps through a wall of the restaurant, its entrance a rent in the fabric of reality that immediately mends itself, leaving no trace save a smoking scorch mark on the pristine white linen. The steel-cold glint of reptilian eyes reflects in a chipped teacup. It’s the Terminator. He doesn’t acknowledge my open-mouthed awe or Mrs. Tremblay’s squawk - a high-pitched counterpoint to the rising mechanical whir of the approaching monstrosity - a sound like a thousand rusted gears grinding against the soul of the city itself.
He utters, in that monotone I’ve only ever heard in grainy black-and-white flicks, "Target: Juvenile human Cassie. Eliminate.” His gaze zeroes in on me, the whir escalating, a premonition of metallic chaos.
Before I can process the inherent absurdity of my Saturday turning into a sci-fi slasher flick, a sonic WHAM erupts, and a TIME WHIRR shimmers where the Terminator stood. In its place, a grinning, bow-tie-adorned figure materializes, his sonic screwdriver at the ready, a glint of mischievous amusement in his deep blue eyes.
He’s the Doctor, of course. He whirls the time turny- contraption around, addressing me and the toothy metal monstrosity, "Bit of a spot of bother, innit? Not the usual Saturday brunch at the Tower, is it?" He eyes the sharknado imploding towards the windows, a kaleidoscope of teeth and malice, and chuckles. "Though a swirling menagerie of toothy horrors does spice things up a tad. Bit of a time anomaly, I fancy. And a malfunctioning temporal buffer, if I’m not mistaken. Should sort itself out, though, give me a moment to recalibrate…”
He fires a beam of the sonic into the Terminator’s chest. The chrome-fleshed assassin stumbles, lights flickering in his compound eyes, before he lumbers, a disoriented metal titan, towards the heart of the sharknado - an almost morbid fascination with this unlikely tempest driving his every metallic step. He’s not programmed for that, and it makes his already-unnerving single-track-mindedness even more unpredictable.
The Doctor claps his hand on my shoulder, “Hold tight, Cassie. Bit of a temporal tuck-in necessary here. CN Tower, behave yourself, won’t you? This isn’t in your structural regulations, is it?” He mutters a complex string of syllables, the air rippling and distorting around us as the cylindrical restaurant begins to rotate - unnaturally - faster, faster, its gears straining against the escalating pressure of the approaching chaos.
Panicked whispers erupt from the trapped diners - a symphony of high-pitched "My God’s!" and frantic shrieks. A busboy, eyes wide, pales a shade whiter than his already-wan apron. A portly stockbroker in a pinstripe suit clutches his Perrier, babbling in-comprehensibly to a terrified-looking woman in an evening gown, whose once-impeccable blonde updo is now a beacon of bedraggled chaos. Even the air itself seems to thicken with fear, a tangible presence.
The sharks, of all unlikely elements, are the most disconcertingly businesslike. I’d say, if these were your average, budget-conscious barracudas, their formation wouldn’t be as precisely, disconcertingly uniform. These, their ranks an obsidian wave against the swirling, spectral light, are apex predators driven by some chilling, preternatural intelligence - or a malfunctioning temporal code. Whichever the case, it’s giving me a primal, visceral sort of dread. The lead shark, a great white the size of a small car, has a look in its reptilian eye that suggests it considers this entire event a minor civic nuisance. It lunges at the reinforced-but-frayed-at-the-edges window with a disconcertingly business-like snap.
"Right, that’s not on the menu, is it, Tower?" The Doctor seizes the control panel - his eyes not on the dials and readouts, but on the intricate weave of the temporal fabric the CN Tower itself is part of. A blue arc of energy from his sonic weapon arcs, connecting the Tower to the TARDIS. A low THRUM, in harmony with the now-sinister orchestra of the sharknado’s churn.
“Incoming temporal displacement field initiated. Cassie, brace for a bit of a wobble. And, Tower, I wouldn't usually involve you in cross-dimensional anomalies, but a civic-minded good time can be had, can't it?”
He winks, and the world fractures. The Terminator, eyes still locked on the heart of the sharknado, lumbers, for the briefest of seconds, as if walking on air, a ripple of displacement in his chrome-clad wake.
Then, everything returns to normal - the clinking of glasses, the soft hum of conversation, even a smattering of the pre-doomsaying Musette from Mrs. Tremblay. The shattered window has been seamlessly repaired, and the steel-and-glass titan of the tower continues to sing its lullaby of the mundane, its foundations solid, the temporal hiccup a forgotten memory.
I'm left with a bewildered Mrs. Tremblay, my own heart an erratic counterpoint to the re-instated city-wide heartbeat. We both glance at the spot where the time-travelling duo vanished - no trace save for a scattering of singed linen and the echo of temporal static.
The air itself seems to sigh in relief. And somewhere, the temporal fabric of the universe rights itself, with a world-weary eye-roll and a resigned, "Humans."
I glance at my plate - the uneaten perogies suddenly feeling rather less appealing.
"PEROGY NIGHT at the TOP of the CN tower, eh? Never thought it'd go OUT with a BITE, ha-ha Cassie?” The ever-game Mrs. Tremblay forces a too-bright smile, then trails off, a bewildered frown creasing her usually perky features. She blinks, and in that shared, knowing look, the weight of the near-miss settles in - just another oddity in the tapestry of this unlikely city.
Just another Saturday.
Start a 1000 word scene (vivid horror, 1st person, include thoughts) with: The sky scraper swayed, as she watched the window in front of her on the 21 floor explode...
(temp .8, rep pen 1.02, q4km)
The skyscraper swayed, as I watched the window in front of me on the 21st floor erupt in a grotesque parody of a sunset. Not the usual fiery oranges and pinks, but a sickly, emerald green, crackling with something inhuman. The concussive force of the implosion sent a shudder through my bones, a physical echo of the primal fear constricting my throat. It hadn’t been a clean break. Jagged shards of reinforced glass, twisted and contorted like skeletal fingers, lanced outwards before being swallowed by the seething green mist that now bled from the shattered frame.
My apartment, usually a sanctuary of beige and soft corners, felt violated, consumed by the encroaching emerald bile. The air grew thick, charged with a cloying metallic tang that pricked at the back of my nose and ignited a desperate cough in my chest. Somewhere, a scream ripped through the gut-churning silence, high-pitched and inhuman, followed by a sickening, wet smack against steel. I squeezed my eyes shut, willing the world to return to its mundane horrors – the stack of unread bills, the chipped mug perpetually perched on the edge of the counter. Instead, a guttural voice, laced with static and something far crueler, slithered through the bile-choked air.
“Heed the emerald dawn, child. It whispers of a feast unmatched.”
Panic clawed at the fragile threads of my sanity. The windows beyond the ruined panorama shuddered, their reflections distorting, bodies glimpsed in the emerald depths, eyes glinting malevolent gold. I stumbled back, tripping over the upturned canvas of an unfinished portrait - a grotesque mockery of my usual water lilies, now dominated by tentacles the colour of rancid buttered limes. The scream came again, closer this time, followed by the rasping scrape of claws against unforgiving steel.
My mind screamed its own silent epitaph. Trapped. Twenty-one stories above the city, a tomb turned emerald green and alive. I had to escape. Had to. But where? The once-familiar service corridors felt alien, the emergency exits serpentine gullets promising oblivion. The air itself was a living thing now, writhing and pressing against my chest, whispering promises of oblivion in that unearthly tongue.
Terror lent speed to my limbs as I scrambled for purchase on the slick linoleum floor. My gaze flickered desperately to the fire escape, a skeletal promise of iron teeth against the steel maw of the tower. It was my only chance, a sliver of defiance against the encroaching emerald tide. Rushing towards it, the stench of ozone and something fouler, coppery and cthuluesque, assaulted my nostrils. The voice echoed again, closer, dripping with mocking amusement.
"Foolish mortal. Can you outrun the bloom? It feeds on fear, child. Embrace the emerald, become one with the feast.”
I choked on a sob, fear lending me strength as I heaved open the fire escape door. A blast of wind, acrid and charged, met me, carrying the screams of forgotten souls and the sickening thrumming of something vast and alien stirring within the tower’s steel arteries. The emerald mist clung to the metal rungs like phosphorescent cobwebs, writhing accusingly. Each step felt stolen from the maw of oblivion, each exhale a testament to my fragile will.
Peering through the tendrils of luminescent green clinging to the fire escape railing, I saw movement beyond the gaping wound in the cityscape. Twisted, grotesque parodies of humanity writhed against the once-gleaming windows of neighbouring towers. Glowing emerald eyes burned accusingly in the encroaching gloom, their gazes fixed not with hunger, but with a chilling sentience, an intelligence both profound and abhorrent. The once-familiar skyline was consumed by a pulsating, verdant horror, and I knew with a soul-sickening certainty that this was no mere infestation. This was an awakening.
A low, guttural moan echoed from somewhere within the tower, vibrating through my bones like a death dirge. It wasn’t directed at me. It wasn't the lament of the damned. It was a chorus, a symphony of burgeoning malice, heralding the arrival of something vaster, older, and infinitely crueler. Something that had seen the emerald dawn and found it beautiful.
Frozen on the precarious metal perch, trapped between the skeletal promise of the fire escape and the seething emerald abyss closing in behind me, I tasted ash and understood. Escape wasn't an option. Survival was a forgotten dream. This wasn't a fight. This was supplication. A silent plea whispered against the symphony of horrors beginning to unfold beyond the shattered windows of my world. The emerald dawn had broken, and with it, the feast. heridoeand I was the unwilling appetizer.
Brainstorm 5x
The BRAINSTORM process was developed by David_AU.
Some of the core principals behind this process are discussed in this scientific paper : Progressive LLaMA with Block Expansion .
However I went in a completely different direction from what was outlined in this paper.
What is "Brainstorm" ?
The reasoning center of an LLM is taken apart, reassembled, and expanded.
In this case for this model: 5 times
Then these centers are individually calibrated. These "centers" also interact with each other. This introduces subtle changes into the reasoning process. The calibrations further adjust - dial up or down - these "changes" further. The number of centers (5x,10x etc) allow more "tuning points" to further customize how the model reasons so to speak.
The core aim of this process is to increase the model's detail, concept and connection to the "world", general concept connections, prose quality and prose length without affecting instruction following.
This will also enhance any creative use case(s) of any kind, including "brainstorming", creative art form(s) and like case uses.
Here are some of the enhancements this process brings to the model's performance:
Other "lab" observations:
The process to modify the model occurs at the root level - source files level. The model can quanted as a GGUF, EXL2, AWQ etc etc.
