sometimesanotion/Lamarck-14B-v0.4-Qwenvergence

Lamarck 14B v0.4 Qwenvergence: it's a big step up for Lamarck in terms of quality. All the same ingredients are involved as in previous releases of Lamarck; they are more effectively combined. This model features slightly improved reasoning from 0.3, but the multi-language and prose are greatly improved.

Merge Details

This model was initialized from model_stock, and refined from there. No fine-tuning, or use of models apart from those listed as the contents of Qwen2.5-14B-Qwenvergence-model_stock except for a very mild application of huihui-ai/Qwen2.5-14B-Instruct-abliterated-v2 which does not seem to have removed most of the refusals.

Models Merged

Top influences: These ancestors are in the Qwenvergence model_stock, reinforced in later steps:

• arcee-ai/Virtuoso-Small - A brand new model from Arcee, refined from the notable cross-architecture Llama-to-Qwen distillation arcee-ai/SuperNova-Medius. The first two layers are nearly exclusively from Virtuoso. It has proven to be a well-rounded performer, and contributes a noticeable boost to the model's prose quality.

• CultriX/SeQwence-14B-EvolMerge - A top contender on reasoning benchmarks.

• VAGOsolutions/SauerkrautLM-v2-14b-DPO - This model's influence is understated, but aids BBH and coding capability.

• v000000/Qwen2.5-Lumen-14B - A leading influence for prose quality.

Prose added:

The prose quality has taken a leap, no doubt also to the way EVA-UNIT-01/EVA-Qwen2.5-14B-v0.2, sthenno-com/miscii-14b-1028, oxyapi/oxy-1-small, and underwoods/medius-erebus-magnum-14b were applied.

Configuration

The following YAML configurations were used to initialize and finalize this model:

name:                Qwenvergence-model_stock
merge_method:        model_stock
base_model:          Qwen/Qwen2.5-14B
tokenizer_source:    base
parameters:
  int8_mask:         true
  normalize:         true
  rescale:           false
models:
  - model:           allura-org/TQ2.5-14B-Sugarquill-v1
  - model:           oxyapi/oxy-1-small
  - model:           sthenno-com/miscii-14b-1028
  - model:           underwoods/medius-erebus-magnum-14b
  - model:           EVA-UNIT-01/EVA-Qwen2.5-14B-v0.2
  - model:           CultriX/SeQwence-14B-EvolMerge
  - model:           arcee-ai/Virtuoso-Small
  - model:           VAGOsolutions/SauerkrautLM-v2-14b-DPO
  - model:           v000000/Qwen2.5-Lumen-14B
dtype:               bfloat16
out_dtype:           bfloat16
---
# Experimental merge methods involving above models
---
name:                Lamarck-14B-v0.4-Qwenvergence
merge_method:        ties
base_model:          sometimesanotion/lamarck-14b-base
tokenizer_source:    base
parameters:         
  density:           1.00
  weight:            1.00
  int8_mask:         true
  normalize:         true
  rescale:           false
models:
  - model:           merges/Qwen2.5-14B-Qwenvergence-slerp
    parameters:
      weight:        1.00
      density:       1.00
  - model:           arcee-ai/Virtuoso-Small
    parameters:
      weight:        1.00
      density:       1.00