Lunzima/NQLSG-Qwen2.5-14B-MegaFusion-v8
Text Generation
•
Updated
•
187
•
2
sometimesanotion PRO
sometimesanotion
AI & ML interests
Agentic LLM services, model merging, finetunes, distillation
Recent Activity
new activity
4 days ago
wanlige/li-14b-v0.4-slerp0.1:Fusion vs. SLERP?
updated
a model
5 days ago
sometimesanotion/Lamarck-14B-v0.7
replied to
their
post
6 days ago
I have tracked down a blocker preventing Lamarck releases to a della_linear bug in newer mergekit versions.
If you use slices in della_linear merges that have multiple models - as you'd expect of a merge! - an attempt to load the output model in torch will get you:
```
ValueError: Trying to set a tensor of shape torch.Size([1, 5120]) in "weight" (which has shape torch.Size([5120])), this looks incorrect.
```
This strategy was key to Lamarck v0.6 and v0.7's success. Their merge recipes haven't been working with newer mergekits.
These work:
```yaml
models:
- model: sometimesanotion/Qwen2.5-14B-Vimarckoso-v3
- model: sthenno-com/miscii-14b-0218
```
```yaml
slices:
- sources:
- { layer_range: [ 0, 2 ], model: sometimesanotion/Qwen2.5-14B-Vimarckoso-v3 }
- sources:
- { layer_range: [ 2, 6 ], model: sometimesanotion/Qwen2.5-14B-Vimarckoso-v3 }
```
This does not:
```yaml
slices:
- sources:
- { layer_range: [ 0, 2 ], model: sometimesanotion/Qwen2.5-14B-Vimarckoso-v3 }
- { layer_range: [ 0, 2 ], model: sthenno-com/miscii-14b-0218 }
- sources:
- { layer_range: [ 2, 6 ], model: sometimesanotion/Qwen2.5-14B-Vimarckoso-v3 }
- { layer_range: [ 2, 6 ], model: sthenno-com/miscii-14b-0218 }
```
@Crystalcareai, do you know of any work on this? Will @arcee-ai need a detailed report? These della_linear recipes used to work. Overall, thank you for all the cool work, I hope to get this fixed!
Organizations
sometimesanotion's activity
liked
a
model
7 days ago
liked
a
model
8 days ago
liked
6
models
9 days ago
microsoft/Phi-4-mini-instruct
Text Generation
•
Updated
•
102k
•
321
YOYO-AI/Qwen2.5-14B-YOYO-V4-p2
Text Generation
•
Updated
•
49
•
2
Lunzima/NQLSG-Qwen2.5-14B-OriginalFusion
Text Generation
•
Updated
•
52
•
2
Lunzima/NQLSG-Qwen2.5-14B-MegaFusion-v8.7
Text Generation
•
Updated
•
77
•
3
wanlige/li-14b-v0.4-slerp0.1
Text Generation
•
Updated
•
224
•
5
CultriX/Qwen2.5-14B-GeneralReasoning
Text Generation
•
Updated
•
33
•
2
liked
a
model
11 days ago
liked
4
models
13 days ago
liked
4
models
14 days ago
liked
a
model
15 days ago
liked
2
models
17 days ago
