Research & sources

How the research was done, model facts, alternatives to llama.cpp, open points and the list of sources.

Approach

The research ran along several strands: official model card and configuration, unsloth documentation of the quants, llama.cpp pull requests and issues on model support and on MTP, community reports on Strix Halo (forums, blogs, fork documentation), and alternatives to llama.cpp. Every statement was checked, where possible, against the source code of the local builds and against our own measurements. The short version is in the project file docs/RESEARCH.md.

Model

• 125 billion parameters, 6 billion active, plus a 51-billion n-gram embedding and a 4-billion MTP head. 48 layers: twelve times three gated DeltaNet layers plus one sparse-attention layer. 512 experts, 10 active plus 1 shared.
• Context 262144 native, up to 1 million with YaRN.
• Thinking on by default; reasoning_effort xhigh (default), medium, low; enable_thinking to switch it off.
• Sampling: thinking temp 1.0 / top_p 0.95 / top_k 20; non-thinking temp 0.7 / top_p 0.8 / presence 1.5.

Alternatives to llama.cpp (as of September 2026)

None of them runs this model on a single Strix Halo: vLLM is validated only on MI355X and all vLLM checkpoints leave the embedding table in BF16 (126 to 173 GB). SGLang rejects gfx1151. ik_llama.cpp supports the model, but not ROCm. Ollama offers only MLX variants. LM Studio and Lemonade use upstream llama.cpp without MTP. The real competition are Strix Halo forks of llama.cpp.

Quality benchmarks

Which benchmarks (KL divergence, Aider Polyglot, HumanEval+, IFEval, SWE-bench Verified Mini, Terminal-Bench 2.0, DeepSWE) are suitable for the quant comparison within an 8-hour budget is set out, with effort estimates and example commands, in docs/QUALITY-BENCHMARKS.md.

Open points

• The EasiiX Q8_0 MTP head (4 GB) is said to have higher acceptance; not tested.
• A rebase of the MTP patch onto llama.cpp with PR #28123 (recurrence rollback) could speed MTP up further.
• Vulkan with the patched sources has not been measured.
• Governor “performance” and tuned “accelerator-performance” were not measured; all numbers were produced with powersave/balanced.

Sources

1. Qwen3.8-Flash-Next model card
2. config.json of the model
3. unsloth: quants, KLD table, MTP
4. unsloth GGUF repository and MTP README
5. dzannotti MTP head and patch
6. EngramHalo.cpp (branch strix-halo-qwen4exp, docs/strix-halo)
7. Independent reproduction on a 128 GB Strix Halo
8. llama.cpp PR #27742 (qwen4exp), #27836 (MTP, draft), #28123 (rollback), #28243 (unsloth MTP), #21038 (KV rotation), #25992 (multi-slot bug), #22629 (cache-ram), #28136 (lazy-mode on-direct)
9. KV cache quantization: KLD comparison
10. Vulkan vs. ROCm on Strix Halo
11. rocWMMA-FA on Strix Halo
12. AMD: Strix Halo system optimization
13. kyuz0 Strix-Halo-Toolboxes
14. vLLM recipe (MI355X only)
15. MTP tuning on high-end GPUs