Qwen3.8-27B for agentic work: speculative decoding and reasoning effort

How the daily-driver Qwen3.8-27B preset got its current shape: MTP speculative decoding disqualified by a decode-degradation bug, the draftless ngram-mod found to degrade the same way, and the xhigh-vs-medium reasoning-effort question settled with a sensor-gated task suite. Picks up where the bench numbers left off; written for anyone serving this model to an agent harness rather than a chat box.

Provenance. The speculative-decoding and reasoning-effort numbers were measured 2026-08-18 to 2026-08-19 on an RTX 5090 (32 GB), llama.cpp b10362 then b10472, Qwen3.8-27B Q4_K_M (unsloth GGUF), 196608-token context x 2 slots, model-card sampling (temp 1.0, top_p 0.95, top_k 20, min_p 0). The template, quant and thinking-budget sections were measured 2026-09-02 on the same GPU and pin, by then on UD-Q4_K_M at 262144-token context x 1 slot. The NInfer engine section was measured 2026-09-06 to 2026-09-07 on the same GPU: NInfer at upstream commit 487f897, the Qwen3.8-27B NVFP4 .ninfer artifact, 262144-token context on one lane, fp8 KV, --spec mtp --draft-tokens 3 --lm-head-draft. The accuracy numbers (HumanEval, BFCL, the task-suite parity run) were measured 2026-09-08 on the same engine and artifact; the context-ceiling finding the same day. Every claim below is measured unless marked community-reported; the upstream bugs are cited to their llama.cpp issues.

TL;DR

MTP (--spec-type draft-mtp) gives +15% generation on short bursts, then degrades to ~0.5 tok/s within about 10 minutes of agentic task traffic. Restart restores it; the degradation returns. Not usable for loops on llama.cpp - the qualifier matters, see the engine section below.
The same model’s MTP sustains on a different engine. NInfer holds 45-64% draft acceptance across a whole multi-milestone build-out with decode flat from 30K to 100K of context, which places the failure in llama.cpp’s speculation path rather than in the model’s recurrent state.
ngram-mod (draftless, a ~16 MB hash pool over the server’s own history) measured +93-100% generation on a cold pool - and then degraded identically to MTP under real task-suite churn. The only stable config for agentic loops on this model is no speculation at all.
reasoning_effort = "medium" is quality-neutral on the task suite (the ceiling tasks fail identically at both efforts) and removes the thinking-binge lottery: xhigh once spent 1308.5 s on a task that also completed in 19.7 s.
Bump your llama.cpp pin: b10472 fixes an abandoned-stream slot-parking bug that freezes slots under harnesses that timeout-kill their clients.
Capping the thinking budget is the obvious next lever and the wrong one: on real work the p99 generation is 9201 tokens and nothing has exceeded 9564, so any cap low enough to bite cuts reasoning that was converging.
The GGUF your config names can stop existing. Unsloth deleted every plain K-quant of this model on 2026-08-19 and re-uploaded a dynamic-quant lineup; the preset here kept naming a file that no longer existed for two weeks.

Which config for which workload

Workload	spec_type	effort	Why
Agentic loops, benches, unattended runs	none	medium	both speculators degrade under churn; medium kills binges
Interactive coding/chat, babysat	`draft-mtp` or `ngram-mod`	xhigh	+15% gen (MTP) or +93% cold (ngram) while the server is fresh; watch for drift
Anything long-lived on b10362	none	-	slot parking on abandoned streams; upgrade
Agentic loops, if you can run NInfer	`mtp` (draft 3)	medium	speculation holds under churn on that engine; Qwen-only, sm_120a-only

The MTP degradation

The symptom: decode collapses from ~60 tok/s to ~0.5 tok/s, while prompt processing stays fast (~1500 tok/s), MTP draft acceptance stays healthy (0.79-0.99), and the GPU idles at ~100 W draw with 100% “utilization” (idle-clock). Restarting the server restores full speed; the degradation returns after roughly 10 minutes of agentic task churn. Reproduced on both b10362 and b10472.

The control: the identical suite with speculation disabled sustains 55-66 tok/s indefinitely - one 5433-second, 8-iteration task ran at full speed past request id 216000. The failure lives in the speculation path, not the harness or the model - and it is not MTP-specific, as the next section shows.

Fresh-server synthetic requests never reproduce it. Long prompts (53k tokens), long generations (3500 tokens), streaming, tools, cache-prefix reuse, and long tool-call arguments all decode at 60-104 tok/s. Only a lived-in server crawls, which is why every short benchmark of this model’s MTP looks fine.

Upstream matches both halves of the symptom: acceptance decay over server lifetime with restore-on-restart¹, and draft-context corruption after mixed long/short prompt sequences on this exact model². Both issues open as of 2026-08-19.

MTP survives in one variant preset (qwen38-xhigh) for babysat interactive work, where sessions are short and a restart is cheap.

The slot-parking bug, fixed by the pin bump

Separately, on b10362: a client that abandons a streaming request (a loop iteration killed by its timeout wrapper, a dead curl) leaves its slot frozen at is_processing=true with n_decoded static, and the slot never frees. With parallel 2, two abandoned streams park the whole server. Verified fixed in b10472: the slot releases about 20 seconds after the client disconnects. If you are on b10362 and run any harness that kills clients, this alone justifies the bump.

ngram-mod: same degradation, different drafter

ngram-mod keeps an n-gram -> next-token pool keyed by a linear congruential generator (LCG) hash (~16 MB, constant memory) and drafts from it; the pool is shared across server slots and accumulates accepted tokens over the server lifetime³. No draft model, no draft context - and initially adopted here on exactly that reasoning: nothing stateful enough to degrade. The validation suite proved that wrong within a day: under sustained loop-task churn, ngram-mod degrades to the same ~0.5 tok/s crawl as MTP (GPU idling at ~106 W, restart restores, fresh-server probes fine). Priority is fixed in llama.cpp: ngram implementations draft before draft-model implementations when both are configured, each serving sequences the other has no draft for.

Measured against a no-speculation baseline of 73.2 tok/s generation:

variant	cold pool	warm pool	TTFT p50
defaults (`n-match 24`, `n-min 48`, `n-max 64`)	141.1 (+93%)	462.1 (+531%)	351.0-361.8 ms
small-n (`n-min 4`, `n-max 8`, `n-match 32`)	146.3 (+100%)	262.6 (+259%)	252.7-259.2 ms

Defaults won the warm-pool peak (long drafts land when content repeats verbatim); small-n won TTFT by ~100 ms. The small-n recommendation comes from community testing on consumer GPUs⁴. None of it survives contact with the actual workload, so neither variant ships: read this table as short-burst evidence only.

Two honest caveats on the warm-pool number. The perf harness reuses canned prompts, so a warm pool is maximally matched to them - 462.1 tok/s is the mechanism’s ceiling, not an agentic-workload expectation. And the pool dies with the server process: any model switch resets it, so the first minutes after a switch run at cold-pool speed.

The shared-mechanism hypothesis for both speculators: every rejected draft forces a rollback of the hybrid Gated DeltaNet recurrent state, and the rollback cost grows with context; at agentic-loop context sizes (24k+ tokens, growing) the rollback dominates until decode crawls, while prompt processing - a different path - stays fast. No-spec never rolls back, which fits the control. Upstream also tracks a separate ngram-mod stuck-loop on failed verification⁵. If a future pin fixes the state-rollback cost, the perf numbers above justify re-testing; until then this stack runs no speculation on the daily-driver preset.

That hypothesis is now partly falsified, and the next section is why: it predicts the failure for any engine running this architecture, and one engine does not show it.

The same speculator, a different engine

Both speculator findings above were measured on llama.cpp, and the rollback hypothesis makes a prediction that reaches past it: if rejected drafts force a recurrent-state rollback whose cost grows with context, any engine running this architecture under agentic churn should crawl the same way. NInfer, a from-scratch C++/CUDA engine for explicitly registered Qwen checkpoints⁶, does not.

Measured over 336 requests of real agentic coding traffic - the engine driving a four-milestone Rust and TypeScript build-out to all-sensors-green, prompt contexts to 88,985 tokens:

ctx bucket	requests	decode p50
0-10K	18	180.0 tok/s
30-50K	38	141.1 tok/s
50-70K	42	142.3 tok/s
70-100K	37	136.5 tok/s

Decode drifts 141 to 136 tok/s across 30K to 100K of context. MTP draft acceptance held at a 64% median (45.5-94.4% range) for the whole run, with zero engine errors and no restarts. On llama.cpp the same speculator on the same model decays to ~0.5 tok/s within ten minutes, and a community measurement thread reports its no-spec decode falling 122 to 69 tok/s as context fills⁷.

So the degradation is a property of an implementation, not of Gated DeltaNet rollback. That does not repair llama.cpp - it narrows where to look, and it retires “speculation cannot work on this model under churn” as a conclusion.

The catch is coverage, and it is severe. NInfer serves five explicitly registered Qwen artifact identities and rejects every other checkpoint, its build targets sm_120a only, and each artifact is a single file carrying its own tokenizer and chat template⁶. Of the eight presets on this box, two map to it and the Gemma and LFM ones never can. It is a second engine for one model family, not a replacement.

Accuracy on the new engine (2026-09-08)

Fast is only half the adoption question, so the day after the speed numbers the accuracy side got its own pass - HumanEval, BFCL function-calling, and the sensor-gated task suite the engine was adopted for.

The task suite first, because it is the decisive one for a coding-agent engine. On the six loop tasks (t1-t6: add/fix a Go or TS function, write the tests for it), NInfer at medium went 4/6 - t1, t2, t4, t5 pass in one iteration each (35-81 s); t3 and t6 fail at the eight-iteration cap. Those two are the test-writing tasks and the hardest in the suite: the llama.cpp Q4_K_M baseline goes 4/6 with the same two misses (t3 1/3, t6 0/3 across runs), and every other model benched here fails both. Same suite, same two failures - the engine change did not move task quality off its own-weights baseline.

The standardized numbers: HumanEval pass@1 is 0.598 (greedy, 164 problems, through the proxy), 0.591 with the extra tests. This is the first eval number the harness has produced that did not error at parse time, so it is the reference point future presets run against rather than a delta. BFCL non_live function-calling came back at simple_python 68%, parallel 76.5%, parallel_multiple 73%, multiple 76%, simple_javascript 48%, simple_java 40% - the multi/parallel categories an agent loop actually uses sit at 73-76%, function-calling is clearly working, and the single-file Java and JS stub categories are the soft spot. (The overall BFCL aggregate is absent: the leaderboard aggregator crashed on a null from the irrelevance category, a harness bug, not a model one.)

Two accuracy questions remain. Long-context quality (as opposed to the proven flat decode) needs a needle-in-context probe - the tokenizer and hot-swap blockers are both now fixed (a local HF tokenizer fallback, and a --no-swap mode that probes the resident model at its configured context), but the probe surfaced a sharper limit in the process: the engine rejects a single maxed-out request around 261.5K tokens even though the configured --max-context is 262144 and a simple one-message test serves 261000-262000. The usable ceiling is below the advertised one, and it is state- and content-dependent. A clean retrieval run at that boundary is still pending. And churn stability beyond a single session is unmeasured: 6.2% of requests in the validation run decoded below 100 tok/s (worst 33.9), clustered at 66-72K context, and the cause is unexplained.

Two configuration traps cost most of a day before any of the above measured cleanly, and both are invisible in the harness report:

Thinking effort is a per-request field on this engine, not a serve flag, and the artifact’s template exposes low|medium|xhigh. The OpenAI protocol’s high is parsed and then rejected with reasoning_effort_not_supported, because the engine validates against the template embedded in the artifact rather than translating vocabularies⁸. A client sending the protocol default gets HTTP 400 on every request.
A missing output cap truncates mid-thought. With no explicit maxTokens the harness applied a 16,384 default; at xhigh one tool call produced 35,747 output tokens, so responses were cut before emitting a tool call or any text. The agent exited 0 having changed nothing, twice, which reads as model incapability rather than a ceiling.

Neither is fixable in the preset, because the template ships inside the artifact. They belong in the proxy, which is the one place that fixes them for every client at once.

Reasoning effort: xhigh vs medium

The A/B ran the sensor-gated task suite with both arms speculation-free, isolating the effort lever.

Quality gate: neutral. The two ceiling tasks (write-new-tests) fail at both efforts - t3 in 5433 s at xhigh vs 5108 s at medium, t6 in 3000-4079 s historical at xhigh vs 1852 s at medium. Same outcomes, cheaper failures.

Speed gate, wall seconds per run (every run passed):

task	xhigh	medium
t1 (go, add fn)	42.5, 39.0, 44.0, 25.4	31.4, 19.3, 20.2, 12.7
t2 (go, fix fn)	19.7, 1308.5, 35.9, 11.5	12.5, 19.6, 10.8, 17.2, 25.3, 17.7
t4 (ts, add fn)	49.1, 14.2, 38.8, 19.6	13.1, 10.8, 11.2, 10.6, 9.8, 10.7
t5 (ts, fix fn)	11.3, 18.7, 13.6, 12.4	10.5, 10.3, 10.6, 11.5, 9.6, 11.3

The xhigh t2 row is the argument: the same one-iteration task measured 19.7 s in one run and 1308.5 s in another - a single ~80k-token thinking binge at decode speed. Medium’s entire distribution fits in 9.6-31.4 s. For an unattended loop, that predictability is throughput.

The community read on this is genuinely split: one set of reports runs xhigh unattended overnight with zero failures and sees no reason to drop; another reports endless re-thinking even at medium; a measurement thread puts medium at roughly the previous generation’s default quality⁹. The task-suite answer for this stack: identical failure outcomes on the hard tasks, no binges on the easy ones. xhigh stays one preset away for babysat work where the extra thinking is worth supervising.

A second argument for medium arrived from operations rather than the task suite. Running an unattended loop on the NInfer engine at inherited xhigh, one tool call produced 35,747 output tokens - inside the 15k-40k band the community reports¹⁰, and enough to blow a 16,384-token response cap mid-thought. Two iterations exited cleanly having written nothing. The task-suite case for medium is that it costs nothing in quality; the operational case is that xhigh’s tail interacts badly with every output ceiling downstream of it.

Mechanics, for llama.cpp: the level goes in as a chat-template kwarg, --chat-template-kwargs '{"reasoning_effort":"medium"}' (compact JSON; the flag word-splits). xhigh is the server default and injects “think carefully” language; community reports put it at 15k-40k thinking tokens per one-shot prompt¹⁰. On NInfer it is a per-request field instead, and high is rejected outright - see the engine section.

The thinking budget, and why nothing sets it

Effort lowers the average thinking spend and bounds nothing, so llama.cpp’s --reasoning-budget (a hard token cap, -1 for unrestricted) looks like the missing half. Running the obvious A/B on it produces a confident wrong answer, for a reason worth spelling out.

When the budget is exhausted the server injects the end-of-thinking tag and the model answers from a chain cut mid-sentence, so the cost lands on answer quality.

Generation lengths per request, read off the server’s own timing lines:

workload	requests	total generated	p95	max
4 fixture loop tasks, medium	18	3,902	412	412
same tasks, xhigh	12	5,368	1,134	1,134
one real greenfield build	202	490,818	7,819	9,564

The fixture suite cannot answer the question at all. At a 412-token ceiling, any budget worth setting is inert, so an A/B on that suite returns “no measurable effect” whether or not the knob does anything - a false negative that reads like evidence.

On the real task, an 8000-token cap would have cut 9 of 202 requests in a run that passed on its first iteration - reasoning that was on its way to a correct answer. And nothing has exceeded 9,564 tokens, so even a generous runaway guard would never have fired.

The budget was proposed here for a specific incident: one loop task took 1165 s inside a single iteration. That justification did not survive the measurement either. The largest generation ever observed is 9,564 tokens, roughly 127 s of wall time at this model’s decode rate, so a 1165 s single-iteration wall is many turns or a stall - a thinking budget would not have touched it. Set a budget when you have evidence of a generation that does not terminate, and set it above the p99 of successful runs. A cap tuned to make a suite faster will make it solve less.

One limit on the table above: those counts include thinking and answer tokens together, while the budget caps thinking only, so they are an upper bound on how often a given cap would bind.

Claims checked and rejected

From the release-week config threads, measured here and rejected:

“Official / recommended” sampling at temp 0.4, top_p 0.90, top_k 15, min_p 0.02. The thread author sourced it from a chat model. The model card’s thinking-mode recommendation is temp 1.0, top_p 0.95, top_k 20, min_p 0¹⁰.
Quantising the KV cache to stretch context (q4_1 main, 73k context on 16 GB). A 16 GB workaround; at 32 GB the model’s full 262144-token context fits on one slot without it, and q4 main KV costs long-context recall. (Two slots at that depth do not fit - this model’s dense KV runs 45.1 KiB per token - which is why the preset trades width for depth.)
Quantising the MTP drafter’s KV. Draft KV quantisation only lowers draft acceptance - fewer accepted tokens means slower, never wronger; the target model verifies everything. It also saves almost no VRAM. One Vulkan user reports the opposite ordering, which reads as backend-specific⁴.
Tuning --spec-draft-p-min to raise acceptance rate. One scripted test showed p-min 0.85 lifting acceptance from 76.5% to 92.5% while dropping throughput 7.4%: higher p-min suppresses drafts instead of improving them. Optimise tok/s, not acceptance.

The chat-template fix (2026-08-23, updated 2026-09-02)

Qwen’s official Jinja template shipped with real bugs documented and reproduced by the release-week community: enable_thinking=false crashes, multi-turn history gets poisoned with blank \think tags, tool calls crash when arguments arrive as JSON strings (the standard OpenAI format), mid-dialogue system messages get dropped, and on Hermes/agent harnesses the model hallucinates user instructions during thinking¹¹¹². The community fix is froggeric/Qwen-Fixed-Chat-Templates, loaded via --chat-template-file.

Deployed here 2026-08-23 at v22.3. Smoke-tested across 13 cases: basic chat, multi-turn memory, tool call + response round-trip, consecutive tool calls (chronological KV preservation), vision, inline <|think_low|> effort steering, mid-conversation system messages, multi-system merge, tool error recovery (two-tier escalation), false-positive disambiguation (code output containing throw new Error), Anthropic thinking field compatibility, and the hallucinated-user-instruction stress test (multi-turn debate). All pass.

The repo ships new versions faster than a stack notices. v22.4 landed the day after v22.3 was deployed here and went unnoticed for ten days: it adds multi-tool token alignment (dropping a stray newline before <tool_call>) and handling for an assistant message.reasoning field. Measured on the four fast fixture tasks, two runs each, same GGUF, container respawned so the new template actually loaded: both versions passed 8 of 8, but loop iterations fell from 12 to 8 - every v22.4 run converged on its first attempt, where v22.3 needed two or three on three of them - and total wall time fell from 543.4 s to 452.0 s. Small sample, and wall time on this suite is noisy, but the iteration count is the signal and it points where the release notes claim: a malformed tool call is exactly what costs an iteration. Check the template repo on a schedule; a template bug reads as a model failure.

None of this transfers to NInfer. Its artifacts embed their own chat template, resolved at startup from the file rather than supplied by a flag, so there is no --chat-template-file to point at the community fix and no way to apply it short of rebuilding the artifact. Whether those template bugs are present there is unmeasured; the effort vocabulary differs, which is evidence the template is a different one.

The quant your config names can stop existing

A preset here carries a HuggingFace repo plus filename, which the container uses as a download fallback when the file is absent locally. Nothing re-validated that pair after the first download.

On 2026-08-19 unsloth deleted every plain K-quant of Qwen3.8-27B and re-uploaded a dynamic-quant (UD-*) lineup with an imatrix pass. The same week, both ggml-org gemma-4 repos got the same treatment. Four GGUFs on this box stopped existing upstream while the configs kept naming them. Nothing failed, because the local files happened to survive - had the volume been lost first, the daily driver, the loop engine and the summarisation model were all unrecoverable.

The check is cheaper than it sounds: HuggingFace’s LFS oid for a file is that file’s sha256, verified here against a 6.7 GB local GGUF, so upstream identity is checkable from the tree API without downloading anything. A weekly job now classifies every preset file as present, renamed, changed, or gone, and copies anything with no upstream copy to off-box storage with a far-end hash check.

Moving to the replacement quant was worth doing on its own merits. Same runtime, same template, both arms back to back in one invocation, four fixture tasks x 3 runs, with the perf pass re-measured on page-cache-warm files:

	pass	median wall	max wall	generation	VRAM peak
plain `Q4_K_M`	12/12	52.9 s	1165.4 s	73.5 tok/s	29,897 MiB
`UD-Q4_K_M`	12/12	44.8 s	124.8 s	75.6 tok/s	29,217 MiB

UD is 0.6 GB smaller on disk, which buys KV headroom rather than costing it, and it holds the full 262144-token context with no silent --fit shrink. It is also the only one of the two that still exists upstream, which is what forced the comparison.

Two readings had to be retracted along the way, both from impatience. A first two-run pass had UD looking 63% slower; it did not survive three runs, because the file had been downloaded minutes earlier and its pages were cold. And the first perf sweep reported UD at 30.8 tok/s prompt processing against the incumbent’s 211.8 - a number that sat incoherently beside UD’s lower time to first token. Re-measured warm, both sat at 26-28 tok/s, which turned out to be a bug in the harness rather than a property of either quant (see below).

The verdict

The boring config wins. The daily-driver preset is now: model-card sampling (temp 1.0, top_p 0.95, top_k 20, min_p 0), reasoning_effort = "medium", no thinking budget, no speculation, the community-fixed chat template, unsloth’s UD-Q4_K_M, and the full 262144-token context on one slot. Every release-week optimisation that looked good in a short benchmark - MTP, ngram stacking, KV quant stretches, “recommended” sampling - either degraded under the actual workload or was never the author’s to recommend. xhigh + MTP stays one preset away for babysat interactive sessions where the +15% matters and a restart is cheap; if upstream fixes the recurrent-state rollback cost, the ngram-mod numbers above justify a re-test. The boring config is boring about llama.cpp specifically: on NInfer the same model runs MTP under churn without degrading, so a second preset now carries speculation deliberately rather than as a short-burst curiosity.

Reading the numbers

The synthetic perf suite (TTFT/gen/pp on canned prompts) is noisy across repetitions: the same preset measured 46-96 tok/s gen across days. Trust task-suite wall times and same-session compares; treat any single perf pass as indicative only.
Its prompt-processing column was worse than noisy, it was wrong, until 2026-09-02. It read prompt_per_second off a roughly 30-token prompt - overhead, not prefill - and took the sample with the best generation rate. It reported 26-28 tok/s while the server’s own timing lines said 2452. Measured properly, on a 17,221-token uncached prompt, this model prefills at 3675.5 tok/s. If a harness reports a prefill rate near its decode rate, suspect the harness.
Read medians, not totals or means. One fixture-task run consumed 1165.4 s in a single iteration and moved its arm’s total by 3x while saying nothing about the arm.
Two runs is not a sample, and a freshly downloaded GGUF is a cold page cache. Both of this page’s retracted readings came from ignoring one of those.
The fixture tasks are a regression gate, not a measuring instrument for anything long-horizon. Their largest single generation is 412 tokens against 9,564 on a real build, so context growth, compaction and thinking length are all invisible to them. Measure those on a real scoped task in a worktree.
MTP’s short-burst +15% (and medium’s +29% on top of it, 110.5-111.6 vs 85.9-86.5 tok/s) is real but describes the first minutes of a server’s life - exactly the regime the degradation finding disqualifies for loops.
ngram-mod’s warm-pool ceiling is inflated by the harness, as covered above. The cold-pool +93-100% is the trustworthy number, and it is already decisive.

Reproducing

The bench harness is the llmc bench module of the llm-compose stack: llmc bench perf --presets a,b for the synthetic suite, llmc bench tasks --presets a,b --tasks t3,t6 --runs 1 for the sensor-gated suite. Preset A/B variants need distinct GGUF filenames because presets deduplicate by model id (derived from the filename stem) - a hardlink costs zero disk. The p5 bench script embeds a /slots watchdog that respawns the server when a slot stops making progress, which is what made the long suite runs survivable while the degradation was live.

References

ggml-org, “MTP draft acceptance collapses over server lifetime,” llama.cpp issue #27151. https://github.com/ggml-org/llama.cpp/issues/27151 ↩
ggml-org, “MTP breaks long/short inference (Qwen3.8-27B),” llama.cpp issue #27296. https://github.com/ggml-org/llama.cpp/issues/27296 ↩
ggml-org, “Speculative decoding,” llama.cpp docs. https://raw.githubusercontent.com/ggml-org/llama.cpp/master/docs/speculative.md ↩
r/LocalLLaMA, Qwen3.8-27B config thread and comment discussion (small-n guidance, draft-KV quantization), August 2026 (mirror). https://shoftech.com/question/qwen-3-8-27b-pushing-1m-tokens-on-16gb-vram-whats-your-llama-cpp-config-for-similar-agentic-coding-73k-context/ ↩ ↩²
ggml-org, “server: stuck-loop escape for ngram-mod (WIP),” llama.cpp PR #25819. https://github.com/ggml-org/llama.cpp/pull/25819 ↩
Neroued, “NInfer,” GitHub. https://github.com/Neroued/ninfer ↩ ↩²
r/LocalLLaMA, long-context decode-decay measurements for llama.cpp against vLLM and NInfer, August 2026. ↩
Neroued, “HTTP serving,” NInfer docs. https://github.com/Neroued/ninfer/blob/master/docs/serving.md ↩
Isaak Tsalicoglou, “Qwen3.8-27B: slower tokens, faster and better results,” OVERBRING Labs. https://overbring.com/blog/2026-08-17-qwen3-8-27b-wall-clock/ ↩
Qwen, “Qwen3.8-27B model card,” Hugging Face. https://huggingface.co/Qwen/Qwen3.8-27B ↩ ↩² ↩³
froggeric, “Qwen-Fixed-Chat-Templates,” Hugging Face. https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates ↩
r/LocalLLaMA, “I fixed the Qwen 3.8 chat template bugs” (334 points), August 2026. ↩