qwen3.8:27b — dense
66 blocks · no experts · every block active
28.9 tok/s · ctx 131,072 · 63 of 66 layers resident
What a spilled layer actually costs, where sparse and dense models respond differently, and why decode rate does not predict finishing. Three chips, paired tasks, and a controlled dual-3090 spill curve. Not a frozen paper.
A model’s speed is not solely an inherent property. In one controlled dual-3090 run, moving about 10% of layers to CPU cut decode by 62–69%; about 20% cut it by 76–84%. Prefill moved differently, and neither rate predicts task completion by itself.
This page combines earlier single-card measurements on an RTX 3090, a Ryzen AI MAX 390 and an RTX 2080 with later paired-task and dual-3090 spill runs. In the daemon comparisons, decode is eval_count / eval_duration, not wall clock; the new spill curve is a separate direct llama.cpp synthetic-prompt run. Task time is the local-lane L2 battery. The measurements answer different questions; none is UID-verified or registered as a claim.
Two things decide how fast a local model writes: whether all of it fits on the card, and how much of it does work on each token. The first is a budget you can drag. The second is architecture, and no amount of tuning moves it.
A model is a stack of identical blocks. Each block is a layer, and a token passes through every one of them, in order, to produce a single output token. Nothing skips ahead. The count is fixed by the model: qwen3.8:27b has 66, gemma4:31b 61, qwen3.6:35b 42, gemma4:26b 31.
Each layer holds its own slice of the weights, and each does the same two things in the same order: attention, which looks back over everything written so far, then feed-forward, which is where most of the parameters live. That second half is the part a mixture-of-experts model makes sparse. The first half never is.
So when a card cannot hold the whole model, the split is made in whole layers — some on the GPU, the rest in system memory. A readout of 63/66 means three layers' worth of weights are off the card, and every token has to leave the GPU, cross to system memory and come back, three times, before it can be produced. That is why a 4.5% shift in where the weights sit costs 27.6% of the speed: it is not a little more work, it is a detour onto the slowest link in the machine, repeated for every token you read.
Weights are fixed. The KV cache grows with the context you reserve, not with how long your conversation actually is — a short prompt under a large reservation spills exactly as hard. When the two together exceed GPU memory, whole layers get pushed to the CPU. Switch between one and two 3090s to see the aggregate budget; dual mode is an estimate, not a per-card placement guarantee.
Measured dual checks, 30 Oct 2026 (separate from this estimate): at 262,144 context, both Qwen routes processed 256,013 input tokens with all layers on GPUs (66/66 for 27B; 42/42 for 35B-A3B); that fit smoke used different K/V caches (F16 vs Q8_0). A cache-matched Q8_0 spill sweep found about 62–69% decode loss at ~10% CPU layers and 76–84% at ~20%. See the measured curves and controls.
The 35b is the bigger model on disk and decodes more than twice as fast. Fire a token through both and watch which parameters light up.
66 blocks · no experts · every block active
28.9 tok/s · ctx 131,072 · 63 of 66 layers resident
41 blocks · 256 experts each · 8 fire per token
86.9 tok/s · ctx 131,072 · all 42 layers resident
3.1% of the experts fire. The 35b routes each token to 8 of its 256 experts per block, so despite holding more weight it moves far less of it per token. The 27b is dense — all 27 billion parameters do work on every single token. That is the gap, and no context tuning or card shuffling changes it.
A model does not "think about" your prompt and then answer. It runs one pass to produce one token, appends it, and runs the whole thing again. Everything on this page — the cache that fills your card, the sparse half of a block, the gap between decode rate and finishing time — falls out of that loop.
Why the two differ by 20× per token. Prefill pushes every prompt token through the stack in one batched pass, so the weights are read once and amortised across thousands of tokens. Decode pushes one token through, reads the same weights, and throws them away — then does it again for the next token. Same arithmetic, no one to share it with. That is why decode is bound by memory bandwidth rather than compute, why a sparse model wins there, and why tok/s describes only the second half of the work.
Seconds to finish an agentic L2 task, warm trials. Lower is better. Read each chip against its own two tasks: the ordering changes between them, on every chip.
| Task | Model | seconds | |
|---|---|---|---|
| constant-and-callers | qwen3.6:35b | 13.0 | |
gemma4:26b | 15.0 | ||
qwen3.8:27b | 17.9 | ||
| csv-summarize-repair | qwen3.6:35b | 21.2 | |
qwen3.8:27b | 29.7 | ||
gemma4:26b | 46.0 |
The fastest decoder of all four is last on the long task, by 3×. These rows predate the card-clearing check described below: the card was not verified empty before each run, and num_ctx was not recorded per group at the time. The two qwen rows were re-measured on a cleared card on 25 Sep — next table.
Same card, same daemon, minutes apart, num_ctx 16,384 and temperature 0.8 on both, card 0 verified empty (54 MiB) before launch, residency proved for each model. Seven trials each, cold discarded.
| Task | Model | mean s | range | |
|---|---|---|---|---|
| constant-and-callers | qwen3.6:35b MoE | 7.6 | 6.8–8.5 | |
qwen3.8:27b dense | 15.1 | 11.4–18.2 | ||
| csv-summarize-repair | qwen3.6:35b MoE | 19.8 | 12.8–23.6 | |
qwen3.8:27b dense | 20.1 | 15.4–27.0 |
The architecture advantage is real on the short task and absent on the long one. On constant-and-callers the MoE model finishes in half the time, and the ranges do not overlap — nothing about the draw can produce that. On csv-summarize-repair the two are 1.6% apart with ranges that overlap almost completely: this measurement does not separate them, and the 10 Sep table's 21.2-against-29.7 reads as a larger gap than the instrument can resolve.
Why n matters more than it looks. The long task's within-model spread is up to 84% of its own mean — one model's trials ran 12.8 to 23.6 seconds without anything changing. At n=3 that spread is wider than the difference being measured, so the draw picks the winner, and it will pick a different one on a different night. Every crossover on this page that rests on three trials of a long task should be read as unresolved rather than as a finding. All 28 cells in this run passed, none unproven.
| Task | Model | seconds | |
|---|---|---|---|
| constant-and-callers | gemma4:26b | 20.6 | |
qwen3.6:35b | 26.1 | ||
qwen3.8:27b | 57.9 | ||
| csv-summarize-repair | qwen3.6:35b | 53.0 | |
qwen3.8:27b | 60.8 | ||
gemma4:26b | 146.6 |
The 26b is least penalised on the short task and most on the long one, on the same chip. There is no single "this chip is N× slower".
| Task | Model | seconds | |
|---|---|---|---|
| constant-and-callers | gemma4:26b | 40.9 | |
qwen3.6:35b | 59.2 | ||
| csv-summarize-repair | qwen3.6:35b | 120.4 | |
gemma4:26b | 218.6 |
The order flips again: the 35b is slower on the short task and 1.8× faster on the long one. Both are regions, not points — the 26b's trials ran 124.5 / 218.6 / 330.8, the 35b's 108.4 / 136.2 / 120.4. n=3 with a cold trial here, against n=4 warm on the other chips. The 35b passed 2 of 3 on csv-summarize-repair; that miss was a real coding error, not a timeout, on a task whose own header records a ~75% base rate at n=100. The dense 27b has no row on this card: at 2.3 tok/s an agentic task times out rather than finishing slowly. The last bar runs past the axis — true value at right.
The same models, measured as tokens per second. Compare this ordering against the one above.
| Chip | Model | Placement | tok/s |
|---|---|---|---|
| RTX 3090 · ctx 131,072 q8_0 | qwen3.6:35b | 42/42 on GPU | 86.9 |
qwen3.8:27b | 63/66 — 3 on CPU | 28.9 | |
qwen3.8:27b | 66/66 at ctx 98,304 | 42.9 | |
| MAX 390 · ctx 16,384 | qwen3.6:35b | integrated | 60.6 |
gemma4:26b | integrated | 56.4 | |
qwen3.8:27b | integrated | 19.7 | |
| RTX 2080 · ctx 16,384 | qwen3.6:35b | 42/42, 7,094 MiB | 28.2 |
gemma4:26b | 31/31, 7,414 MiB | 20.0 | |
qwen3.8:27b | 16/66, 6,434 MiB | 2.3 | |
| RTX 3090 · ctx 16,384 f16 | gemma4:26b | solo, reference | 171.3 |
qwen3.6:35b | solo, reference | 132.7 | |
qwen3.8:27b | solo, reference | 56.4 | |
gemma4:31b | solo, reference | 33.7 |
Weights stream; they do not have to fit. On the 2080 the 22.6 GB MoE model reports every layer offloaded in 7,094 MiB and beats the 18.6 GB MoE model on the same card. Process memory moves 448 MiB against 14.2 GB of page cache — the weights are memory-mapped and paged from disk, so there is no fixed VRAM-plus-RAM envelope. A dense layer must be resident to compute; an MoE layer only needs the experts a token routes to.
Force layers onto the CPU and the two architectures fail differently. Each curve is scaled to its own full-residency rate; the absolute rates are not comparable across the two. The shape is the point.
gemma4:26bforced offload, 31 layers
| 31/31 | 224.8 | |
| 30/31 | 172.7 | |
| 20/31 | 54.7 | |
| 12/31 | 35.5 |
Slows to a sixth and keeps answering. An earlier sweep floors it near 13 tok/s at 100% CPU.
gemma4:31bVRAM cap, 61 layers
| 55/61 | 12.3 | |
| 50/61 | 8.1 | |
| 45/61 | 6.0 | |
| 40/61 | 4.8 |
At 40/61 the task times out — 2 of 2, re-confirmed. It stops finishing, not just slowing.
Both Qwen models start fully resident on the same two cards, then move a similar fraction of layers to CPU. Each bar is normalized to that model’s own all-GPU decode rate; compare the shape, not the absolute tok/s across models.
qwen3.8:27bQ4_K_M · Q8_0 K/V · 262,144 context
| CPU / GPU layers | rate retained | tok/s · % |
|---|---|---|
| 0 / 66 (66 / 66 GPU) | 36.25 · 100% | |
| 6 / 66 · 9.1% (60 / 66 GPU) | 11.39 · 31.4% | |
| 13 / 66 · 19.7% (53 / 66 GPU) | 5.91 · 16.3% |
qwen3.6:35b-A3BQ4_K_M · Q8_0 K/V · 262,144 context
| CPU / GPU layers | rate retained | tok/s · % |
|---|---|---|
| 0 / 42 (42 / 42 GPU) | 125.85 · 100% | |
| 4 / 42 · 9.5% (38 / 42 GPU) | 47.64 · 37.9% | |
| 8 / 42 · 19.0% (34 / 42 GPU) | 30.64 · 24.3% |
| Model | all GPU | ~10% CPU | ~20% CPU |
|---|---|---|---|
qwen3.8:27b | 1,312.7 tok/s | 522.0 · −60.2% | 320.5 · −75.6% |
qwen3.6:35b-A3B | 1,999.3 tok/s | 693.1 · −65.3% | 434.3 · −78.3% |
How to read it. About 10% CPU layers cut decode by 68.6% for the dense 27B and 62.1% for the 35B-A3B; at about 20%, the losses are 83.7% and 75.7%. The MoE retained more of its own decode baseline here, while its prefill loss was slightly larger. This is one workload, not a general architecture ranking.
Controls and limits. Identical 32,798-token synthetic input and 128-token output cap; one warm-up plus n=3 measured requests per point. The 262K context was configured, not filled. --n-gpu-layers moves associated KV layers too, so this is combined CPU layer/weight/KV placement, not weights-only spill. Equal layer fractions are not equal weight bytes; the MoE’s active work also differs. No task quality or coding-agent completion was measured. The full protocol and raw evidence are retained in Project Phoenix’s QWEN-DUAL-SPILLOVER-001 packet.
Everything above measures speed. That is the smaller half. Six separate suites measure whether a model is any good, and they were built at different times for different questions — they share no fixtures, no task format and no scoring. A number from one is not comparable to a number from another, and none of them is comparable to a throughput figure.
| Category | What it asks | Why it exists |
|---|---|---|
| Task shape | Does the same model succeed more when the task is more structured? Three levels: L0 goal-shaped ("make the tests pass"), L1 file-named, L2 plan-shaped — exact paths, anchored edits, one tool call per step, an explicit success criterion. | The foundational suite; everything else post-dates it. Pass rate rises L0→L2 for every model measured. Shape predicts success better than any "difficulty" claim. |
| Coding implementation | Write working code against a frozen Python substrate — a real nutrition dataset, not a toy. | The reference baseline the other coding-adjacent work defines itself against. |
| Judgment under rules | Preference calls between two pieces of work, and dispute handling that has to stay anchored to a supplied document rather than to the model’s own view. | Scored against answers that were argued out with the people who adjudicate that work for a living — not one author’s guess at a rubric. Held internally; described here, not published. |
| Data comprehension without drift | Reproduce exact facts from a frozen registry extract — implant counts, market shares, an HHI — while preserving product boundaries and refusing to drift into chatbot or clinical-advice framing. | Catches the failure that matters most in practice: a plausible-looking answer with an invented number in it. Scoring is exact literal values, present or absent. |
| Cold-read comprehension | Recover a project's boundary map from five repository documents the model has never seen, with no prior context. | Tracks input-corpus drift as a first-class variable — the corpus grew 63% between epochs, so results across epochs are declared non-comparable rather than quietly pooled. |
| Symbolic constraint compliance | Image generation from one fixed prompt where several meaning-carrying constraints — relative size, pose, arrangement — must all land at once. | A different modality with the same question: does the model satisfy every constraint simultaneously, or only the easy ones? |
Two axes, never pooled. Correctness is one measurement per model; it transfers across chips. Speed is one measurement per machine; it does not — the ordering on this page reverses between chips and between tasks. Re-running a correctness battery on new hardware costs hours and reproduces an answer already in hand. Re-running speed is mandatory.
And the standing rule that keeps them honest: a suite that predates a given measurement contract does not silently inherit it. Each carries its own protocol, its own epoch, and its own note about what it does not establish. Where two runs disagree, the newer one does not automatically win — the one whose configuration was actually recorded does.
gemma4:26b curve is a separate MoE measurement; the new Qwen sweep uses different models and dual-card conditions. Layer fractions do not equal parameter-byte fractions, and CPU placement moves KV buffers too. Do not transfer a per-layer penalty from one curve to another.gemma4:31b. Its expert metadata has not been observed, so "dense" for that model is inference from block arithmetic, not a read.csv-summarize-repair varies by up to 84% of its own mean between identical trials, so three trials cannot separate models that sit within about 30% of each other. A paired n=6 run found the two qwen models 1.6% apart on it, where n=3 had shown a 1.4× gap. Short-task orderings here have non-overlapping ranges; long-task orderings mostly do not.num_ctx, KV type and placement; rows from different groups are not comparable.