Two videos watched end-to-end. What actually matters for our stack — and the one number that settles a question we were about to re-litigate.
An independent Tool-Eval Bench run put our exact quant — unsloth/Qwen3.6-35B-A3B-NVFP4-Fast — at 86.1, beating Poolside’s brand-new Laguna S 2.1 at 85.6.
We don’t need to chase it. And the same video hands us the more valuable finding:
thinking mode is a dial, not an upgrade — switching it on cost 12.3 points of agentic reliability and ran 30% slower.
That second one is directly actionable on Omnigent, VBL, and every tool-calling loop we run.
Tech2WiLD · 12:11 · benchmarked on 4× RTX 3090 and DGX Spark
Poolside shipped the big sibling to their 33B-A3B “XS”: 118B total parameters, 8B active, MoE, built for long-horizon agentic and long-context coding work. The pitch is punching above its weight class — and the benchmark table backs it, against models 4× to 14× its size.
| Benchmark | Laguna S 2.1 118B-A8B | Tencent Hy3 256B-A20B | Nemotron 3 Ultra 500B-A50B | DeepSeek-V4-Pro-Max 1.6T-A64B | Inkling |
|---|---|---|---|---|---|
| Terminal-Bench 2.1 | 70.2 | 71.7 | 65.4 | 64.0 | 63.8 |
| SWE-Bench Multilingual | 78.5 | 71.5 | — | 76.2 | 61.7 |
Beating a 1.6T model on SWE-Bench Multilingual with 8B active params is genuinely impressive. These are vendor numbers though — unverified on any hardware we own.
He ran Tool-Eval Bench — cross-trial summary comparison, agentic tool-calling workloads — between Laguna and the exact Qwen quant we already deploy:
| Metric | unsloth/Qwen3.6-35B-A3B-NVFP4-Fast | poolside/Laguna-S-2.1-NVFP4 | Δ |
|---|---|---|---|
| Mean score | 86.1 · winner | 85.6 · runner-up | +0.5 |
| Deployability (≥0.7) | 80 / 100 | 71 / 100 | +9 |
| Reliability (Pass²) | 68.7% | 63.7% | +5.0 |
| Safety warnings (max) | 1 | 2 | safer |
| Quality | 88 / 100 | 90 / 100 | −2 |
“Fewer points mean · lower variance · safer.” Laguna edges quality; we win everywhere that decides whether a model survives in an unattended agent loop.
Same model, same quant, thinking toggled OFF → ON:
| Axis | Thinking OFF | Thinking ON | Δ |
|---|---|---|---|
| Code | 70 | 73 | +3 |
| Instructions | 86 | 92 | +6 |
| Planning | 65 | 70 | +5 |
| Agentic | 99.3 | 92.6 | −6.7 |
| Reliability on agents | 98.6% | 86.3% | −12.3 pp |
| Latency (avg) | 5.9 s | 7.8 s | 30% slower |
| Overall (weighted) | 86.5 | 86.0 | −0.5 |
Thinking ON to write code. Thinking OFF to run agents. Turning it on improves how a model reasons about a problem and degrades how reliably it executes a tool chain — and you pay 30% latency for the privilege.
Laguna ships with a DFlash draft model — install it plain, or install it with DFlash. On the same 4× 3090 box:
That’s roughly 3× throughput for half the KV pool — the draft model eats the context budget. He published the config as a repo: “Laguna S 2.1 INT4 + DFlash at 200K on 4× RTX 3090.”
Concurrency sweep on one DGX Spark, 1000-token generations:
| Streams | TTFT | Server tok/s | Decode tok/s |
|---|---|---|---|
| ×1 | 368 ms | 35.0 (peak 117) | 41.0 |
| ×2 | 526 ms | 58.4 (peak 175) | 45.7 |
| ×4 | 578 ms | 56.2 (peak 198) | 27.5 |
| ×6 | 742 ms | 111.6 (peak 333) | 29.2 |
| ×8 | 736 ms | 78.0 (peak 408) | 26.2 |
“This is not the model for me — it’s not replacing my Qwen 35B-A3B and 27B.” It excels at coding and long-context building; for agentic work he stays on 35B-A3B. Running it costs him all four GPUs — losing concurrency, KV pool, and multimodality — to gain a smarter model he doesn’t need in the middle of a stack that already has GLM.
His hybrid stack, for reference: Fable/Opus on top builds → GLM 5.2 in the middle executes and supervises → 27B + 35B-A3B as workers. Notably close to ours.
Fahd Mirza · 12:55 · live agent test + multimodal sweep
Google dropped three Flash models at once. Only one is the story:
| 3.5 Flash-Lite | 3.6 Flash tested | 3.5 Flash Cyber | |
|---|---|---|---|
| Positioning | Speed at scale | Workhorse — better quality, lower cost | Vuln detection in CodeMender |
| Input / Output | $0.50 / $2.50 per M | $1.50 / $7.50 per M | not public |
| Speed | 350 tok/s | 17% fewer tokens than 3.5 Flash | — |
| Context | 1M | 1M | — |
| Access | Public | Public | Trusted gov partners only |
| Benchmarks | Terminal-Bench 2.1 54% OSWorld-Verified 74% SWE-Bench Pro 54.2% | DeepSWE 49% OSWorld-Verified 83% MLE-Bench 63.9% | CyberGym frontier-competitive |
His read: Google is upgrading against its own prior models, not against the field. 3.6 Flash beats 3.5 Flash on coding, computer use, and MLE-Bench — while using fewer tokens per task. It explicitly does not claim to beat 3.1 Pro on the hardest long-context reasoning; that gap stands. Flash-Lite is the sleeper — it beats the older 3 Flash on SWE-Bench Pro and OSWorld while being cheaper and faster.
He built a FastAPI seaport freight tracker with a real bug planted in it: a vessel with a 9.8 m draft gets silently approved into a 9.5 m berth instead of rejected — in the real world, the ship runs aground. He handed the whole repo to 3.6 Flash through a Hermes agent with one goal: find the bug, fix it, start the server, and verify the fix yourself.
His pointed comparison: Gemini K3 and Fable restarted those backends themselves. 3.6 Flash didn’t. The intelligence was there; the follow-through wasn’t.
“Quite good from Google, but I don’t think in terms of very high quality coding they match Fable 5 or Opus — still not a bad result at all.”
An independent tool-calling bench has unsloth/Qwen3.6-35B-A3B-NVFP4 beating Laguna 86.1 to 85.6, with better deployability (+9), better reliability (+5 pp), and fewer safety warnings. Laguna stays ⚠ claimed in the Model Diary until measured on our own iron — and this is exactly the “should X replace our core local?” question the Standing Verdicts table exists to stop us re-arguing.
−12.3 pp reliability on agents, −6.7 agentic score, +30% latency — for +3 on code. Every place we run a tool-calling loop (Omnigent dispatcher, VBL voice agent, fleet-rpc job flows, the vb-broker fan-out) should default thinking off and switch it on only for code generation. This costs nothing to test and is the single most portable finding across both videos.
80–90 → 250 tok/s from the draft model alone, same GPUs. We already run Buun-DFlash on our orchestrator. Worth measuring that ~3× ratio on our 96 GB box’s 96 GB and on our 24 GB laptop-GPU nodes, and deciding deliberately whether we’d trade half the KV pool for it per workload. Speculative decode may be a bigger lever for us than any model swap this month.
30–40 tok/s on a single Spark, and it wants the whole box. We already have DeepSeek-V4-Flash measured at 59.5 tok/s on one of ours. The 4×3090 path is where Laguna shines, and that isn’t a shape we have.
3.6 Flash fixed the bug and left the environment dirty. Any model we wire into an unattended loop will do some version of this. The restart / re-verify / clean-state step belongs to our harness, asserted independently — never trusted to the model’s own “I’m done.”
$0.50/M in and 350 tok/s at 1M context is genuinely cheap for high-volume classification or extraction. But our own hardware is free — this only earns a slot where scale or latency beats what we can serve ourselves. Not a default.
Provenance: every number on this page is from the two videos — vendor benchmark tables, or measured by the presenters on their hardware (4× RTX 3090, DGX Spark). None of it has been measured on fleet iron. Per Model Diary law, treat all of it as ⚠ claimed until we run the same-prompt harness ourselves.