The Archive

Friday's big release was Qwen 3.8 27B , an Apache 2 licensed 27B parameter vision-capable LLM from Alibaba's Qwen research lab. I've been looking forward to this one: 27B is an excellent size for running a model on a reasonably specced laptop, and its predecessor Qwen 3.6 27B was impressive. Qwen's self-reported benchmarks for this model are eye-opening. They show a boost from both Qwen 3.6 27B and the closed-weight Qwen 3.7-Plus, which was one of Qwen's strongest models of any size as recently as May this year . It will be interesting to hear what independent benchmarks have to say about the...

Here's my article with **38 quant pairs** thoroughly benchmarked in KLD with **3 different Qwen 3.6 27B configs**: Q5\_K\_S + 64k context, IQ4\_XS + 64k context, IQ4\_XS + 128k context. This allows us to track not only how cache quantizations affects the precision in a vacuum, but also how it interacts with noise from the model itself. All benchmarks were done using my [BeeLlama.cpp](https://github.com/Anbeeld/beellama.cpp) fork, allowing to include a number of quant types that are not present in mainline llama.cpp: vanilla TurboQuant, TCQ 3-bit/2-bit, and q6\_0. [https://anbeeld.com/articl...

I dont have good experience running q4\_k\_m, the difference to q6 is "a few errors an hour" to " a few errors every couple of days". Edit: How it fails? Just like user DifficultDog8435 and FullstackSensei explained in the comments. They worded it better than me.

User demonstrates 1000 tokens/sec generation throughput on Qwen 3.6 27B with V100 GPUs at high batch sizes.

Qwen 3.6 27B quantized to fit 16GB VRAM at 40 tok/s; community optimization for edge deployment.

BeeLlama v0.2.0 achieves 4-5x token throughput gains on RTX 3090 via DFlash optimizations for Qwen 27B and Gemma 31B models.

User shares llama.cpp configuration for running Qwen 3.6 27B locally with ROCm acceleration and optimizations.

Comparative evaluation of coding agents (GitHub Copilot, Pi, Claude Code, OpenCode) using Qwen 3.6 27B isolates model vs. harness performance.

Qwen 3.6 27B F16 achieves best local agentic Pac-Man code generation benchmark results, failing in 8-bit quantization.

Qwen 3.6 27B performance benchmarks across llama.cpp backends on RTX 3090: ik_llama.cpp achieved 1261 tok/s prefill, 72.9 tok/s decode with 156k context.

Llama.cpp multi-token prediction on Qwen 3.6 27B shows 42% prefill slowdown but 85% token generation speedup on RTX 3090.

Qwen 3.6 27B achieves 52.8 tokens/sec throughput on MI50 GPU with full precision, enabling on-device agent inference.

Reddit discussion comparing Qwen 3.6 27B vs. 100B+ open models for local coding; user survey of high-VRAM model preferences.

Hugging Face co-founder claims Qwen 3.6 27B matches Claude Opus on coding tasks offline, signaling convergence in local model capability.

BeeLlama.cpp fork adds DFlash, TurboQuant, and vision support; runs Qwen 3.6 27B Q5 on RTX 3090 with 200k context at 135 tps.

Benchmark shows TP=2 pinned to NVLink GPU pairs yields +25–53% throughput vs PCIe on Qwen 3.6 27B; TP=4 degrades performance due to cross-pair PCIe bottleneck.

Community fine-tune of Qwen 3.6 27B with reduced safety filters released in multiple quantization formats.

User achieves 50 tokens/sec with Qwen 3.6 27B on RTX 3090 using MTP speculative decoding at 100k context.

User reports Qwen 3.6 27B in Hermes agent harness successfully handles junior IT tasks, signaling maturity of local model + agent systems.

Qwen 3.6 27B achieves 2.5x inference speedup via MTP speculative decoding in llama.cpp; 262k context on 48GB with fixed chat templates.

Empirical quantization degradation analysis for Qwen 3.6 27B across 8 compression levels via chess state-tracking task.

Qwen 27B achieves 54 t/s on V100 GPU with MTP optimization in llama.cpp, nearly 2x baseline speed for code review and tool use tasks.

Developer benchmarked local Qwen 3.6 27B vs cloud models on 150 real coding tasks, finding local matched cloud 97% on 35% of workload, suggesting cost arbitrage opportunity.

Reddit user reports Qwen 3.6 27B found a bug that GPT 5.5 and Claude Opus 4.7 missed, attributing success to extended reasoning.

Informal comparison of Qwen 3.6 27B and Gemma 4 31B on local Pac-Man gamedev task; Gemma produced faster, more logical output despite lower token throughput.

Qwen 3.6 27B/35B models outperform older ~30B alternatives (Qwen Coder, Gemma) on coding and agent tasks.

Qwen 3.6 27B achieves 60 tok/s throughput and 204k context on dual RTX 5060 Ti 16GB with vLLM.

Qwen 3.6 27B quantization benchmark: BF16 (69.78% avg accuracy, 54GB RAM) vs Q4_K_M (66.54%, 15GB) vs Q8_0 across HumanEval, HellaSwag, BFCL.

I'd jump on runpod and ssh in to test my workloads, but they don't have it. Would love to know how well this runs, particularly as context approaches a full 256K. Thanks!

User reports Qwen 3.6 27B outperforms 35B variant for local coding tasks on consumer hardware.