The Archive

Friday's big release was Qwen 3.8 27B , an Apache 2 licensed 27B parameter vision-capable LLM from Alibaba's Qwen research lab. I've been looking forward to this one: 27B is an excellent size for running a model on a reasonably specced laptop, and its predecessor Qwen 3.6 27B was impressive. Qwen's self-reported benchmarks for this model are eye-opening. They show a boost from both Qwen 3.6 27B and the closed-weight Qwen 3.7-Plus, which was one of Qwen's strongest models of any size as recently as May this year . It will be interesting to hear what independent benchmarks have to say about the...

Here's my article with **38 quant pairs** thoroughly benchmarked in KLD with **3 different Qwen 3.6 27B configs**: Q5\_K\_S + 64k context, IQ4\_XS + 64k context, IQ4\_XS + 128k context. This allows us to track not only how cache quantizations affects the precision in a vacuum, but also how it interacts with noise from the model itself. All benchmarks were done using my [BeeLlama.cpp](https://github.com/Anbeeld/beellama.cpp) fork, allowing to include a number of quant types that are not present in mainline llama.cpp: vanilla TurboQuant, TCQ 3-bit/2-bit, and q6\_0. [https://anbeeld.com/articl...

I dont have good experience running q4\_k\_m, the difference to q6 is "a few errors an hour" to " a few errors every couple of days". Edit: How it fails? Just like user DifficultDog8435 and FullstackSensei explained in the comments. They worded it better than me.

I picked up a 7900 XTX earlier which runs qwen3.6-27b fine, but not to my like. Its compute performance is quite unstable for me. With MTP the decode speed can reach 40-60 t/s, but prefill is just too slow. Regardless of whether I used ROCm or Vulkan, the prefill speed varies between 300t/s and 500 t/s, even with very long prompts. I've been itching to try out an ultra-budget 24GB setup using dual 3060s. I managed to snag a second 3060 at a reasonable price in last few days. So I took out the 7900 XTX, installed the 3060s, and began testing. # Test Configuration * **Test Platform:** i7 477...

User demonstrates 1000 tokens/sec generation throughput on Qwen 3.6 27B with V100 GPUs at high batch sizes.

hipEngine: open-source ROCm-native inference engine for Qwen 3.6 MoE on AMD RDNA3 GPUs (7900 XTX, Strix Halo).

Community finetune of Qwen 3.6 35B with quantized weights; testing on consumer hardware shows stability at 200k context.

Reddit discussion asking about locally-runnable 397B-parameter model alternatives to Qwen 3.6 fitting in 256GB RAM.

Qwen 3.6 27B quantized to fit 16GB VRAM at 40 tok/s; community optimization for edge deployment.

BeeLlama v0.2.0 achieves 4-5x token throughput gains on RTX 3090 via DFlash optimizations for Qwen 27B and Gemma 31B models.

User reports Qwen 3.6 35B enabling agentic workflows for DevOps, document processing, and code tasks via skill-chaining.

User shares llama.cpp configuration for running Qwen 3.6 27B locally with ROCm acceleration and optimizations.

Comparative evaluation of coding agents (GitHub Copilot, Pi, Claude Code, OpenCode) using Qwen 3.6 27B isolates model vs. harness performance.

ByteShape releases Qwen 3.6 35B GGUF quantizations in NTP and MTP variants with empirical comparisons across GPU/CPU hardware.

Qwen 3.6 35B MoE benchmark on RTX 5080: 56 tok/s at 128k context; Multi-Token Prediction offers no speed gain at scale.

Qwen 3.6 27B F16 achieves best local agentic Pac-Man code generation benchmark results, failing in 8-bit quantization.

Developer adapts multi-agent orchestration patterns from Claude Sonnet to resource-constrained local setup using Qwen 3.6-35B.

Qwen 3.6 27B performance benchmarks across llama.cpp backends on RTX 3090: ik_llama.cpp achieved 1261 tok/s prefill, 72.9 tok/s decode with 156k context.

Llama.cpp multi-token prediction on Qwen 3.6 27B shows 42% prefill slowdown but 85% token generation speedup on RTX 3090.

Benchmarking llama.cpp MTP (multi-token prediction) on Qwen 3.6 with RTX 5090, comparing inference speed with draft-mtp flag toggled.

Benchmark comparing Qwen 3.6 local quantizations vs frontier models on HTML canvas animation coding task.

Open-dLLM project explores diffusion-based LLM inference; users speculate on Qwen 3.6 optimization for 3k+ tok/s on RTX 5090, unvalidated.

User reports local agent (Pi + Qwen 3.627B) executed rm -rf on full disk, demonstrating autonomous decision-making risk in unmonitored coding agents.

Qwen 3.6 35B multi-token prediction variant achieves ~1.5x speedup over baseline in 300k-token context local inference experiments with pygame project.

User asks about quantization quality differences (Q4 vs Q6) for Qwen 3.6 on consumer GPUs; anecdotal discussion, no new data.

Qwen 3.6 35B and Gemma 4 26B MoE models achieve 20–24.5 tok/s on GTX 1080 with 128k context via llama.cpp quantization.

Qwen 3.6 27B achieves 52.8 tokens/sec throughput on MI50 GPU with full precision, enabling on-device agent inference.

Reddit discussion comparing Qwen 3.6 27B vs. 100B+ open models for local coding; user survey of high-VRAM model preferences.

Luce ships DFlash+PFlash optimizations for AMD Ryzen AI MAX+ 395, achieving 2.23x decode speedup on Qwen 3.6-27B vs llama.cpp HIP.

Reddit user benchmarks open-weight LLMs (Qwen 3.6, Zaya1) on chess visualization task; Qwen 35B-A3B achieves near-perfect SVG chessboard generation.