The Archive

I know this sub loves absurd LLM projects, so sharing my contribution while we wait for the new Qwen 3.7 models to drop! I successfully got a tiny LLM running inside an RTOS, running inside a custom-built JavaScript emulator for the Freescale ColdFire MCF5307, which is a derivative of the legendary [Motorola 68K](https://en.wikipedia.org/wiki/Motorola_68000) that powered the original Mac and Sega Genesis. The RTOS was written back in 2008 with three classmates for our embedded systems university course. It was lost to time, with the hardware and original ROM long gone. A few months ago, I d...

Link: foodtruckbench.com

Last month NVIDIA released [SOL-ExecBench](https://research.nvidia.com/benchmarks/sol-execbench), a new benchmark of 235 production CUDA kernels lifted from DeepSeek, Qwen, Gemma, and Kimi. We took several top-ranked AI-generated submissions and tried using them in production workloads. Many of them broke, sometimes in surprising ways. One of those kernels is the fused embedding-gradient + RMSNorm backward pass, which runs at the end of every transformer training step. We took the fastest submission on the benchmark for it, and dropped it into the training loop of a small transformer. The ke...

Here's my article with **38 quant pairs** thoroughly benchmarked in KLD with **3 different Qwen 3.6 27B configs**: Q5\_K\_S + 64k context, IQ4\_XS + 64k context, IQ4\_XS + 128k context. This allows us to track not only how cache quantizations affects the precision in a vacuum, but also how it interacts with noise from the model itself. All benchmarks were done using my [BeeLlama.cpp](https://github.com/Anbeeld/beellama.cpp) fork, allowing to include a number of quant types that are not present in mainline llama.cpp: vanilla TurboQuant, TCQ 3-bit/2-bit, and q6\_0. [https://anbeeld.com/articl...

I don't see any threads on this model. Is it because it's dense and/or without-**reasoning**? Anyone tried this for coding? >[**Capabilities**](https://huggingface.co/ibm-granite/granite-4.1-30b) Summarization Text classification Text extraction Question-answering Retrieval Augmented Generation (RAG) Code related tasks Function-calling tasks Multilingual dialog use cases Fill-In-the-Middle (FIM) code completions Some people prefer dense in this model size range(Ex: 27B over 35B-A3B). Still no feedbacks from them here. I know that some people love Granite models. Myself...

As per Minimax\_AI twitter [https://x.com/MiniMax\_AI/status/2059286515155599595](https://x.com/MiniMax_AI/status/2059286515155599595) I hope it will speed up Qwen3.7 open weights release. https://preview.redd.it/q1bdhs017n3h1.png?width=898&format=png&auto=webp&s=a9a8ea134a71b9e5b9ea2489fc72420e18c6da67

I dont have good experience running q4\_k\_m, the difference to q6 is "a few errors an hour" to " a few errors every couple of days". Edit: How it fails? Just like user DifficultDog8435 and FullstackSensei explained in the comments. They worded it better than me.

I picked up a 7900 XTX earlier which runs qwen3.6-27b fine, but not to my like. Its compute performance is quite unstable for me. With MTP the decode speed can reach 40-60 t/s, but prefill is just too slow. Regardless of whether I used ROCm or Vulkan, the prefill speed varies between 300t/s and 500 t/s, even with very long prompts. I've been itching to try out an ultra-budget 24GB setup using dual 3060s. I managed to snag a second 3060 at a reasonable price in last few days. So I took out the 7900 XTX, installed the 3060s, and began testing. # Test Configuration * **Test Platform:** i7 477...

For real tho, 9b, 27b, 122b, I don’t really care at this point, just show us that you still love us.

I previously hated on this model, but I have just been impressed by it, and I understand the hype now. I have been working on a HTML5 game console and I decided to see if Qwen3.6 27B can handle making some quick games in it to showcase functionality (save games, console API handling for stat tracking and heartbeat management, meta data for the game, etc) I gave it 3 files, explaining how the API works, the gamepad controls, and a typescript shader for it to apply. Then I just game it a very simple prompt "make a breakout game for this console, in the working directory are reference files on...

Safetensors, llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved: [https://huggingface.co/llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved](https://huggingface.co/llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved) GGUFs, llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved-GGUF [https://huggingface.co/llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved-GGUF](https://huggingface.co/llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved-GGUF) NVFP4, llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-N...

I've fine-tuned Qwen 3.5 0.8B on the dataset provided by Pangram with their EditLens paper. It's available via a [Chrome extension](https://chromewebstore.google.com/detail/slop-hammer/gfjdmhfokmhedlgfggmmgchpppmhkdgg); you can just click selected text and it's going to give you the probability distribution of how likely it is AI-generated. It takes under 1s on my M1 MacBook Pro. Pangram did release Llama 3.2 3B trained on their dataset, but I found this model slightly too legacy (too big for the capabilities). Qwen 0.8B (base) ended up being as good after roughly 20h of fine-tuning on a sin...

User reports Qwen3.6 35B outperforms Gemma4, GLM 4.7 Flash, others for local agentic tasks; seeks comparable MoE alternatives.

MiMo-V2.5-coder released as open-weights coding model alternative to Qwen and DeepSeek for 128GB+ systems.

User demonstrates 1000 tokens/sec generation throughput on Qwen 3.6 27B with V100 GPUs at high batch sizes.

hipEngine: open-source ROCm-native inference engine for Qwen 3.6 MoE on AMD RDNA3 GPUs (7900 XTX, Strix Halo).

Reddit discussion comparing inference speed/quality tradeoffs between Qwen3.6-35B and Gemma4-26B on consumer GPU hardware.

Community finetune of Qwen 3.6 35B with quantized weights; testing on consumer hardware shows stability at 200k context.

Reddit discussion asking about locally-runnable 397B-parameter model alternatives to Qwen 3.6 fitting in 256GB RAM.

Qwen 3.6 27B quantized to fit 16GB VRAM at 40 tok/s; community optimization for edge deployment.

User reports running Qwen3.6-35B at 262k context on RTX 3070 Ti 8GB with 30 tps using Q4 quantization; claims 1M context possible with performance degradation.

BeeLlama v0.2.0 achieves 4-5x token throughput gains on RTX 3090 via DFlash optimizations for Qwen 27B and Gemma 31B models.

ByteShape releases optimized quantization for Qwen3.6-35B achieving 30% faster inference than Unsloth on 6GB VRAM.

cHunter789 releases Qwen-27B IQ4_KS quantization (14.1GB) optimized for 16GB NVIDIA GPUs via ik_llama.cpp.

Disclaimer: I work for Numind, the company behind this open-weight model We just released a 4B model based on Qwen3.5-4B, under Apache-2.0 license. The goal is to make information extraction from complex documents more practical with an open model: PDFs, screenshots, forms, tables, receipts, invoices, multi-page documents, and other visually structured inputs. Try it, we have a huggingface space that is completely free (you don't even have to sign-up): [https://huggingface.co/spaces/numind/NuExtract3](https://huggingface.co/spaces/numind/NuExtract3) If you ever used [NuMarkdown](https://hu...

User reports Qwen 3.6 35B enabling agentic workflows for DevOps, document processing, and code tasks via skill-chaining.

Qwen 3.7 open-weights model released; community discussion on LocalLLaMA highlights adoption momentum.

Reddit discussion speculating on Qwen 3.7 Max performance and open-weight availability; no official details or benchmarks provided.

User reports 110 tok/s inference speed on Qwen 35B with 12GB VRAM using ik_llama.cpp, outperforming standard llama.cpp after MTP merge.

User shares llama.cpp configuration for running Qwen 3.6 27B locally with ROCm acceleration and optimizations.