The Archive

Mixture of experts (MoE) has become one of the defining architectural trends in large-scale AI model training. DeepSeek, Qwen, and Mixtral are examples of MoE... Mixture of experts (MoE) has become one of the defining architectural trends in large-scale AI model training. DeepSeek, Qwen, and Mixtral are examples of MoE models that match or exceed the performance of dense model counterparts at a fraction of the training compute. MoE models provide efficient training through conditional computation. Instead of one dense feed-forward network (FFN) shared… Source

Evaluates edge-deployable 2–8B vision-language models (Qwen3-VL, Gemma3) versus BioCLIP on species identification from camera-trap imagery.

Layerwise intervention study reveals how Qwen, Llama, Gemma route query info and retrieve internal knowledge across layers during QA.

Kernel-managed shared memory system for multi-agent personalization with privacy controls tested on GPT-4o, Llama-3.1, Qwen-2.5.

Controlled study of mid-training domain allocation on Qwen3-8B shows per-domain coverage optima exist but alignment cannot undo suboptimal choices.

Analysis of Qwen2.5-1.5B rollouts reveals within-group verifier-error correlation of 0.530, suggesting group-based RL with automatic verifiers requires design-effect adjustment for statistical validity.

NVFP4 W4A4 quantization on all 496 linear layers of Qwen3.8-27B hybrid LLM including Gated DeltaNet recurrent blocks maintains performance across evals.

Graph Machine architecture uses dynamic sparse routing with O(n) state for efficient pretraining; reduces Qwen3-0.6B compute by replacing 75% dense layers.

Comparison of seven LoRA variants for dysarthric ASR adaptation on Whisper and Qwen3 baselines.

SonicCaps: 15M-caption audio dataset with fine-grained descriptions from Qwen3-Omni for improved audio-language retrieval.

Empirical study on Qwen2.5-3B and Phi-3.5-mini shows hints in code generation provide steering, not missing information; unrelated hints rescue 40-50% as many failures.

HiveTraceGuard-Pro, a 0.6B LoRA guardrail model tuned on Qwen3, detects Russian and English prompt injection and jailbreaks with binary safety scoring.

LLM-labeled NER training for Persian customer chat anonymization, comparing DeepSeek-V3, Qwen3, and GPT-OSS labeling quality.

Controlled study of 13 LLM scales (Qwen, GPT) on ontology learning tasks shows mixed effects of model size on performance.

Study finds KV-cache quantization in RAG systems reduces faithfulness to retrieved evidence even when accuracy holds on Qwen2.5-7B.

Puro-2B: open-source 1.5B LM trained on RTX 5090 for <$5k, targeting cost-accessible pretraining.

Eval-aware CoT framings predict model compliance differently; capabilities-framing on Qwen3-32B strongly predicts rule-following.

Qwen releases Qwen3.8-Flash-Next, a 125B-parameter MoE model with 6B active tokens and multimodal capabilities, previewing Qwen4 architecture.

Alibaba released the model weights for Qwen3.8-Flash-Next as a preview of the upcoming Qwen4 architecture for developers to experiment with and evaluate. It’s... Alibaba released the model weights for Qwen3.8-Flash-Next as a preview of the upcoming Qwen4 architecture for developers to experiment with and evaluate. It’s a multimodal mixture-of-experts (MoE) model with a 125B-parameter main model supplemented by an additional 51B N-gram embeddings, with 6B parameters activated per token. It has a native 262,144-token context window, extensible to 1M tokens… Source

LocalLSTC architecture adds persistent control context to GUI agents; replacing GPT-5 with Qwen3.5-9B drops OSWorld success from 60.9% to 37.7%.

Study identifies seven measurement artifacts in LLM self-improvement claims; Qwen3-8B LoRA self-training masks false gains without proper null controls.

Qwen 3.8 27B scores 52 on the Artificial Analysis Intelligence Index That's the same score as GPT-5.6 Luna (max), and just one point behind GLM-5.2 (max) and DeepSeek V4 Pro 0813 (max) - that GLM is 753B and that DeepSeek is 1.6B parameters, and Luna is size unknown but presumably a whole lot bigger than 27B. Qwen 3.8 27B is a truly astonishing model . Via Hacker News Tags: ai , generative-ai , llms , qwen , ai-in-china , artificial-analysis

Friday's big release was Qwen 3.8 27B , an Apache 2 licensed 27B parameter vision-capable LLM from Alibaba's Qwen research lab. I've been looking forward to this one: 27B is an excellent size for running a model on a reasonably specced laptop, and its predecessor Qwen 3.6 27B was impressive. Qwen's self-reported benchmarks for this model are eye-opening. They show a boost from both Qwen 3.6 27B and the closed-weight Qwen 3.7-Plus, which was one of Qwen's strongest models of any size as recently as May this year . It will be interesting to hear what independent benchmarks have to say about the...

Running large AI models on resource-constrained edge devices requires model compression to reduce model size and computation. What compresses well, however, need not deploy well. We survey dozens of recent works that report compression results on real hardware and extract practical deployment guidelines from them. Following these guidelines, we deploy compact language and image models on GPU, CPU, and Raspberry Pi platforms across question answering and image segmentation. No single technique wins across tasks. For question answering, Qwen3.5 0.8B reaches 93.85 SQuAD F1 and 92 EM under Q5_K_M...

Tool: CORS Chat I built this today ( with GPT-5.6-Sol xhigh ) to help test Qwen 3.8 27B running in LM Studio on both my M5 MacBook Pro and an NVIDIA DGX Spark. It provides a web UI for exercising an OpenAI-Responses-compatible chat endpoint. I've tried it against LM Studio with the --cors option and OpenRouter, and both work fine. Conversations are persisted in the browser and can be exported as copy-pasted JSON. One fun detail is that it notices SVG images that are being generated and progressively renders them in the chat while the tokens are still streaming in. Tags: svg , ai , generative-...

Alibaba released the open weights for Qwen3.8-2.4T-A95B (Qwen3.8-Max), its largest open-weight model, bringing near-frontier capabilities to the open... Alibaba released the open weights for Qwen3.8-2.4T-A95B (Qwen3.8-Max), its largest open-weight model, bringing near-frontier capabilities to the open ecosystem. It has 2.4T total parameters with 95B activated per token. It has 2.4T total parameters with 95B activated per token. It’s a fine-grained mixture of experts (MoE) architecture with a hybrid of full and linear attention, a context window of… Source

Reinforcement learning with verifiable rewards yields no group-relative signal when rollout groups are uniformly correct or uniformly wrong, which account for 63.0-68.0% of groups in our experiments. We propose SKALD (Skill-Anchored Latent Distillation), an on-policy self-distillation framework that uses two context views of the same Qwen3-Base model: a question-only student and a teacher conditioned on an abstract, explicit-answer-filtered skill card. The student is trained on its own prefixes, transferring the skill-induced advantage into shared parameters without privileged input at test t...

Reranking medical procedures against patient queries is a critical component of health insurance information retrieval, complicated by a substantial lexical gap between patient language and clinical nomenclature. We present a systematic comparison of two reranking paradigms for this production task: (1) small cross-encoders (MedCPT, MiniLM-L12) fine-tuned with listwise learning-to-rank objectives across layer freezing configurations, and (2) Qwen3-Reranker-4B, a 4B-parameter instruction reranker whose prompt is iteratively refined via an agentic optimization loop driven by GPT-4.1. On a purpo...

LLM inference accounts for over 90% of AI operational energy, scaling directly with input token count---a critical inefficiency for telecom network analytics and numerical time-series data analysis (NTSDA), where raw multivariate KPI windows from 4G/5G cell sites expand into thousands of floating-point tokens. Vision-Language Models (VLMs) eliminate this mismatch by encoding time-series as 2D plots, achieving 3.6-10.4x input token reduction across Llama-3.2-90B, Qwen2.5-VL-72B, and Pixtral-12B architectures. This translates to 1.8-2.5x measured inference energy reduction, saving approximately...

Agogic demonstrates music tokenization—not model scale—drives text-to-symbolic-music generation quality, using Qwen backbone with seven representations and Frechet Music Distance.