The Archive

Chain-of-Thought (CoT) generation has substantially improved the performance of Large Language Models (LLMs) on complex reasoning tasks, including code generation, data analysis, and exam-style question answering. Despite these advances, purely neural LLMs continue to struggle with elementary logical reasoning problems and lack the determinism, soundness, and reliability characteristic of symbolic reasoning systems. Conversely, classical symbolic methods such as SAT solving and exact cover guarantee correctness and completeness, but require problems to be expressed in highly specialized forma...

Muon has recently emerged as a promising optimizer for pretraining large language models, offering a speedup over AdamW in minimizing pretraining loss. However, it remains unclear whether Muon's advantage persists after fine-tuning. In this paper, we study the downstream behavior of Muon and AdamW and uncover a surprising trend: beyond a sufficiently large pretraining data budget, AdamW outperforms Muon after fine-tuning. Moreover, as model size increases, AdamW overtakes Muon at progressively smaller pretraining token-parameter ratios. To explain this reversal, we view fine-tuning as a pertu...

Large language models (LLMs) show growing promise for scientific discovery, yet basic questions remain about what enables reliable discovery. How can we trace the scientific discovery capacity of LLMs to their training data? We study this question as one of missing probability mass and develop a theoretical account in which monofacts-the facts observed once during training-set the discovery capacity. In materials science and regulatory genomics, we directly manipulate monofact rates in training sets and find that more monofacts consistently yield more discoveries across two open-weight model ...

Multimodal benchmark for emotion detection across text, audio, and video distinguishing concealment from ordinary inconsistency.

PSM-Bench evaluates LLM reasoning on perovskite photovoltaics; OpenPerov framework learns from 4,532 papers to guide materials design.

TopoQuant uses topological data analysis to measure structural drift in quantized LLMs, revealing compression-performance tradeoffs.

Gradual pruning with sparse adaptation preserves LLM quality at extreme sparsity better than one-shot methods.

BioBERT and BioMistral evaluated on TOP benchmark for clinical trial outcome prediction using domain-specific representations.

Compares LLM strategies (CoT, LoRA, SLM-enhanced) for extracting entity relations from academic paper abstracts and full text.

Neuro-symbolic framework combines LLM decoding with grammar refinement to guarantee formal specification satisfaction.

Representation learning analysis shows harmful and harmless queries are distinguishable at intermediate LLM layers, enabling over-refusal reduction.

Claude Opus 5 infers difficulty patterns in USMLE/PANCE exam items using LLM-induced rubrics to identify interpretable factors beyond prediction.

SocialArena benchmark evaluates LLM social reasoning across mental health, tutoring, and companionship via three-tier framework.

LLMON post-training framework uses structured notation to reduce prompt injection and improve control, trained on 4.6B tokens.

TGTV framework integrates spatiotemporal graph learning with LLMs for interpretable UAV fault detection and diagnosis.

Large Language Models (LLMs) are increasingly used in multi-turn agentic workloads that require sustained interaction with complex environments. Existing reflective prompt optimizers such as GEPA learn from failed rollouts, but agentic failures can also come from prompt weaknesses, harness limitations, stochastic execution, or noisy judges. We argue that such failures may not provide useful learning signal, and may even be harmful; and thus it is essential to curate which tasks are provided to the prompt optimizer for better performance. We introduce GEPA-Curator, a framework that augments th...

Scientific discovery often begins with an unexpected observation that prompts a new research question. In a world with altered physical laws, can large language models (LLMs) notice unexpected behavior, such as heavier objects falling more slowly, and discover the law through experiments without being told what to investigate? To evaluate this ability, we introduce LongChang, an automated framework that creates simulated worlds with altered physics, where one or more laws differ from ordinary physics. The evaluated LLMs act as explorers, manipulating objects and building tools to investigate ...

NPC-RubricBench evaluates LLMs across six longitudinal cancer care phases using 941 patient-phase units from 405 nasopharyngeal carcinoma cases.

Reva accelerates real-time vulnerability detection in LLM-generated code via contextual probing, reducing multi-minute reasoning loops to practical screening latency.

Self-training framework enables post-training LLM improvements without external teachers, using only high-level topic guidance to generate curriculum.

LLaCO: continuous diffusion for non-autoregressive text generation with competition-aware semantic scheduling to avoid token overtaking.

Empirical benchmark of recent LLMs vs. fine-tuned encoder-decoder models on knowledge-graph-to-text generation, narrowing performance gap.

Mechanistic interpretability reveals causal subspaces controlling metaphor representation in LLMs across architectures.

LLMs combined with constraint reasoning to generate contrastive explanations for MIP-based optimization decisions, addressing interpretability in DSS.

As large language models (LLMs) continue to grow in scope and capability, privacy concerns are on the rise. Since users increasingly interact with a single general-purpose system across many distinct contexts, there is a natural tendency to pursue a one-size-fits-all solution to privacy. Unfortunately, this overlooks a crucial insight from decades of privacy research: privacy is inherently contextual. In this paper, we argue that existing work in prompt privacy is drifting towards attempts to design a unified framework, which is highly misleading and can foster a false sense of security. Furt...

BitMTP combines ultra-low-bit quantization with multi-token prediction for efficient LLM decoding without quality loss.

ACE-Compression framework evaluates which core LLM abilities survive compression beyond task performance metrics.

LLMs hallucinate in cross-language code translation via semantic conflicts, target-language violations, and inconsistent dependencies; existing mitigations insufficient.

LatComm introduces reusable latent context compression between LLMs, enabling compact encoding that is readable across multiple models and questions.

Mental World Model framework for simulating subjective cognition and Theory of Mind extends LLMs beyond linguistic inference with dynamic evolution.