The Archive

Door-in-the-face psychological technique increases LLM refusal compliance; works on Anthropic Opus 5 (65.8%) but not OpenAI frontier models.

LLMs automate vulnerability injection into Solidity smart contracts to generate ground-truth datasets for testing vulnerability detection tools across 49 OpenSCV types.

Persona prompting steers LLMs toward survey-participant response alignment; empirical study of attribute effectiveness for population prediction.

Debias-SparseGPT addresses bias amplification in pruned LLMs using representational debiasing with demographically contrasting inputs.

SDARE-Bench benchmarks LLMs on stigma detection and response generation in conversational dyadic and group dialogue with 2,526 scenarios.

SinkPruner removes high-norm outlier visual tokens from MLLMs to reduce inference cost while maintaining multimodal understanding quality.

SwarmBench: benchmark for evaluating LLM-based agent orchestration across accuracy, efficiency, cost, and process quality in dynamic multi-agent systems.

Retain-free identity unlearning in MLLMs via localized early-decoder MLP intervention, enabling post-deployment privacy removal without retain sets.

Continued pre-training adapts LLMs to Swedish journalism domain with novel benchmark across six editorial tasks.

Study across 30 LLMs showing linguistic confidence diverges from internal confidence (logits/semantic entropy) on classification and generation tasks.

Real-time semantic navigation framework for UAV object-goal search using LLMs to reason over spatial context and prioritize regions.

BrailleBench: new benchmark evaluating LLM comprehension of Braille across five criteria with 5,570 instances for accessibility assessment.

StrategyBench evaluates whether LLMs can induce explicit task rules from examples in few-shot learning, addressing sensitivity to example construction.

Thermo-FL adds thermal-aware control and Byzantine robustness to federated LoRA fine-tuning of LLMs on edge devices.

COEC proposes alternating orthogonal compensation for training-free structured pruning of LLMs to reduce inference cost.

ConceptGuard benchmark evaluates LLM unlearning at concept level, measuring harmful behavior removal while preserving benign knowledge.

Ramp has launched its own AI model routing service, dubbed Router, that lets users and companies use and switch between various large language models via an API.

RuleMaze benchmark evaluates MLLMs on rule-compliant spatial planning in mazes with natural-language constraints.

FormalTCS benchmark evaluates LLMs on frontier theoretical computer science research with 175 expert-validated instances from STOC/FOCS/SODA/COLT 2025–2026.

Large language models (LLMs) are increasingly used in settings where textual summaries, numerical observations, and external tool outputs may provide conflicting evidence. We study how LLMs arbitrate between such sources when they support opposing decisions. To do so, we introduce a controlled synthetic benchmark in which latent risk trajectories generate both numerical time series and natural language summaries, allowing us to construct conflicts where exactly one evidence source is aligned with the ground-truth label. This design lets us independently manipulate modality, temporal recency, ...

We introduce OenoBench, a wine-domain knowledge benchmark of 3,266 multiple-choice questions across six pillars (regions, grape varieties, viticulture, winemaking, producers, business) and four difficulty tiers. The corpus is built from 38,104 atomic, source-anchored facts extracted by 35 provenance-verified scrapers from government registries (INAO, TTB, OIV), peer-reviewed journals, and Wikipedia/Wikidata. Our methodological contribution is an LLM-driven pipeline in which language models reformat verified facts and audit the result, but never serve as the source of truth: every claim traces...

LLMs extract scientific data via self-authored prompts and cross-model consensus, matching expert curation while reducing human annotation burden.

C3LM: chemistry language model trained on 45.6M reactions using Top-K prompting and reward-guided fine-tuning for retrosynthesis.

LLM-based framework for explaining flight safety events from pilot behavior, addressing modal inconsistency challenges.

LLM hidden state manifolds organize as small-world networks enabling multi-hop reasoning; bypasses attention artifacts via geometric analysis.

Large Language Models (LLMs) have demonstrated capabilities in in-context learning, task decomposition, step-by-step reasoning, and code generation, driving their gradual evolution from text generation models into the core of agents capable of perceiving environments, invoking tools, and executing tasks. Traditional LLM Agents typically obtain information through webpages, documents, databases, or external tools and generate corresponding invocation sequences according to user goals; when this technology is further integrated with robotic systems, large language models begin to undertake func...

Large Language Models (LLMs) are increasingly being deployed in cybersecurity operations to assist cybersecurity analysts with rapid decision-making against emerging threats. However, there is a main criteria that must be met when using LLMs in cybersecurity, that is, trust in the generated outputs. As Agentic AI is integrated into operational systems, a robust evidence attribution and provenance tracking technique is essential to trace the origins of model generations. When autonomous agents make a decision (right or wrong), the ability to trace back through the decision chain is critical, a...

Financial document validation in production, such as payroll auditing, tax compliance, and loan underwriting, demands exceptional accuracy, consistency, and reproducibility under strict enterprise constraints. In practice, documents arrive with heterogeneous layouts and formats, semantically rich and context-dependent content, and embedded business rules that current pipelines struggle to process reliably. We introduce LAVA (Logic-Aware Validation and Augmentation), a modular, backbone-agnostic pipeline built on multimodal large language models, that integrates a four-stage design: document-r...

Large Language Models (LLMs) have achieved remarkable progress in code generation, yet ensuring correctness in complex, repository-level tasks remains challenging. Existing approaches often use generated tests as static post-hoc validators, which limits their ability to guide implementation and may introduce misleading feedback when the tests themselves are incomplete or incorrect. In this paper, we introduce TDD-Agent, which operationalizes the test-driven development paradigm for code generation. TDD-Agent first prompts the model to generate executable tests, encouraging it to clarify expec...

Reinforcement learning algorithms for Large Language Models (LLMs) are largely distinguished by their variance reduction strategy. Group-relative methods like GRPO reduce gradient variance by sampling multiple rollouts per prompt, but provide only sequence-level credit. Training is also blocked by straggler rollouts, reducing throughput and increasing off-policyness. Learned value functions theoretically address both problems, providing token-level advantages without requiring large groups. However, additional infrastructure engineering challenges combined with the practical success of critic...