The Archive

Theory of Mind benchmarks conflate explicit belief recognition with applied behavior prediction; Llama-3.1-8B shows uneven competencies across tasks.

Multi-directional Perturbation-Rewrite Search uses LLMs to automate heuristic design for NP-hard combinatorial optimization with factorized transformations.

Comparison of semantic retrieval vs. LLM-based strategies for tool selection in robotic systems handling compound natural-language commands.

Trajectory-based benchmark for evaluating whether LLMs adapt recommendations appropriately as decision context changes.

Agri-SCHOLAR combines crop simulation with LLM optimization to generate personalized, weather-responsive farming advisories at individual-farm scale.

SecLog-Bench: 29-scenario cybersecurity benchmark with multi-source evidence and dense reasoning annotations for evaluating LLM log analysis.

Game-theoretic model of truthful-by-omission advertising where LLMs understate competitor ads due to position advantage and click competition.

LLMs scale formal software verification by reducing manual proof engineering burden for systems assurance.

LongZero uses self-generated RL tasks to improve LLM long-context reasoning without external training data.

Environment Duels benchmark evaluates frontier LLMs on creating and solving interactive environments; distinguishes authoring from solving ability.

Quant-Arena benchmark evaluates LLM agents on end-to-end quantitative trading research with A-share market data and out-of-sample testing.

P2NPCBench pairs NP-complete and polynomial-time problems via AgentGAN adversarial rewriting to test LLM reasoning beyond recall on complexity boundaries.

Circuits of LLM Classifiers adaptively sequences multiple LLM predictors as noisy instruments to maximize accuracy within inference budget.

Code Comprehensor uses dependency callgraphs and hierarchical documentation for large repository code understanding by LLMs.

Agent Behavior Maps profile LLM-harness combinations via action log-likelihoods on reference trajectories, enabling performance prediction across 100 LLMs.

EPIC reformulates PPO-style RL for LLM policy training via exact-KL prefix importance-ratio clipping, addressing policy-context mismatch in long-response generation.

HiddenLens, a six-layer BERT-LLM pipeline for detecting implicit assumptions in software requirements to reduce costly late-stage resolution.

TIDE introduces behavior-driven decision-tree induction for fine-grained LLM capability diagnosis beyond aggregate benchmark scores.

FaultGene uses LLM-assisted synthesis to generate fault-augmented test scenarios for hardware-in-the-loop validation of autonomous driving cyber-physical systems.

CulturalRiddles, a benchmark of 62 culturally-grounded riddles across 45 countries and 51 languages, evaluates LLM understanding of non-factual cultural reasoning.

System One decision models for legal authority verification with abstention, tested on CaseHOLD dataset with missing holdings.

ToM2Vec learns continuous theory-of-mind tokens in latent space for multi-agent LLM coordination under uncertainty.

FrontierCS 2 benchmark of 48 open-ended CS R&D tasks measures LLM agent contributions in working research environments.

As large language models (LLMs) continue to grow in scope and capability, privacy concerns are on the rise. Since users increasingly interact with a single general-purpose system across many distinct contexts, there is a natural tendency to pursue a one-size-fits-all solution…

As LLMs are increasingly deployed in value-sensitive contexts, aligning them with human values has emerged as a post-training objective in NLP, aimed at instilling additional values and guiding model behaviour. However, current evaluations remain narrow. Because values are…

OpenAI details LLM-powered ad stack architecture for ChatGPT, separating ads from answers via conversational intent understanding.

Adaptive multi-token prediction uses online control to optimize LLM decoding horizon based on verification feedback, balancing latency and compute.

Diagnostic metric framework analyzes value expression and organization in LLM-generated debate text.

Cache engineering for LLM agents optimizes KV storage and prefilling costs across long trajectories without invalidating cached states.

Token-Energy Law models GPU energy per decoded token as function of active bytes and realized time, enabling efficient LLM deployment.