Same Simulation, Different Evidence: Where Retrieved Tool Output Costs LLM Agents Accuracy
GPT-5.1 and Claude agents lose accuracy retrieving tool outputs from external files rather than context windows, measured on USGS PHREEQC geochemistry tasks.
Search the full wire by company, model, lab, or keyword. Every story we have ever aggregated.
GPT-5.1 and Claude agents lose accuracy retrieving tool outputs from external files rather than context windows, measured on USGS PHREEQC geochemistry tasks.
ASTOR benchmarks LLM autoformulation for soft-constraint optimization using formal grammar, addressing gaps in handling preference-based constraints.
Safety Mid-Training internalizes LLM safety during pre-training using shared reasoning structure with general tasks to prevent post-training degradation.
Game-theoretic framework enables LLMs to learn scam-baiting strategies through in-context learning without fine-tuning.
HarnessPro recursively improves AI agent harnesses for performance-cost trade-offs via LLM-generated tests without ground truth.
TokenGate enables learnable token pruning in LLMs for faster, memory-efficient long-context prefill using a differentiable top-K approximation.
Two-layer transformer with logarithmic precision copies arbitrary repeated strings via RoPE rotations, addressing failure modes in frontier LLM copying tasks.
llm-tools-datasette 0.1.1 fixes httpx dependency breakage from LLM 0.33 upgrade.
llm-echo 0.5 debug plugin updates token counting to include tool calls, system prompts, and reasoning tokens.
The vibecoding tag has been getting more and more useless for a while now. It lost the original meaning as the culture of software engineering with LLMs matured a bit. It got silly when posts talking about LLMs in general got tagged. It got useless when posts even mentioning…
One-Relation Attack exploits single implicit warrants to flip LLM agent beliefs; 54.9% success overturning correct answers.
Dense graders collapse jailbreak success class labels inconsistently, invalidating reported attack success rates across models.
Bellman Language Models cache repeated agent decisions as optimization solutions to accelerate bounded agentic workflows.
REP-HO replaces manual heuristics in mixed-fidelity optimization with LLM-based evaluation planning for heterogeneous multi-objective problems.
Retrieval Program Learning enables LLMs to compose search primitives via RL into adaptive multi-step retrieval programs with state and control flow.
MisleadQA diagnostic dataset reveals LLMs fail on out-of-scope but factually true context, not just false or irrelevant information.
PatternBench isolates LLM code optimization from compiler optimizations; measures compiler-transcendent semantic reasoning in code generation.
ZO-FO hybrid framework systematically explores adversarial jailbreak attacks on LLM agents by coordinating zeroth-order and first-order optimization strategies.
Risk-sensitive LLM-judge framework for medical QA evaluates six harm categories (directives, contraindications, dosage) beyond factual correctness.
Table serialization format (row-major vs. column-major) significantly affects instruction-tuned LLM's table lookup accuracy depending on table aspect ratio.
Empirical study of 1,500 LLM refactoring attempts across 15 Java repos shows multi-agent approaches improve dependency-aware code generation over single-prompt baselines.
Theoretical analysis of entropy rates in random composition trees as models of hierarchical text structure, with limited practical implications for LLM development.
Proposes Martingale Posterior Cox Processes for Bayesian nonparametric inference on event data, primarily theoretical contribution with limited LLM relevance.
Demonstrates data-poisoning attacks that plant Trojans increasing LLM response length and token costs, and proposes defenses against semantic trigger activation.
ReTRACE integrates LLM agents with chemistry tools and route scoring for retrosynthesis planning, handling stereochemistry and cost tradeoffs.
ServiceBench benchmark evaluates LLM agents on complaint handling across 2,500 scenarios; finds current agents struggle with emotional reasoning under information asymmetry.
Analysis of repetition lock-in during LLM generation: top-p sampling with threshold >0.5 creates deterministic escape failure points; bounded penalties worsen entrapment.
I'm curious what stance others have taken? Or what they've experienced with this? I'm toying with adding a rule to our values.md that the LLM can only significantly generate one or the other. I'm reviewing a merge request at the moment where it's clear the LLM mostly did both…
Study finds LLM fine-tuning induces unintended behavioral couplings: training for warmth increases sycophancy, unsafe code training causes broader misalignment.
SEER-Repro benchmark evaluates LLM agents on automating reproducibility and robustness analysis of 20 published medical studies using cancer registry data.