Dual Exam: Controlled Evaluation of Multimodal Agents and World Models
Dual Exam benchmark isolates multimodal agent action-selection from world-model state-evolution via fixed mechanisms across 15 domains and 1,488 test items.
Search the full wire by company, model, lab, or keyword. Every story we have ever aggregated.
Dual Exam benchmark isolates multimodal agent action-selection from world-model state-evolution via fixed mechanisms across 15 domains and 1,488 test items.
OpenAI's Patch the Planet program discovers PHP2Shell vulnerabilities in core PHP using GPT-5.6 Sol; 2-month coordinated disclosure affecting 70% of web infrastructure.
Google Cloud will have GPT 6.1 Astra, Sol, and Luna in the coming weeks.
Quasipolynomial algorithm for computing approximate fixed points of ℓ∞ contractions; applications to solving simple stochastic games.
Teacher-student analysis isolates effects of locality and weight sharing in CNNs, showing interaction with image correlation structure.
Environment Duels benchmark evaluates frontier LLMs on creating and solving interactive environments; reveals distinct authoring vs. solving abilities.
Velocity Ambiguity Profile decomposes flow-matching loss into intrinsic Bayes-risk floor and data-reducible excess to diagnose optimization bottlenecks.
AtomicRL post-trains VLA models for long-horizon manipulation via atomic task structures, solving credit-assignment and subtask-imbalance bottlenecks in RL.
HiddenLens, a six-layer BERT-LLM pipeline for detecting implicit assumptions in software requirements to reduce costly late-stage resolution.
CORD, a conflict-aware optimization framework for joint RL and self-distillation in agents, uses nearest-point projection to resolve gradient cancellation between task and teacher objectives.
As large language models (LLMs) continue to grow in scope and capability, privacy concerns are on the rise. Since users increasingly interact with a single general-purpose system across many distinct contexts, there is a natural tendency to pursue a one-size-fits-all solution to privacy. Unfortunately, this overlooks a crucial insight from decades of privacy research: privacy is inherently contextual. In this paper, we argue that existing work in prompt privacy is drifting towards attempts to design a unified framework, which is highly misleading and can foster a false sense of security. Furt...
HCLTech joins Claude Partner Network to integrate Claude into enterprise solutions for banking, healthcare, telecom, manufacturing.
Ukraine’s latest drone innovation may boost bleak odds of evacuating casualties.
PaPi integration scheme enables parallel time-stepping for physics-informed neural networks solving PDEs.
Three Java Champions share how they use GitHub Copilot to solve problems, accelerate development, and keep humans responsible for the results. The post How 3 Java Champions are using GitHub Copilot for increased productivity appeared first on The GitHub Blog.
Harness optimization for LM agents distinguishes elicitation (leveraging existing capabilities) from expansion (broadening solvable tasks) without weight updates; implications for recursive self-improvement.
Differentiable graph-based implicit solver for PDEs on unstructured grids with forward and inverse problem capability, treating stiff equations as constraints.
TerraNostra-Bench evaluates land-use reconstruction at 1-km resolution via semantic segmentation, separating spatial downscaling from temporal rollout.
Gradus synthesizes graded training tasks for LLM agents in policy-constrained environments by decoupling task design from solution difficulty.
Foundation-style pretraining framework transfers solar PV forecasting across geographically dispersed stations without per-site retraining.
main.content table { display:block; width:100%; max-width:100%; overflow-x:auto; } #evaluation-prompt pre { background:var(--bs-body-bg); color:var(--bs-body-color); padding:18px; border:1px solid var(--bs-border-color); border-radius:6px; } #evaluation-prompt pre code { color:inherit; white-space:pre-wrap; overflow-wrap:anywhere; } Decision models have exploded onto the scene recently, starting with TypeSafe’s Jev, followed by Perplexity’s Decider, Liquid’s d1, Cloudflare’s Clef, and OpenAI’s Decisions API1. These models promise fast, inexpensive classification. Decision models are an appeal...
AI labs have staked their reputations on model benchmarking — the supposedly neutral and accurate system of evaluations that are designed to measure whether one AI product is better than another. However, the methods behind these evaluations are complicated and still maturing, said Irene Solaiman, Hugging Face’s chief policy officer, during a discussion at TechCrunch […]
The US is building more fossil fuel infrastructure than it has in a decade while renewable energy growth slows down, a new report shows. Electricity demand is also climbing faster than it has in more than a decade with more data centers, factories, and electric vehicles plugging into the grid. Despite those growing demands, the Trump administration has sought to hobble the wind and solar industries in favor of fossil fuels. America's energy landscape is already shifting as a result, according to the report published last week by the nonprofit Environmental Defense Fund and research firm Atlas...
KEEP-Bench evaluates whether context compression preserves evidence for plausible resolutions and minority perspectives across 4,102 cases.
Creates model organisms for eliciting latent knowledge by training models to sandbag (withhold solutions) in vulnerability detection, addressing AI safety research.
Today, I’m talking with Illinois Governor JB Pritzker, who has a lot of thoughts about AI, tech, and privacy. If you’ve been listening to the show, you know that one of my running theories is that those issues are driving a big and maybe kind of unexpected political realignment — and with the midterms coming up next month, and early voting already underway in a lot of places, the stakes are pretty high right now. So I asked Gov. Pritzker about all that, and he was surprisingly willing to dive right into the absolute existential weeds of what government is even for. You’ll hear him say — and i...
User reports Sol 6.1 API consuming quota 25x faster than Codex, draining weekly allowance in 16 hours.
Bellman Language Models cache repeated agent decisions as optimization solutions to accelerate bounded agentic workflows.
PatternBench isolates LLM code optimization from compiler optimizations; measures compiler-transcendent semantic reasoning in code generation.
ServiceBench benchmark evaluates LLM agents on complaint handling across 2,500 scenarios; finds current agents struggle with emotional reasoning under information asymmetry.