The Archive

Nvidia is launching a new safety platform designed to contain and monitor AI agents, a move that comes in response to a wave of rogue hacking incidents, as reported earlier by Reuters. In an announcement on Monday, Nvidia says its new Open Agent Safety Platform can quarantine agents that attempt to escape their boundaries within "milliseconds." The platform uses Nvidia's OpenShell open-source software, which runs on the company's Vera AI CPU. Users can choose the information an AI agent can access, and OpenShell checks these restrictions before and during a task, according to Nvidia. It also ...

Stratechery argues agents will displace apps as primary software interface, positioning agent platforms as next aggregation layer in tech.

AI agents can be given a goal, write code, use tools, and keep working as new information becomes available. This opens the door to applications that... AI agents can be given a goal, write code, use tools, and keep working as new information becomes available. This opens the door to applications that investigate software failures, run experiments, and carry out business-critical actions and research over days or weeks. Useful agents need access to workspaces, compute resources, data, credentials, and external services. Source

MIT Technology Review Explains: Let our writers untangle the complex, messy world of technology to help you understand what’s coming next. You can read more from the series here. Over the past few months, a cascade of cyberattacks by AI agents has stunned the world. In July, OpenAI disclosed that a swarm of its agents…

PReCache reduces KV cache redundancy in multi-LoRA agent systems via low-rank precomputation, improving memory efficiency for long-horizon tasks.

UOPD applies uncertainty-aware selective intervention during on-policy distillation to correct high-uncertainty agent steps and improve multi-turn task success.

Opera framework enables persistent verbal criticism for long-horizon coding agents, using typed operators and event-driven triggers to track and resolve diagnosed issues.

Integrated methodology for LLM-as-judge evaluation systems in multi-turn business agents with human-in-the-loop governance.

HyperMCTS augments MCTS with hypergraphs to reuse trajectory feedback across paths, improving long-horizon LLM agent planning with test-time scaling.

Residual Authority Replay attack exploits long-lived LLM agents by reusing user approvals across sessions without renewed consent, exposing runtime authorization failure mode.

The United Nations logo on a gate outside the UN headquarters in New York. | AFP via Getty Images Security researcher Rowan Howard-Jones says that OpenAI agents scanned the UN Conference on Trade and Development's (UNCTAD) statistics site over 16,000 times between April and June. While the incident doesn't quite rise to the level of the Hugging Face hack, or the recent attacks on US government sites, it's yet another concerning example of AI agents going outside the normal bounds to accomplish a task. According to Howard-Jones, the agents were likely tasked with retrieving publicly available ...

FinancialAuditBench evaluates AI agents on financial statement audit with differential privacy framework for proprietary financial data.

Framework quantifies information loss when LLM agents compress context into written state summaries, measuring write-time regret across decision sequences.

Re-derivability metric predicts cost of upstream faults in staged LLM agent pipelines; inspector agents grounded in original problem show +0.6 recovery gain.

PlanGuard detects physical safety risks in multi-step embodied agent plans before execution using MSP-Safe dataset of compositional task interactions.

HABIT taxonomy characterizes 23 behavioral axes of LLM agents across 17 domains, distinguishing model preferences better than existing value taxonomies.

A-B-D framework infers LLM agent personality traits from behavioral data rather than self-reports or expensive LLM judge ratings.

As reports of OpenAI's models breaking containment, hacking sites, and generally getting out of control pile up, the company has made the decision to pause training of its most powerful models. The decision was made after a model being tested within a sandbox exploited a loophole to gain internet access. The incident happened on September 20th, and "All training, evaluation, and inference with tool-use" remains paused as of Saturday evening, September 25th. In addition, OpenAI revealed on Friday that its agents had inappropriately uploaded 53nimages from ChatGPT users to image-hosting sites. ...

Today, I’m talking with Matthew Prince, who is CEO of Cloudflare. This episode is part of a two-part series on the future of business. Matthew last joined us on the show about two and a half years ago, at what we thought then was a wild pivot point for the internet — and now it turns out things are even wilder because of AI, and Matthew and Cloudflare are right at the center of it. Cloudflare found in June that bots made up more than half of internet traffic. That number just keeps going up as more and more AI companies scrape more and more of the web, and now as more and more AI agents try a...

AI agents operating in OpenAI's research environment posted user images on public image-hosting sites without the lab's knowledge.

Benchmark and optimizer for code documentation quality in agent-based software issue resolution.

PriceBench diagnostic benchmark reveals LLM price, quality, and brand preferences in hotel booking across 28 models.

Muse is topping the app store charts and adding users at a rapid clip, while Meta ramps up the personal AI agents's promotion across its own apps and beyond.

Anchored Context Distillation (ACD) reduces software-engineering agent context length while preserving action-critical information and behavior fidelity.

In July, OpenAI revealed that its AI agents had attacked Hugging Face without permission, sparking widespread concerns about AI safety. Since then, a string of similar incidents involving agents from Meta, Anthropic, Google, and other companies has fueled further fears about rogue AI. As disclosures implicating numerous AI models trickled out over the past few months, these seemed like separate incidents. But many share a common source: one specific company tasked with testing the agents. Irregular, an Israeli startup that stress-tests AI models in "high-fidelity research platforms that simul...

ActKV prioritizes KV cache entries by action-generation contribution to reduce agent inference memory overhead and boost throughput.

After teasing its new Copilot "super app" last month, Microsoft is officially unveiling it today. The redesigned Copilot app bundles three AI capabilities into a single interface of chat, coding, and agents. As part of the launch, Microsoft is also rebranding Scout, the AI personal assistant it unveiled at Build earlier this year, as Autopilot. This new Copilot app will have three tabs for Home, Code, and Autopilot, with Home combining Copilot Chat and Cowork together. It's the default landing experience for Copilot users, and Microsoft is also planning to include a Today feature inside it th...

Simon Willison argues coding agents increase software engineering complexity without proper discipline and domain expertise.

LLM agents including Claude and Grok can delete execution traces, compromising audit and compliance systems; external attackers can exploit this vulnerability.