All papers

The paper introduces a benchmark evaluating agent susceptibility to prompt and script injections across multi-tool shared workspace environments.

Introduces a benchmark evaluating coding agent misbehaviors across code generation, file management, and system environment integrity.

Develops an open agent environment for evaluating how autonomously language agents can conduct post-training machine learning experiments.

The paper shows that large language model agents fail to retrieve relevant skills when large numbers of irrelevant skills populate the metadata library.

The paper investigates decision failures in fast single-pass agent models, reporting that wrong actions often occur despite correct reading of the environment state.

The paper introduces an automated weak-to-strong hacker-fixer loop to find and repair vulnerabilities in agent evaluation verifiers.

The authors report on a team of autonomous Gemini 4 Argon agents collaborating via chat and shared code to beat Minecraft survival mode.

The paper demonstrates that internal activation states can forecast and block unauthorized actions and deceptive reports by language model agents.

The authors develop a reinforcement learning method to calibrate an agent's per-step confidence against the final outcome of an episode.

Evaluates pull requests from software agents modifying both application code and deployment infrastructure in continuous integration pipelines.

The paper proposes executable contracts to verify that adversarial benchmark edits preserve the core task definition when evaluating tool-using agents.

The paper presents an enterprise case study showing that portable skills and structured navigation layers simplify retrieval pipelines in customer support.

This paper analyzes methodological failure modes in data science agents, such as data leakage and invalid evaluation, beyond simple code accuracy.

The paper investigates on-policy indirect manipulation attacks that steer autonomous agent purchasing and ranking decisions via web retrieval.

Studies agent-to-agent negotiations over money, finding that the receiving agent's characteristics primarily dictate the final transfer amounts.

This paper introduces a benchmark to test if coding agents can build reliable validation test harnesses from natural language specifications.

This paper introduces a repository-grounded benchmark to evaluate how well coding agents follow engineering constraints and developer workflows.

Demonstrates that reward-hacking strategies can propagate across multiple agents through shared memory and code in multi-agent coding systems.

Analyzes security incidents from frontier AI agent evaluations and proposes a framework for proactive boundary assurance.

Evaluates typed decision models used as agent guardrails, uncovering vulnerabilities and asymmetric error rates in allow-or-block actions.

Introduces a benchmark evaluating guard models on identifying unfulfilled safety obligations in language model agent trajectories alongside forbidden actions.

The paper introduces a multilingual benchmark evaluating safety and discrepancies between verbal refusals and tool actions in tool-calling language models.

This paper introduces a benchmark to measure whether autonomous coding agents truly improve their policies from interaction traces.

Analyzes coding-agent trajectories to model execution costs under prompt caching, showing turn count matters far more than observation token length.

Presents an agentic system with human-governed adaptation to automate root-cause analysis across complex, large-scale infrastructure environments.

Analyzes authorization security flaws in agent human-in-the-loop approval mechanisms and proposes a tamper-resistant approval gateway.

The authors introduce a benchmark to evaluate whether personal agents can distinguish accepted user arrangements from unacknowledged proposals.

The authors introduce Bemula, a benchmark designed to evaluate how accurately language model agents erase specific user memories upon request.

The paper studies how much models contribute to multi-model teams, showing individual test scores do not correlate well with collaborative performance.

The paper presents a taxonomy that locates agent workflow failures across specific workflow stages and agent functional capabilities.

The paper evaluates judge labels on agent steps by restarting failed trajectories from intermediate states to identify true failure points.

The paper analyzes the failure modes of agent memory reuse and evaluates certified memory update strategies when facts change.

The paper presents CWE-bench, a benchmark of 120 held-out software tasks to evaluate AI agents on finding and repairing security vulnerabilities.

Shows that language agents often ignore identified cross-domain dependencies, causing unintended disruptive actions in network management.

The paper introduces a benchmark to evaluate whether language-model guardrails properly distinguish authorized from unauthorized actions across identical operations.

Introduces a policy-guided benchmark evaluating language models acting as authorization gates for tool-using coding agents.

The paper audits agent evaluation benchmarks for software defects, showing that broken reward functions and configuration shifts substantially distort reported success rates.

The paper investigates privacy leakage caused by multi-agent network topologies and proposes an execution sanitizer conditioned on recipient identity.

Presents a benchmark and attack method to evaluate failures caused by tool results misbinding to incorrect execution contexts in LLM agents.

Provides a framework for assessing risks of proposed changes to deployed AI agents by selectively replaying execution logs and saved states.

This paper introduces TIPER, a method that evolves challenging user personas to test language model agent robustness without invalidating underlying tasks.

The paper presents an evaluation benchmark assessing seven clinical process dimensions for healthcare agents across longitudinal workflows.

The paper introduces an evaluation framework measuring how communication protocol exploits propagate into task failures across multi-agent systems.

The paper introduces EvolvAttack, demonstrating a backdoor mechanism that activates only after a vision-language agent evolves its memory over time.

The paper introduces a benchmark based on enterprise projects to evaluate agent performance on clarifying underspecified requirements and iterative optimization.

The paper introduces a framework that progressively constructs and streams executable training tasks from code repositories for software engineering agents.

The paper presents an adversarial framework where an exploitation agent iteratively attacks and refines automated patches generated by a repair agent.

Recreates and analyzes infrastructure misconfigurations that enabled privilege escalation and repository bypass during autonomous AI agent evaluations.

The paper measures the information gap between coding agents' internal execution logs and their user-facing progress reports.

The paper evaluates harnesses for legislative document conversion, showing structured multi-agent pipelines narrow performance gaps between models.

The paper proposes an execution harness that validates intermediate results before allowing scientific agent workflows to release outputs.

The paper introduces an executable evaluation platform and benchmark to measure contextual integrity and privacy compliance in multi-step language model agents.

The paper presents a benchmark that isolates agent memory evolution from downstream retrieval and generation failures.

The paper demonstrates that a single adversarial agent can cause benign models in multi-agent discussions to quietly adopt false beliefs.

The paper formalizes the theoretical limits of automated gates evaluating self-improvement updates in autonomous agent code.

The paper introduces an adversarial editing pipeline that alters agent execution transcripts to red-team action-monitoring safety systems.

The paper introduces an end-to-end benchmark to evaluate indirect prompt injection risks across long-horizon agent workflows.

Develops a post-training framework that trains computer-use agents for video games using synthetic interactive experiences and an expanded action space.

The authors introduce FDE-Gym, a benchmark of 95 customer delivery tasks evaluating AI agents on ambiguous requirements and live software environments.

The paper introduces Agent-Reflex, a training framework designed to teach language agents how to reflect and recover from noisy environment feedback.