All papers

The paper evaluates judge labels on agent steps by restarting failed trajectories from intermediate states to identify true failure points.

The paper analyzes the failure modes of agent memory reuse and evaluates certified memory update strategies when facts change.

The paper presents CWE-bench, a benchmark of 120 held-out software tasks to evaluate AI agents on finding and repairing security vulnerabilities.

Shows that language agents often ignore identified cross-domain dependencies, causing unintended disruptive actions in network management.

The paper introduces a benchmark to evaluate whether language-model guardrails properly distinguish authorized from unauthorized actions across identical operations.

Introduces a policy-guided benchmark evaluating language models acting as authorization gates for tool-using coding agents.

The paper audits agent evaluation benchmarks for software defects, showing that broken reward functions and configuration shifts substantially distort reported success rates.

The paper investigates privacy leakage caused by multi-agent network topologies and proposes an execution sanitizer conditioned on recipient identity.

Presents a benchmark and attack method to evaluate failures caused by tool results misbinding to incorrect execution contexts in LLM agents.

Provides a framework for assessing risks of proposed changes to deployed AI agents by selectively replaying execution logs and saved states.

This paper introduces TIPER, a method that evolves challenging user personas to test language model agent robustness without invalidating underlying tasks.

This paper introduces WildTrace, a benchmark evaluating training data detection methods across realistic, multi-stage language model post-training pipelines.

The paper presents an evaluation benchmark assessing seven clinical process dimensions for healthcare agents across longitudinal workflows.

The paper introduces an evaluation framework measuring how communication protocol exploits propagate into task failures across multi-agent systems.

The paper introduces EvolvAttack, demonstrating a backdoor mechanism that activates only after a vision-language agent evolves its memory over time.

The paper introduces an agentic reinforcement learning approach that uses formal program verifier diagnostics as intermediate rewards to produce verified code.

The paper presents an adversarial framework where an exploitation agent iteratively attacks and refines automated patches generated by a repair agent.

Recreates and analyzes infrastructure misconfigurations that enabled privilege escalation and repository bypass during autonomous AI agent evaluations.

The paper measures the information gap between coding agents' internal execution logs and their user-facing progress reports.

The paper evaluates harnesses for legislative document conversion, showing structured multi-agent pipelines narrow performance gaps between models.

This paper evaluates how reliably typed decision models switch to fallback options and remain consistent under equivalent decision rules.

The paper proposes an execution harness that validates intermediate results before allowing scientific agent workflows to release outputs.

The paper introduces an executable evaluation platform and benchmark to measure contextual integrity and privacy compliance in multi-step language model agents.

The paper reveals vulnerabilities in prompt-predictability metrics used to estimate human contribution in AI-assisted writing.

This paper examines the verifiability gap in quantized model supply chains, showing that common GGUF artifacts lack necessary build environment disclosures.

The paper presents a benchmark that isolates agent memory evolution from downstream retrieval and generation failures.

The paper demonstrates that a single adversarial agent can cause benign models in multi-agent discussions to quietly adopt false beliefs.

The paper formalizes the theoretical limits of automated gates evaluating self-improvement updates in autonomous agent code.

Introduces a benchmark measuring how output-only model judges fail to detect tampered runtime environments in agent deployments.

Examines failure modes in label-free evaluation where models reach high consistency across perturbations while generating identical incorrect answers.

The paper introduces an adversarial editing pipeline that alters agent execution transcripts to red-team action-monitoring safety systems.

The paper introduces an end-to-end benchmark to evaluate indirect prompt injection risks across long-horizon agent workflows.

The paper evaluates output non-determinism in commercial language model deployments during medical device regulatory compliance assessments.

Evaluates probability calibration and operational allow-block-defer policies for prompt-injection detectors under distribution shifts.

The paper defines evaluation metrics to show that language models continue generating reasoning traces even when instructed not to think.

The paper introduces Agent-Reflex, a training framework designed to teach language agents how to reflect and recover from noisy environment feedback.

The paper formalizes a framework and theoretical generalization bounds for evolving an agent's software harness without updating model weights.

The paper introduces a benchmark evaluating the robustness of typed decision models against text, image, and prompt injection attacks.

The paper introduces a benchmark to measure how tool-calling agents perform when interacting with emotionally varied, impatient users.

The paper analyzes reward hacking in verifiable instruction following and introduces a likelihood bonus method to maintain answer quality.

The paper introduces ProbabilityBench to evaluate model uncertainty calibration against analytically computed, ground-truth instance-level probabilities across three text environments.

This paper proposes a rubric-based reward model that derives criteria strictly from instructions and evaluates responses pointwise using tool assistance.

The paper proposes a training-free framework that uses uncertainty signals to escalate computer-use tasks from weak models to strong models.

This paper introduces UnCoBench to benchmark how language model agents handle unhelpful, incomplete, or missing communications from partner agents.

This paper evaluates language-model-generated disaster situation reports against human-written reports using a domain-specific sixteen-slot information schema.

This paper audits whether decision models and generative language models can serve as valid synthetic respondents for psychometric surveys.

The authors identify a derivation gap where LLM judges reject correct tool arguments lacking direct textual references, and propose a dedicated reference model.

Introduces Contrastive Vision On-policy Distillation to prevent vision-language models from relying on answer shortcuts rather than visual evidence.

The paper audits an enterprise confidence scoring mechanism for text-to-SQL models on the BIRD and Spider benchmarks.

The paper presents PrivacyGuard, a framework using source-linked sanitization and local references to prevent personal data leakage in cloud GUI agents.

Introduces a training objective enabling language models to update beliefs using report lineage rather than raw citation repetition.

Proposes a conformal routing method that accounts for model failure dependencies to optimize multi-model query coverage under risk budgets.

Identifies and evaluates temporal, semantic, and population preprocessing misalignments that cause data leakage and bias in clinical machine learning pipelines.

Presents a Monte Carlo simulation framework to quantify distributional response shifts and model risks across language model lifecycles.

Introduces a benchmark evaluating how misleading visualizations and charts affect the multi-step execution and decisions of web agents.

This paper proposes PUlSE, using internal layer representations to detect factual errors and trigger step-level retrieval during medical reasoning.

The paper benchmarks how caregiver-directed language models compromise medical advice when users express clinically inferior care preferences.

The study reports that natural code transformations confuse vulnerability analysis agents, proposing functional changes as a potential defensive strategy.

The paper demonstrates a hybrid architecture where cloud-generated agent code delegates localized private-data queries to edge models.

The study evaluates using language models to directly reconstruct mobile network schemas versus generating validation code tools.