Every paper checked — nothing hidden
All papers
Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.
All3,345In brief3More (on-topic)587Not relevant2,721Not assessed24Older45Could not confirm25Blogs with no paper54All blogs57
| Paper | Topic | Relevance | Feedback |
|---|---|---|---|
| Identity, Incentives, and Cooperative Behavior for AI: Reverse-Engineering Multiagent Cooperation The paper develops an analytical framework to interpret and reverse-engineer incentives and cooperative behaviors in multi-agent artificial intelligence systems. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Provides experimental designs for multi-agent cooperation rather than standardized LLM evaluation benchmarks. | |
| Embedding Geometry Shapes LLM Routing: Tracing Failure Modes and Improving Generalization The paper uses intermediate transformer hidden states instead of semantic embeddings to improve classifier-based prompt routing across unseen tasks. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on routing classifier algorithms rather than LLM evaluation benchmarks or methodologies. | |
| Finite-sample certification of coverage regret for budgeted document selection under adaptive stratified auditing The paper derives statistical confidence bounds on coverage regret for selecting budget-constrained document subsets in retrieval systems under adaptive auditing. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on statistical audit bounds rather than LLM evaluation or benchmark design. | |
| Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory Modules The paper presents a method to retrofit attention models with hierarchical test-time neural memory modules to improve long-context inference efficiency. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Evaluates on standard context benchmarks without proposing new evaluation methodologies or benchmarks. | |
| Generalizing Identifiability Results in Causal Discovery from Complete Data to Missing Data Settings The paper provides theoretical conditions and algorithms for preserving causal graph identifiability when working with missing data distributions. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Deals with causal graph identifiability, not LLM evaluation benchmarks or methodologies. | |
| Narrowing the Collaboration Gap, Probably The paper examines communicating calibrated action probabilities rather than direct actions to improve multi-agent collaboration under partial information. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Studies multi-agent communication mechanisms rather than benchmark design or general evaluation methodologies. | |
| REPORT-OPS: Hybrid Evidence-Guided Reporting with Human-Supervised Refinement for HPC The paper presents an agent that combines language models with operational policies to generate verified operational reports for high-performance computing systems. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Presents an HPC reporting agent rather than general LLM evaluation benchmarks. | |
| Do Syntax Heads Influence Model Behavior? The paper analyzes whether syntactic attention heads in transformers play a direct causal role in syntactic language predictions. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on mechanistic interpretability of syntax heads rather than benchmark evaluation. | |
| Write Once, Plan Many: Building Reusable Visual Planning Systems from One Scene The paper presents a framework using vision-language models to generate reusable visual planning code from a single practice scene. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on visual planning synthesis rather than LLM evaluation benchmarks. | |
| Refined Topology-Pass Scheduling: Damped Trust-Region Compensation for Dynamic k-NN Graphs in SE(3)-Equivariant Structure Prediction The paper develops an efficient two-pass scheduling method to update nearest-neighbor graphs in SE(3)-equivariant structure prediction models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on molecular structure graph optimization rather than LLM benchmarks. | |
| Copy-Paste Language Models The paper proposes adding an explicit copy-paste prediction head to language models to improve in-context retrieval and copying efficiency. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on model architecture for in-context copying rather than evaluation benchmarks. | |
| MIND: Instruction-Conditioned Incremental Grounding for Robots The paper introduces a framework that incrementally updates robot spatial judgments after environmental changes without full scene recomputation. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on robotic instruction grounding rather than LLM benchmark methodologies. | |
| Beyond Maliciousness: Static and Runtime Attack-Surface Profiling of Agent Skills The paper profiles the security attack surface and execution privileges of third-party agent skills across public marketplaces. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on security profiling of agent tools rather than general LLM evaluation benchmarks. | |
| MemeTrace: A Multimodal LLM-Based Agent for Safety-Aware Meme Provenance Analysis The paper presents a multimodal agent designed to trace the origin, spread, and potential safety risks of online memes. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Presents an agent application rather than benchmark design or general LLM evaluation. | |
| Exploring Interpretability in Deep Learning-Based Forest Classification - Separating Similar Forest Classes using Sentinel-2 Time Series The paper evaluates recurrent neural networks and feature attribution methods for tree-species classification using satellite time-series imagery. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Applies remote sensing classification rather than general LLM evaluation or benchmarking. | |
| Dimension-Free Scaling Laws for Size Generalization: An Exponential Separation The paper establishes theoretical bounds and sample complexity laws for models generalizing across varying input sizes in structured data. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Theoretical analysis of sample complexity rather than empirical LLM evaluation methods. | |
| Whose turn is it? Context-guided model routing for stateful agents The paper investigates context-aware model routing rules across turns in stateful multi-step agent applications. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on model routing optimization rather than benchmark design or LLM evaluation. | |
| Beyond ICD Codes: Semantic IDs for Generative Medical Coding The paper proposes training language models on hierarchically clustered semantic identifiers instead of standard ICD codes for clinical coding. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes a representation framework for medical coding rather than general benchmark methodologies. | |
| DTDD: Divergence-Triggered Dynamic Distillation for Reliable On-Policy Supervision The paper introduces a distillation framework that triggers teacher takeovers when student policies diverge during long-horizon reasoning. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes a model distillation training method rather than an evaluation benchmark. | |
| Generalization Analysis of Biased Stochastic Gradient Methods for Minimax Problems The paper provides a theoretical generalization and stability analysis of biased stochastic gradient methods for minimax optimization problems. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Mathematical analysis of minimax optimization rather than empirical LLM benchmark evaluation. | |
| LLM-jp-4 Technical Report The paper details the development, training data criteria, and performance of the open Japanese sovereign models, LLM-jp-4. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Contains standard model benchmarks rather than proposing new evaluation methodologies. | |
| Why Do Agent Teams Fail Across Boundaries? A Failure Taxonomy for Agent-to-Agent Systems with Private States The paper presents a taxonomy of failure modes in multi-agent systems where agents interact across organizational boundaries with private states. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on multi-agent failure taxonomy rather than general LLM evaluation benchmarks. | |
| When Is Smaller Greener? Right-Sizing Language Models for Agentic AI The paper presents a decision model to evaluate language model efficiency based on energy and cost per completed agentic task. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes a sizing decision model rather than an evaluation benchmark. | |
| Asymptotic Behavior of Transformers The paper mathematically analyzes long-sequence generation in transformers, establishing temperature thresholds required for internal cache convergence. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Mathematical proof of convergence properties, not an empirical benchmark. | |
| Seeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memory The paper provides an analytical framework for how multimodal agents form, represent, and adapt memory over long-horizon tasks. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Provides a conceptual survey framework rather than an evaluation benchmark. | |
| Beyond Corrected Actions: Training LLM Agents on Textual Critiques The paper proposes a fine-tuning method that trains language model agents using textual expert critiques alongside corrected actions. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces a fine-tuning methodology rather than an evaluation benchmark. | |
| Policy-Verified Self-Training: Learning Agent Compliance from Verifiable Policy Rewards The paper introduces a self-training method that uses runtime policy verification feedback as rewards to teach agents safety compliance. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes a training framework rather than an LLM evaluation benchmark. | |
| DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents The paper proposes a self-supervised method that uses repetitive tool calls as negative signals to reduce looping behavior in long-horizon agents. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces a preference optimization training method, not an evaluation benchmark. | |
| EvoWeave: Composing Reusable Agent Experience through Operation Level Graphs The paper introduces an operation-level graph framework that helps interactive agents retrieve and compose reusable fragments from past task experience. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes an agent memory mechanism rather than a standard benchmark. | |
| ixi-sLM: On-Device sLM for AI Call Agent Service The paper presents techniques to adapt and quantize a small language model for on-device deployment in phone call agent services. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Engineering report on model deployment, not an evaluation benchmark. | |
| STRATA: Self-Maintaining Structured Experience Graphs for Self-Evolving Language Agents The paper introduces a structured experience graph memory that organizes and maintains agent workflows across hierarchical abstraction levels. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes an agent memory architecture, not an evaluation framework or benchmark. | |
| ELARSpeech: A 30K-Hour Dataset Across 675 Underrepresented Languages for Long-Tail Multilingual Speech Technologies The paper introduces a speech dataset spanning 675 underrepresented languages to support speech recognition and text-to-speech technologies. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Presents a multilingual speech dataset rather than an LLM evaluation benchmark. | |
| EpiGuard: An Autonomous Infectious-Disease Early Warning Agent Platform The paper presents an autonomous agent platform designed to monitor surveillance data and generate early warnings for infectious disease outbreaks. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Describes an applied autonomous agent platform rather than an LLM benchmark. | |
| SERO: Learning from Executable Oracle Continuations for Long-Horizon Agentic Reinforcement Learning The paper proposes an agentic reinforcement learning method that uses executable oracle continuations to guide exploration during long-horizon tasks. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Presents an RL training method rather than an evaluation benchmark. | |
| Scaling LLM Inference via Parallelism: Throughput-Latency Trade-off The paper presents a systematic framework for selecting optimal parallelism strategies to balance latency and throughput during distributed language model serving. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on inference systems profiling rather than general LLM benchmarking. | |
| Vision-Language Models Integrate Conflicting Evidence Like Biased Bayesian Observers The paper proposes a Bayesian cue-combination model to explain how vision-language models integrate conflicting visual and textual evidence. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes a Bayesian predictive model rather than an evaluation benchmark. | |
| Recursive Formation and First-Class Reuse of Compositional Capabilities The paper introduces a framework for recursively composing, verifying, and persistently reusing modular agent capabilities across diverse problem domains. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Presents a functional framework rather than a general LLM evaluation benchmark. | |
| From Failures to Learnable Worlds: Policy-Conditioned Environment Generation for Navigation Agents The paper presents a method that uses agent failure patterns to generate tailored simulation environments for embodied navigation learning. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Generates targeted training environments, not a general evaluation benchmark. | |
| AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety The paper introduces an adversarial co-evolution reinforcement learning framework to strengthen language model agents against tool-use manipulation attacks. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces a reinforcement learning safety framework, not an evaluation benchmark. | |
| Beyond the Trial-and-Error Loop: Hybrid Projection and Automated Tuning for Distributed Training The paper proposes a hybrid projection and simulation method to automatically configure distributed training parallelisms across GPU clusters. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Presents an automated systems tuning tool, not benchmark methodology. | |
| VideoHarness-SRSI: Structured Recursive Self-Improvement for Video Understanding Harness The paper examines failure modes in recursively self-improving execution harnesses for agentic video understanding models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on optimizing executable harnesses rather than benchmark methodology. | |
| Integrating Simulation and Chain-of-thought Reasoning in Multimodal-Language Models For Physical Reasoning The paper compares physics simulations and chain-of-thought language reasoning for physical problem solving in multimodal language models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Presents a hybrid reasoning method, not an evaluation benchmark. | |
| xLightGCN: Explainable Graph Neural Network Based Recommendation Systems with Most Influential Peers The paper proposes a graph-based method to generate explainable recommendations by identifying influential user peers in graph neural networks. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces an explainable recommender method, not an LLM benchmark. | |
| Are You Reading the Right Thing? Rethinking Internal Signals for Prompt-Injection Detection The paper analyzes five internal representation reading methods to examine how they detect prompt injections in language models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Studies prompt injection detection signals rather than general benchmark validity. | |
| AdCoEvo: A Co-Evolving Multi-Agent Advertising Delivery System The paper presents a multi-agent system designed to automate advertising operations and coordinate tasks like bidding and budget allocation. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Presents an advertising operations framework rather than evaluation benchmarks. | |
| MIPS: Mitigating Mode Collapse in Reinforcement Learning for Large Outcome Spaces The paper proposes marginal inverse probability scaling to reduce mode collapse in reinforcement learning with large outcome spaces. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Develops an RL policy gradient variant, not benchmark methodology. | |
| PFedMA: Who Should Join Next as Federated Multi-Agent Orchestration Evolves? The paper introduces a federated multi-agent framework that guides agent selection and query refinement under dynamic privacy constraints. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Designs a federated framework rather than an evaluation benchmark. | |
| MemCode: Discrete Semantic Representations for Long-Term Agent Memory The paper introduces a discrete codebook representation that pairs associative indexing with vector search for long-term agent memory. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes an agent memory method rather than testing benchmarks. | |
| Beyond Retrieval: Evidence Access and Event Aggregation in Agent Memory The paper introduces an event ledger and executable aggregation framework to help agents accurately query and summarize trajectory history. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes a memory system rather than an evaluation benchmark. | |
| IMU-Bench: Benchmarking Wrist IMU Representations Across Kinematic, Semantic and Clinical Tasks The paper introduces a multi-task benchmark and a masked autoencoder to evaluate transfer learning for wrist-worn inertial measurement units. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Evaluates wearable IMU sensor representation, not LLM benchmark methodologies. | |
| Robust Real-Time Monocular Human Pose Estimation with Insole-Based Sensor Fusion The paper combines camera video and insole sensor signals to improve real-time monocular human pose estimation under visual occlusion. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Evaluates sensor fusion architectures, not LLM benchmark methodologies. | |
| Structure-Preserving Document-Level Data Augmentation via Discourse Parsing The paper introduces a document-level data augmentation method using discourse structure parsing to preserve logical flow during text generation. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes a data augmentation method, not an evaluation framework. | |
| Structure Before Intelligence: Gating What Agentic Scientific Workflows Release The paper proposes an execution harness that validates intermediate results before allowing scientific agent workflows to release outputs. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on execution harnesses rather than general LLM benchmarking. | |
| VLRLA: Deployment-Time Adaptation of Vision-Language-Action Models through In-Context Reinforcement Learning The paper proposes an in-context reinforcement learning method enabling vision-language-action policies to adapt during robot deployment. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Presents an adaptation method rather than benchmarking or evaluation protocols. | |
| Revisiting Triplet Contrastive Learning for 3D Shape Understanding The paper learns 3D shape representations by mapping text hierarchies into hyperbolic space without relying on synthetic 2D renderings. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Develops representation learning losses rather than LLM evaluation benchmarks. | |
| Retrieve What Helps, Only When It Helps: Agentic Reinforcement Learning for Selective Retrieval-Augmented Machine Translation The paper proposes a reinforcement learning framework that decides when and what external knowledge to retrieve for machine translation. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Develops a retrieval framework rather than proposing LLM evaluation methodology. | |
| LabCompiler: A Framework for Verifiable Human-Machine Laboratory Execution from Imperfect Protocols The paper introduces a compiler framework that transforms defective natural-language protocols into verifiable laboratory execution workflows. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces a domain recovery test rather than a general LLM evaluation benchmark. | |
| Evidence-Anchored Overdose Death Prediction from Death Investigation Reports The paper presents an automated framework to predict suspected drug overdose deaths from investigation reports using expert-guided rationales. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Applies NLP to a specific domain task rather than LLM benchmarking methods. | |
| TEX-MEX: Similarity-Guided Temporal Contrastive Learning for Time Series The paper presents a contrastive learning method for time series that unifies morphological similarity and temporal proximity. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Does not evaluate large language models or LLM benchmarks. | |
| REWIND: Uncertainty-Guided Trajectory Repair for Test-Time Reasoning The paper introduces a training-free controller that uses token entropy to target and regenerate faulty reasoning steps at test time. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes an inference-time reasoning algorithm rather than benchmark creation or evaluation. |
1,321–1,380 of 3,345← PreviousNext →