Every paper checked — nothing hidden

All papers

Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.

PaperTopicRelevanceFeedback
In-Context Learning for System Identification: Probabilistic Formulation, Long-Sequence Handling, and Benchmark Validation
Develops a probabilistic transformer meta-learning architecture to perform zero-shot system identification directly from long training sequence contexts.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Applies meta-learning to dynamical systems rather than evaluating LLMs on standardized tasks.
Web$hell: A Unified Interface for Agentic Web Access
Proposes Web$hell, a unified interface combining broad information gathering, browser interactions, and evidence management for web-based agents.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Introduces an agent web-interaction interface rather than a standardized evaluation benchmark.
CNN-Transformer with Sigmoid Label for High-Resolution Sub-Degree DOA Estimation
Combines a CNN-Transformer network with a continuous sigmoid label encoding to overcome grid quantization errors in direction-of-arrival estimation.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Focuses on signal processing DOA accuracy rather than LLM evaluation or benchmark critique.
Local Diagnostics of Learned Event Representations for Anomaly Detection at the HL-LHC
Evaluates neural embedding and diagnostic strategies to interpret detected anomalies in high-luminosity collider particle physics data.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Studies physics collision anomaly detection rather than evaluating general language models.
Rollout-Time Intervention: Unlocking Agentic Reinforcement Learning Beyond Policy Limits
Proposes Rollout-Time Intervention to convert failed agent trajectories into successful off-policy training signals during reinforcement learning.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Presents an RL training algorithm for agents rather than proposing a benchmark or evaluation methodology.
Collective Autonomous Vehicle Routing Needs Social Awareness for Coexistence with Human Drivers
Incorporates marginal-cost rewards into multi-agent reinforcement learning to reduce autonomous vehicle travel times without worsening human driver delays.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Studies traffic routing optimization algorithms rather than language model evaluation or benchmarks.
Agentic Large Language Models
Presents a textbook providing an overview of the concepts, architectures, and interactions of agentic large language models.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Acts as a broad book on agentic LLMs rather than focusing on evaluation methodologies or benchmarks.
Branching On-Policy Distillation
Introduces BranchOPD, which branches student rollouts from shared prefixes at uncertain decision points to improve on-policy distillation efficiency.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Introduces a knowledge distillation technique rather than benchmarking methodology or evaluation critique.
Humanize MLE Agents: Multi-Agent Session Alternation Beyond Single-Agent Limits
Presents a framework alternating multi-agent sessions across context resets to overcome reasoning degradation in long-horizon machine learning engineering.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Proposes an execution framework for ML engineering agents rather than a standardized evaluation benchmark.
Beyond Temporal Proximity: Explanatory World Models for Credit Assignment in Reinforcement Learning
Presents Explanatory World Models, using language model causal explanations to redistribute reinforcement learning rewards across distant actions.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Proposes an algorithmic credit assignment framework rather than benchmark analysis or evaluation methodology.
Retrieval-Augmented Multimodal Language Model with Heterogeneous Evidence Integration for Clinical Prognosis
Proposes PRISM, a multimodal clinical prognostic model combining patient records with retrieved population-level embedding memory banks.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Evaluates a clinical prognosis architecture on existing benchmarks rather than proposing general LLM benchmark methods.
Causal Effect Identifiability in Causal Abstractions: A Strict Hierarchy of Criteria
Formalizes a theoretical hierarchy of identifiability criteria for estimating causal effects across equivalence classes of causal graphs.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Addresses theoretical causal identifiability mathematics rather than evaluating language models.
Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation
Proposes Road MOPD, using non-routed teachers as cross-domain critics to resolve gradient conflicts in multi-teacher on-policy distillation.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Proposes a model training method rather than introducing or analyzing LLM evaluation benchmarks.
Defence against the dark arts of distillation: Unsupervised discovery of subliminal learning
Introduces an unsupervised method to identify and remove subliminally transferred traits from model parameter updates during knowledge distillation.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Focuses on discovering hidden traits in weights rather than general LLM benchmarking protocols.
MENTOR: Learning from Failure with On-Policy Pedagogical Intervention
Presents MENTOR, a multi-agent framework that intervenes adaptively during failed reasoning rollouts to prevent zero-reward collapse in RLVR.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Proposes a pedagogical intervention training method rather than an evaluation framework or benchmark.
AudioTETRIS: Token Elimination via Transitional Importance Signals for Efficient Audio LLMs
Proposes AudioTETRIS, a training-free token pruning framework that uses acoustic transition signals to compress audio representations efficiently.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Presents an efficient token compression method rather than benchmark design or LLM evaluation.
Humanize Olympiad Agents: A Multi-Session Workflow for Olympiad Problem Solving with Solve, Review, Wiki, Verify
Introduces HOA, a multi-session LLM agent framework coordinating review, wiki knowledge, and verification for complex Olympiad problem solving.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Proposes an agent workflow rather than introducing or critiquing evaluation benchmarks.
Model-agnostic Knowledge Augmented Machine Learning for Weakly Annotated Bioimaging Data
Proposes a model-agnostic method to integrate biological domain knowledge into machine learning models trained on weakly annotated bioimaging.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Introduces a domain-knowledge augmentation technique for bioimaging rather than an LLM benchmark.
Learning Size Independent Constraint Models from Input Data and Positive and Negative Example Solutions
Presents a constraint acquisition tool that synthesizes size-independent constraint programs from input data and example solutions.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Presents a constraint acquisition tool rather than an LLM evaluation suite or benchmark.
SelectArena: A Modern Benchmark for Feature Selection on Real-world Tabular Data
Introduces SelectArena, a benchmark designed to evaluate tabular feature selection methods across real-world domain datasets and contemporary models.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Benchmarks classical and tabular models on tabular datasets, not LLM capabilities or evaluation methodology.
A Multi-Agent LLM System for Automated Tuning of Specialized Neural Network Models
Presents a multi-agent LLM framework that automatically tunes specific internal algorithm components while preserving overall neural network structures.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Focuses on an automated model-tuning framework rather than LLM evaluation or benchmark design.
A Step Harder, Not a Step Too Far: The Benefits and Limits of Curriculum Difficulty Progression in GRPO
Proposes a dynamic curriculum for GRPO that re-estimates sample difficulty to progressively transition policy training toward harder problems.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Explores training curriculum progression rather than benchmarks, saturation, or evaluation protocols.
Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction
Introduces UPTURN, a framework that trains a small classifier to inject redirection prompts as new user turns during reasoning failures.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Proposes a self-correction inference technique rather than novel benchmarks or evaluation methodologies.
Not All Dimensions Are Equal: Outlier-Aware Attention Linearization with Grouped Quadratic Kernels
Presents Outlier-Aware Attention Linearization, using quadratic approximations to retain outlier dimensions during efficient Transformer attention evaluation.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Develops a linearized attention mechanism rather than evaluation benchmarks or LLM-as-a-judge protocols.
Variance-Reduced Policy Gradient is Nearly Minimax-Optimal
Introduces Loopless Variance-Reduced Policy Gradient, proving near-optimal sample complexity under weak gradient domination using first-order updates.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Theoretical analysis of policy gradient sample complexity, not an LLM benchmark or evaluation study.
Calibration in One Go: Jointly Learning Accurate and Calibrated Neural Networks
Employs a bilevel Stackelberg formulation to jointly optimize classification accuracy and confidence calibration without separate calibration datasets.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Addresses model training loss formulations rather than LLM benchmarking or evaluation frameworks.
Adaptive DHiSparse: Speculative Decoding over a Load-Adaptive KV Hierarchy
Combines load-adaptive hierarchical key-value caching with speculative decoding to accelerate long-context language model serving.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Investigates inference serving throughput rather than benchmark methodology or LLM evaluation frameworks.
Amortized Counterfactual Prediction with Graph Neural Networks
Presents a graph neural process framework to answer counterfactual queries directly from observational data without prior causal graph discovery.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Proposes a prediction architecture rather than an LLM evaluation framework or benchmark.
LAPT-Scale: Scaling Embodied Policies with Latent Action Pre-Training on Human Videos
Pre-trains embodied robot policies on human egocentric videos using learned continuous latent action representations.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Proposes a training method for robotics rather than a benchmark or evaluation methodology.
Orthros: Phase-Aware Heterogeneous Attention for Efficient Transformers
Proposes an attention mechanism that switches from linear complexity during prefilling to full attention during token decoding.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Focuses on architectural design for attention rather than LLM evaluation benchmarks.
SafeDiSTL: Decomposing Joint Constraints for Safe Multi-Agent Reinforcement Learning
Decomposes joint temporal logic constraint violations to allocate credit among agents in cooperative reinforcement learning.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Proposes an RL credit assignment method rather than an LLM evaluation benchmark.
A Privileged Teacher That Only Scores Cannot Search: Critique-Conditioned On-Policy Self-Distillation for Reasoning
Examines error localization and credit assignment failures in critique-conditioned on-policy self-distillation for mathematical reasoning.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Investigates distillation credit signals rather than LLM benchmarking or evaluation frameworks.
Matched-Context Group Policy Optimization for LLM Agent Training
Constructs context-matched preference comparisons from sibling rollouts to improve credit assignment during language model agent training.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Introduces a reinforcement learning training method rather than an evaluation benchmark.
When Edge Independence Fails: Joint Graph Diffusion with Latent Sociability Priors
Introduces a latent sociability prior to mitigate edge-independence issues in discrete graph diffusion models.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Studies graph generation priors rather than language model evaluation frameworks.
DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs
Proposes a calibration-free post-training quantization method that jointly optimizes weight and activation scaling for 4-bit model inference.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Develops a quantization method rather than an evaluation benchmark for language models.
ReplaySSM: Cache SSM Inputs, Not State
Replaces frequent state memory updates in state-space models by caching inputs and periodically checkpointing state during decoding.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Proposes an inference acceleration mechanism rather than benchmark or evaluation methodologies.
Your Discrete Graph Diffusion Model is Secretly Memorizing
Demonstrates that discrete graph diffusion models exhibit conditional memorization when reverse trajectories are conditioned on corrupted training graphs.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Investigates memorization in graph diffusion rather than LLM evaluation or benchmarks.
FORTE: A Production Framework for Feature Evaluation and Deprecation in Industrial Recommender Systems
Describes an industrial framework for evaluating, ablating, and deprecating features in production recommender systems without performance regressions.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Deals with feature deprecation in recommender systems rather than LLM benchmarking.
SpatialASM: Internalizing Active Mentalization for Spatial Reasoning
Trains vision-language models to perform active spatial reasoning through novel-view generation and simulated mentalization trajectories.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Introduces a training framework for LVLMs rather than a foundational evaluation methodology.
Studying Comics at Scale with Artificial Intelligence: The PAGE Framework and Three Studies of American Comics and Japanese Manga
Applies vision-language models and structured annotation to parse and analyze historical comic books and manga at scale.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Applies existing models to historical analysis rather than evaluating LLM capabilities.
X-AgentRed: Red-Teaming Personal Agents in Evolving Multi-Service Ecosystems
Presents an automated red-teaming framework that searches for harmful compositional tool-use paths across multi-service environments.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Focuses on red-teaming tool trajectories rather than general LLM capability benchmarking.
RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics
Combines dual-encoder semantic retrieval with a contextual bandit to adaptively select agent tools using execution reliability feedback.
Peer-reviewed
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
The primary focus is an agent tool retrieval system, not general LLM benchmarking.
Task Context Shapes Component Roles in Language Models
Investigates how changing tasks redistributes attention head influence and alters internal activations in open-weight large language models.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Presents mechanistic interpretability analysis rather than standard LLM evaluation benchmarks.
Revisiting Tabular Deep Regression: Objectives and the Limits of Global Early Stopping
Evaluates regression objective functions and global early stopping limits across neural network backbones and established tabular datasets.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Investigates regression objectives in tabular neural networks, not language model evaluation.
Hide and Seek: HIDE Concepts to SEEK explanations for black box models
Introduces a model-agnostic, concept-level attribution method for black-box vision models by masking concepts located with vision-language models.
Preprint
LLM evaluation and benchmarks
via arxiv · 12 Oct, 16:28
Not relevant
Proposes an interpretability attribution method rather than LLM benchmarking or evaluation frameworks.
3,301–3,345 of 3,345← Previous