Every paper checked — nothing hidden
All papers
Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.
All3,345In brief3More (on-topic)587Not relevant2,721Not assessed24Older45Could not confirm25Blogs with no paper54All blogs57
| Paper | Topic | Relevance | Feedback |
|---|---|---|---|
| In-Context Learning for System Identification: Probabilistic Formulation, Long-Sequence Handling, and Benchmark Validation Develops a probabilistic transformer meta-learning architecture to perform zero-shot system identification directly from long training sequence contexts. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Applies meta-learning to dynamical systems rather than evaluating LLMs on standardized tasks. | |
| Web$hell: A Unified Interface for Agentic Web Access Proposes Web$hell, a unified interface combining broad information gathering, browser interactions, and evidence management for web-based agents. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces an agent web-interaction interface rather than a standardized evaluation benchmark. | |
| CNN-Transformer with Sigmoid Label for High-Resolution Sub-Degree DOA Estimation Combines a CNN-Transformer network with a continuous sigmoid label encoding to overcome grid quantization errors in direction-of-arrival estimation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on signal processing DOA accuracy rather than LLM evaluation or benchmark critique. | |
| Local Diagnostics of Learned Event Representations for Anomaly Detection at the HL-LHC Evaluates neural embedding and diagnostic strategies to interpret detected anomalies in high-luminosity collider particle physics data. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Studies physics collision anomaly detection rather than evaluating general language models. | |
| Rollout-Time Intervention: Unlocking Agentic Reinforcement Learning Beyond Policy Limits Proposes Rollout-Time Intervention to convert failed agent trajectories into successful off-policy training signals during reinforcement learning. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents an RL training algorithm for agents rather than proposing a benchmark or evaluation methodology. | |
| Collective Autonomous Vehicle Routing Needs Social Awareness for Coexistence with Human Drivers Incorporates marginal-cost rewards into multi-agent reinforcement learning to reduce autonomous vehicle travel times without worsening human driver delays. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Studies traffic routing optimization algorithms rather than language model evaluation or benchmarks. | |
| Agentic Large Language Models Presents a textbook providing an overview of the concepts, architectures, and interactions of agentic large language models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Acts as a broad book on agentic LLMs rather than focusing on evaluation methodologies or benchmarks. | |
| Branching On-Policy Distillation Introduces BranchOPD, which branches student rollouts from shared prefixes at uncertain decision points to improve on-policy distillation efficiency. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces a knowledge distillation technique rather than benchmarking methodology or evaluation critique. | |
| Humanize MLE Agents: Multi-Agent Session Alternation Beyond Single-Agent Limits Presents a framework alternating multi-agent sessions across context resets to overcome reasoning degradation in long-horizon machine learning engineering. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an execution framework for ML engineering agents rather than a standardized evaluation benchmark. | |
| Beyond Temporal Proximity: Explanatory World Models for Credit Assignment in Reinforcement Learning Presents Explanatory World Models, using language model causal explanations to redistribute reinforcement learning rewards across distant actions. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an algorithmic credit assignment framework rather than benchmark analysis or evaluation methodology. | |
| Retrieval-Augmented Multimodal Language Model with Heterogeneous Evidence Integration for Clinical Prognosis Proposes PRISM, a multimodal clinical prognostic model combining patient records with retrieved population-level embedding memory banks. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Evaluates a clinical prognosis architecture on existing benchmarks rather than proposing general LLM benchmark methods. | |
| Causal Effect Identifiability in Causal Abstractions: A Strict Hierarchy of Criteria Formalizes a theoretical hierarchy of identifiability criteria for estimating causal effects across equivalence classes of causal graphs. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Addresses theoretical causal identifiability mathematics rather than evaluating language models. | |
| Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation Proposes Road MOPD, using non-routed teachers as cross-domain critics to resolve gradient conflicts in multi-teacher on-policy distillation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a model training method rather than introducing or analyzing LLM evaluation benchmarks. | |
| Defence against the dark arts of distillation: Unsupervised discovery of subliminal learning Introduces an unsupervised method to identify and remove subliminally transferred traits from model parameter updates during knowledge distillation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on discovering hidden traits in weights rather than general LLM benchmarking protocols. | |
| MENTOR: Learning from Failure with On-Policy Pedagogical Intervention Presents MENTOR, a multi-agent framework that intervenes adaptively during failed reasoning rollouts to prevent zero-reward collapse in RLVR. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a pedagogical intervention training method rather than an evaluation framework or benchmark. | |
| AudioTETRIS: Token Elimination via Transitional Importance Signals for Efficient Audio LLMs Proposes AudioTETRIS, a training-free token pruning framework that uses acoustic transition signals to compress audio representations efficiently. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents an efficient token compression method rather than benchmark design or LLM evaluation. | |
| Humanize Olympiad Agents: A Multi-Session Workflow for Olympiad Problem Solving with Solve, Review, Wiki, Verify Introduces HOA, a multi-session LLM agent framework coordinating review, wiki knowledge, and verification for complex Olympiad problem solving. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an agent workflow rather than introducing or critiquing evaluation benchmarks. | |
| Model-agnostic Knowledge Augmented Machine Learning for Weakly Annotated Bioimaging Data Proposes a model-agnostic method to integrate biological domain knowledge into machine learning models trained on weakly annotated bioimaging. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces a domain-knowledge augmentation technique for bioimaging rather than an LLM benchmark. | |
| Learning Size Independent Constraint Models from Input Data and Positive and Negative Example Solutions Presents a constraint acquisition tool that synthesizes size-independent constraint programs from input data and example solutions. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents a constraint acquisition tool rather than an LLM evaluation suite or benchmark. | |
| SelectArena: A Modern Benchmark for Feature Selection on Real-world Tabular Data Introduces SelectArena, a benchmark designed to evaluate tabular feature selection methods across real-world domain datasets and contemporary models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Benchmarks classical and tabular models on tabular datasets, not LLM capabilities or evaluation methodology. | |
| A Multi-Agent LLM System for Automated Tuning of Specialized Neural Network Models Presents a multi-agent LLM framework that automatically tunes specific internal algorithm components while preserving overall neural network structures. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on an automated model-tuning framework rather than LLM evaluation or benchmark design. | |
| A Step Harder, Not a Step Too Far: The Benefits and Limits of Curriculum Difficulty Progression in GRPO Proposes a dynamic curriculum for GRPO that re-estimates sample difficulty to progressively transition policy training toward harder problems. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Explores training curriculum progression rather than benchmarks, saturation, or evaluation protocols. | |
| Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction Introduces UPTURN, a framework that trains a small classifier to inject redirection prompts as new user turns during reasoning failures. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a self-correction inference technique rather than novel benchmarks or evaluation methodologies. | |
| Not All Dimensions Are Equal: Outlier-Aware Attention Linearization with Grouped Quadratic Kernels Presents Outlier-Aware Attention Linearization, using quadratic approximations to retain outlier dimensions during efficient Transformer attention evaluation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Develops a linearized attention mechanism rather than evaluation benchmarks or LLM-as-a-judge protocols. | |
| Variance-Reduced Policy Gradient is Nearly Minimax-Optimal Introduces Loopless Variance-Reduced Policy Gradient, proving near-optimal sample complexity under weak gradient domination using first-order updates. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Theoretical analysis of policy gradient sample complexity, not an LLM benchmark or evaluation study. | |
| Calibration in One Go: Jointly Learning Accurate and Calibrated Neural Networks Employs a bilevel Stackelberg formulation to jointly optimize classification accuracy and confidence calibration without separate calibration datasets. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Addresses model training loss formulations rather than LLM benchmarking or evaluation frameworks. | |
| Adaptive DHiSparse: Speculative Decoding over a Load-Adaptive KV Hierarchy Combines load-adaptive hierarchical key-value caching with speculative decoding to accelerate long-context language model serving. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Investigates inference serving throughput rather than benchmark methodology or LLM evaluation frameworks. | |
| Amortized Counterfactual Prediction with Graph Neural Networks Presents a graph neural process framework to answer counterfactual queries directly from observational data without prior causal graph discovery. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a prediction architecture rather than an LLM evaluation framework or benchmark. | |
| LAPT-Scale: Scaling Embodied Policies with Latent Action Pre-Training on Human Videos Pre-trains embodied robot policies on human egocentric videos using learned continuous latent action representations. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a training method for robotics rather than a benchmark or evaluation methodology. | |
| Orthros: Phase-Aware Heterogeneous Attention for Efficient Transformers Proposes an attention mechanism that switches from linear complexity during prefilling to full attention during token decoding. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on architectural design for attention rather than LLM evaluation benchmarks. | |
| SafeDiSTL: Decomposing Joint Constraints for Safe Multi-Agent Reinforcement Learning Decomposes joint temporal logic constraint violations to allocate credit among agents in cooperative reinforcement learning. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an RL credit assignment method rather than an LLM evaluation benchmark. | |
| A Privileged Teacher That Only Scores Cannot Search: Critique-Conditioned On-Policy Self-Distillation for Reasoning Examines error localization and credit assignment failures in critique-conditioned on-policy self-distillation for mathematical reasoning. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Investigates distillation credit signals rather than LLM benchmarking or evaluation frameworks. | |
| Matched-Context Group Policy Optimization for LLM Agent Training Constructs context-matched preference comparisons from sibling rollouts to improve credit assignment during language model agent training. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces a reinforcement learning training method rather than an evaluation benchmark. | |
| When Edge Independence Fails: Joint Graph Diffusion with Latent Sociability Priors Introduces a latent sociability prior to mitigate edge-independence issues in discrete graph diffusion models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Studies graph generation priors rather than language model evaluation frameworks. | |
| DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs Proposes a calibration-free post-training quantization method that jointly optimizes weight and activation scaling for 4-bit model inference. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Develops a quantization method rather than an evaluation benchmark for language models. | |
| ReplaySSM: Cache SSM Inputs, Not State Replaces frequent state memory updates in state-space models by caching inputs and periodically checkpointing state during decoding. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an inference acceleration mechanism rather than benchmark or evaluation methodologies. | |
| Your Discrete Graph Diffusion Model is Secretly Memorizing Demonstrates that discrete graph diffusion models exhibit conditional memorization when reverse trajectories are conditioned on corrupted training graphs. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Investigates memorization in graph diffusion rather than LLM evaluation or benchmarks. | |
| FORTE: A Production Framework for Feature Evaluation and Deprecation in Industrial Recommender Systems Describes an industrial framework for evaluating, ablating, and deprecating features in production recommender systems without performance regressions. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Deals with feature deprecation in recommender systems rather than LLM benchmarking. | |
| SpatialASM: Internalizing Active Mentalization for Spatial Reasoning Trains vision-language models to perform active spatial reasoning through novel-view generation and simulated mentalization trajectories. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces a training framework for LVLMs rather than a foundational evaluation methodology. | |
| Studying Comics at Scale with Artificial Intelligence: The PAGE Framework and Three Studies of American Comics and Japanese Manga Applies vision-language models and structured annotation to parse and analyze historical comic books and manga at scale. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Applies existing models to historical analysis rather than evaluating LLM capabilities. | |
| X-AgentRed: Red-Teaming Personal Agents in Evolving Multi-Service Ecosystems Presents an automated red-teaming framework that searches for harmful compositional tool-use paths across multi-service environments. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on red-teaming tool trajectories rather than general LLM capability benchmarking. | |
| RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics Combines dual-encoder semantic retrieval with a contextual bandit to adaptively select agent tools using execution reliability feedback. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant The primary focus is an agent tool retrieval system, not general LLM benchmarking. | |
| Task Context Shapes Component Roles in Language Models Investigates how changing tasks redistributes attention head influence and alters internal activations in open-weight large language models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents mechanistic interpretability analysis rather than standard LLM evaluation benchmarks. | |
| Revisiting Tabular Deep Regression: Objectives and the Limits of Global Early Stopping Evaluates regression objective functions and global early stopping limits across neural network backbones and established tabular datasets. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Investigates regression objectives in tabular neural networks, not language model evaluation. | |
| Hide and Seek: HIDE Concepts to SEEK explanations for black box models Introduces a model-agnostic, concept-level attribution method for black-box vision models by masking concepts located with vision-language models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an interpretability attribution method rather than LLM benchmarking or evaluation frameworks. |
3,301–3,345 of 3,345← Previous