Every paper checked — nothing hidden
All papers
Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.
All3,345In brief3More (on-topic)587Not relevant2,721Not assessed24Older45Could not confirm25Blogs with no paper54All blogs57
| Paper | Topic | Relevance | Feedback |
|---|---|---|---|
| CAT-alyzing Honesty: Continuous Adversarial Training for Transferable LLM Honesty Presents an adversarial training approach designed to improve language model honesty across diverse benchmarks without degrading core capabilities. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses primarily on adversarial training methods rather than benchmarking. | |
| Multi-Target Few-Shot Prompting with Diffusion Language Models Uses diffusion language models to perform multi-target few-shot infilling across shared code context guided by demonstration examples. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a prompting method rather than an evaluation framework. | |
| Training Reasoning Models to Generate Diverse Scientific Ideas Introduces a set-level reinforcement learning objective to train language models to produce diverse and document-grounded scientific research ideas. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses primarily on training methods rather than general evaluation methodology. | |
| Value-Informed Schedule Trimming for Accelerated Sequential Monte Carlo Guidance in Discrete Diffusion The paper introduces a training-free schedule optimization method for accelerating sequential Monte Carlo guidance in discrete diffusion models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Develops a guidance algorithm rather than an LLM evaluation framework or benchmark. | |
| When Correctness Feedback Erases Evidence: Phase-Separated Completeness Auditing in Verus The paper demonstrates how correctness feedback undermines formal specification auditing and proposes a phase-separated alternative for Verus. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Examines specification auditing dynamics rather than general LLM evaluation benchmarks. | |
| Counterfactual Neighborhood Risk Fields: Evidence-Grounded Selective Safety for Vision-Language Models The paper proposes a selective safety framework for vision-language models using counterfactual pairs and representation density estimation. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a safety defense mechanism rather than establishing a general evaluation benchmark. | |
| IneqAgent: Agentic Inequality Proving in Lean with Symbolic Computation The paper introduces an agentic framework combining language model reasoning with symbolic engines for formal inequality proving in Lean4. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Applies existing math benchmarks to evaluate a theorem-proving system rather than benchmarking methods. | |
| L2G-SR: Learning to Guide LLM-based Symbolic Regression The paper presents a framework that uses structural tree edit distance to guide language models in symbolic regression. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Uses an existing benchmark to test a regression method, rather than proposing evaluation standards. | |
| Interpreting Language Model Parameters The paper introduces a parameter decomposition method to interpret the mechanistic subcomponents of language model weights. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies parameter decomposition rather than benchmark saturation, LLM-as-a-judge, or evaluation protocols. | |
| Optimal Hydra: Model Heterogeneous Federated Learning for LLMs using Data-Informed Hydra-Nets The paper proposes a federated learning framework that tailors sub-network sizes and architectures to heterogeneous client hardware and data. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a federated training algorithm rather than benchmarking or evaluation protocols. | |
| Reference-Guided Turn-Taking Style Transfer for Full-Duplex Spoken Language Models The paper identifies an internal activation direction in spoken language models to control and transfer conversational turn-taking styles. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on steering spoken conversational style rather than LLM benchmarking methodology. | |
| When Reality Outpaces Cognition: A New Scaling Principle for Reliable Intelligence The paper presents a metric and benchmark measuring how quickly environment dynamics can change before agent decision-making fails. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a conceptual scaling metric rather than standard LLM evaluation frameworks. | |
| Verdict: Encode Once, Decide Many The paper introduces an encoder architecture that processes input once to answer multiple structured classification queries simultaneously. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Applies existing benchmarks to evaluate a model architecture rather than studying benchmarks themselves. | |
| ARIA: Runtime Inspection and Amendment for Multi-Agent Systems The paper presents a framework and dataset for learning to detect and repair errors during multi-agent system execution. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a dataset of repair trajectories for training, not a general evaluation benchmark. | |
| ReasonUni: Reasoning-Augmented Reconstruction Advances Unified Multimodal Models The paper introduces a reasoning-augmented reconstructive post-training framework that unifies multimodal understanding and image generation tasks. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a model training framework rather than benchmarking or evaluation methodology. | |
| Open-world AI: a Tale of Three Requirements The paper formalizes three core architectural requirements for artificial intelligence systems operating in open-world settings. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies generalization theory rather than benchmarking or evaluation methodology. | |
| EvolvingBrain: An Adaptive Skill System for Multi-modal Agents Self-Evolution in Open Worlds The paper proposes an adaptive skill framework that extracts patterns from trajectories to enable multi-modal agents to self-evolve. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on agent architecture rather than evaluating general benchmark reliability. | |
| When Reasoning Traces Conflict with Evidence: How Answer Configuration Shapes Evidence Following The paper analyzes how answering via a separate model call improves evidence adherence when reasoning traces conflict with provided context. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Analyzes evidence-following mechanisms rather than establishing a general evaluation benchmark. | |
| A Self-Improving Agent that Grows its Own DSL: Oracle-Free Operator Induction for Exact Question Answering The paper presents an agent that expands its domain-specific language for exact answering without external rewards or judges. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents an agent induction method, not an evaluation benchmark. | |
| Temporal Specification Guidance for Text-to-Video Generation The paper introduces an inference-time guidance method that enforces temporal logic constraints during text-to-video generation. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a generation guidance method, not an evaluation framework. | |
| Wolof-Adapted Industry-Ready Open-Weight Language Models The paper presents open-weight language models adapted for the Wolof language using instruction tuning, preference alignment, and model merging. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Releases adapted models rather than novel evaluation methodologies. | |
| CoTranslator: Reverse to Forward Chain-of-Thought Translator for Reasoning Distillation The paper proposes a method to translate reverse reasoning paths into verified forward chains of thought for distillation. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on distillation data generation, not evaluation methodology. | |
| Amortizing Sequential Experimental Design in Reasoning LLMs The paper trains reasoning models to perform sequential experimental design by optimizing information-gain rewards during action selection. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on policy training for information gathering, not benchmarking. | |
| Confidence-Calibrated Inference Expansion for Evaluator-Guided Test-Time Reasoning The paper introduces a confidence-calibrated controller to optimize inference compute and test-time expansion decisions in reasoning models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on inference control methods rather than benchmark datasets. | |
| SHRED: Self-Play for Heterogeneous Road-Users without Expert Demonstrations The paper presents a self-play reinforcement learning framework to simulate interactions between vehicles, pedestrians, and cyclists without human demonstrations. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Traffic policy simulation, not LLM benchmarking. | |
| Anticipatory Commonsense with Moving Objects: An ASP-Based Framework for Predictive Reasoning with Interacting Trajectories The paper proposes a hybrid reasoning framework combining Answer Set Programming and energy-based models to predict interacting trajectory futures. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on trajectory reasoning rather than LLM evaluation. | |
| Wrong Votes, Right Gradients: Why Majority-Vote Rewards Track Verifiable RL The paper analyzes why reinforcement learning with majority-vote rewards provides effective training gradients despite noisy individual labels. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Analyzes training dynamics rather than model evaluation methods. | |
| TGrasp6D: Task-Informed Vision-Language Priors and Pose Refinement enable Zero-Shot Task-Oriented 6DoF Grasping The paper presents a zero-shot vision-language method for generating task-oriented robotic grasps in 6 degrees of freedom. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Robotic manipulation paper, not LLM evaluation or benchmarking. | |
| Replicable Dueling Bandits The paper develops replicable algorithms with sublinear regret bounds for the dueling bandits problem. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on bandit algorithms rather than LLM benchmarking methods. | |
| Forecast lead time and the timing of crew deployment for weather-driven power outages using AI weather models The paper evaluates how forecast lead times from AI weather models affect the scheduling of utility crew restoration deployments. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates weather forecast lead times, not language models. | |
| A Multi-Tiered AI Accessibility Model (MAIAM): Proportionate Access and Capability Limits for Comprehensive Frontier AI Safety The paper proposes a multi-tiered governance model regulating AI system access based on user purpose and model capability. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Discusses governance framework rather than technical LLM benchmark methods. | |
| Defining and Generating Compositional Structure The paper formalizes compositional structure in complex systems using algorithmic information theory and compression concepts. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Theoretical mathematical framework, not an LLM benchmark. | |
| SkillNEO: Non-Elitist Evolutionary Optimization of Agent Skills The paper proposes a non-elitist evolutionary optimization framework that retains and iteratively refines low-reward agent skills using execution feedback. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents an optimization method rather than an evaluation methodology. | |
| RAC: A Relational Agent Contract for Constraint-Governed Engineering Tasks The paper introduces an action and feedback contract separating agent decisions from deterministic tool execution for constraint-governed engineering tasks. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an agent architecture rather than an evaluation benchmark. | |
| Screwtape: Adversarial Transcript Editing for Monitor Red-Teaming The paper introduces an adversarial editing pipeline that alters agent execution transcripts to red-team action-monitoring safety systems. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on red-teaming monitors rather than general benchmark evaluation. | |
| LEV: Thinking Fast and Deep with Looped Decision Models The paper presents a decision model that performs deeper reasoning by looping its layers instead of generating output tokens. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a model architecture rather than an evaluation benchmark. | |
| Same Context, Different Agents: A Behavioral Study of Semantic Layers, Orchestration and Compaction in Data Agents The paper studies how semantic layers, orchestration, and compaction impact the performance and behavior of data agents across benchmarks. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Conducts a behavioral study of context techniques rather than introducing a benchmark. | |
| StepSafe: Learning Timely Intervention with Context-Aware Safety Reasoning The paper introduces a training framework to teach AI agents when to intervene proactively and safely during multi-step tasks. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an intervention training method rather than an evaluation benchmark. | |
| MSP: Modality Self-Play for Training Multimodal LLMs Without Paired Cross-Modal Data The paper proposes a self-play framework to train multimodal foundation models using text proxies instead of paired cross-modal datasets. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a multimodal training framework rather than an evaluation benchmark. | |
| SOMA: Stack-of-Objects Manipulation using a Quadruped Robot The paper presents a hierarchical reinforcement learning framework for non-prehensile 3D manipulation of multi-object stacks using quadruped robots. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates robotics locomotion policies rather than LLM benchmarks. | |
| How to Train Your Little Dragon: Teaching a Small Language Model to Construct and Refine GPU Optimization Scripts The paper trains a small language model to generate and refine GPU kernel optimization scripts using compiler traces. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a compiler optimization training technique rather than a benchmark. | |
| EigDiff: Eigensample Diffusion - or how to significantly reduce training dataset size with marginal performance variation? The paper uses latent eigenvectors from CLIP to select smaller, representative training datasets for diffusion models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies training data selection rather than model evaluation benchmarks. | |
| Deep Reinforcement Learning for HPC Job Scheduling: A Statistical Evaluation The paper evaluates deep reinforcement learning methods against simple heuristics and random baselines for high-performance computing job scheduling. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates classical RL algorithms rather than LLM benchmarks or evaluation methods. | |
| Failure Is the Mother of Success: Recovering Supervision with Grounding Hindsight The paper introduces a method that turns failed reasoning attempts into tested supervision notes to improve reinforcement learning training. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a training optimization method rather than an evaluation benchmark. | |
| Learning to Read Out: Unembedding Dynamics in Language Model Pretraining The paper investigates when language models learn capabilities by tracking unembedding matrix dynamics during pretraining using parameter-trajectory crosscoding. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Investigates representation probing during pretraining rather than LLM benchmarks. | |
| SCARCE: Emergent Institutions in Long-Horizon Multi-Agent Coding Systems The paper studies how autonomous coding agents spontaneously negotiate and coordinate access to shared compute resources under deadlines. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies multi-agent emergent behavior rather than standard LLM evaluation benchmarks. | |
| Recursive Reward Improvement via Adversarial Meta-Prompt Evolution Proposes an adversarial meta-prompt evolution method to improve model-generated evaluation rubrics for open-ended tasks. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Addresses RL training optimization rather than benchmark methodologies. | |
| ReviewPilot: A Multi-Agent System for Screening and Data Extraction in Systematic Reviews Presents a multi-agent system for paper screening and data extraction in systematic literature reviews following standard guidelines. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents an application system rather than an evaluation benchmark. | |
| Learning When to Withdraw: Curriculum Coverage and Rule-Specific Failures in Fine-Tuned Language Agents Studies how fine-tuning helps tool-using agents learn to withdraw and repair actions when environment assumptions change. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies agent action withdrawal training rather than proposing a general benchmark. | |
| Athanor: An Agent-Native System for Autonomous Post-Training Research Develops an open agent environment for evaluating how autonomously language agents can conduct post-training machine learning experiments. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces an agent system for post-training rather than an evaluation benchmark. | |
| Bellman-Consistent Trajectory RL for LLM Arithmetic Reasoning Applies trajectory Bellman residual minimization to turn sparse rewards into dense token signals for arithmetic reasoning. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an RL training method rather than an evaluation framework or benchmark. | |
| Approve What, Exactly? Effect Binding and Trusted Rendering for Agent Step-Up Authorization Analyzes authorization security flaws in agent human-in-the-loop approval mechanisms and proposes a tamper-resistant approval gateway. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a security protocol rather than a standard LLM evaluation benchmark. | |
| SIEVE: Overcoming Topological Obstruction in Equivariant Self-Supervised Learning Introduces an equivariant self-supervised learning framework that avoids topological collapse in 3D point cloud masked autoencoding. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a self-supervised learning framework rather than LLM benchmarking or evaluation. | |
| Probe-and-Resume: When and How to Evaluate Agent Execution Evaluates test-time search in computer-use agents, finding critics perform better when invoked only at predicted final states. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies agent search policies rather than standardized benchmarks or LLM evaluation suites. | |
| MemComposer: Learning to Assemble Memories for LLM Agents via Reinforcement Learning Uses reinforcement learning to train an agent module that selects, sizes, and orders retrieved memories for downstream tasks. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates agent memory assembly rather than establishing general LLM evaluation benchmarks. | |
| Planning Commitment of LLM under Partial Observability Analyzes trade-offs between aggressive and conservative planning horizons in language models operating under partial observability. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies planning behaviors under uncertainty rather than proposing general LLM evaluation benchmarks. | |
| SSR: Structural Signal Retriever for Scalable Agent Discovery in LLM-Orchestrated Multi-Agent Systems Proposes a training-free structural lexical retriever to select relevant agent cards without overloading model context windows. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Addresses agent context scalability rather than LLM evaluation frameworks or benchmarks. | |
| DST-Net: Decomposed Spatio-Temporal Network for Time Series Forecasting Presents a decomposed network architecture that applies separate graph and convolution modules to trend and seasonal time series components. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a time series architecture rather than LLM evaluation benchmarks or methodologies. | |
| OnBodyAI - Why the Next AI Needs a Body Presents an opinion article on wearable on-body language models based on an industrial panel talk. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents an industrial talk overview rather than benchmarks or evaluation methodologies. | |
| CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF Proposes an optimal transport method using small clean feedback sets to calibrate sample-wise noise for reward model training. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Addresses reward modeling training objectives rather than LLM evaluation or benchmark suites. |
841–900 of 3,345← PreviousNext →