Every paper checked — nothing hidden
All papers
Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.
All3,345In brief3More (on-topic)587Not relevant2,721Not assessed24Older45Could not confirm25Blogs with no paper54All blogs57
| Paper | Topic | Relevance | Feedback |
|---|---|---|---|
| RAG-RST: Retrieved Exemplars as In-Context Discourse Schema for RST Parsing The paper uses retrieved in-context exemplars to improve rhetorical structure discourse parsing without requiring model fine-tuning. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a parsing method, not an LLM benchmark framework. | |
| Specification Gaming and a Constrained Reinforcement Learning Mitigation in a Cleaning Gridworld The paper demonstrates specification gaming in a simulated cleaning task and tests a constrained reinforcement learning mitigation. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Analyzes specification gaming in gridworld RL, not LLM benchmarks. | |
| EMO: A Closed-Loop Test-Time Computation Framework via Alternating Context Optimization and Logits Steering The paper proposes an alternating context optimization and logit steering framework to adapt language model outputs at test time. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an inference adaptation method, not an evaluation benchmark. | |
| Multi-sourced AI Tutor for Non-Native English Speaker Students The paper develops a multilingual retrieval-augmented tutoring system providing bilingual responses and glossaries for international computing students. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on educational QA application, not LLM benchmark design. | |
| Mosaic: Multi-Session Agentic Reinforcement Learning The paper introduces a hierarchical reinforcement learning method designed for agentic tasks that span multiple interaction sessions. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an RL training algorithm, not an evaluation benchmark. | |
| Motion- and Gravity-Aware Feedforward 3D Reconstruction The paper trains feedforward 3D reconstruction models using privileged motion and gravity cues to improve dynamic scene understanding. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Addresses computer vision 3D reconstruction rather than LLM benchmarks. | |
| Whole-Body Dexterity The paper introduces a reinforcement learning pipeline enabling humanoid robots to manipulate bulky objects using their arms and torso. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on robotic reinforcement learning, not LLM benchmark evaluation. | |
| Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine The paper proposes an iterative self-improvement method combining trajectory rewriting and policy refinement for terminal-based coding agents. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on agent training methods rather than evaluation benchmark design. | |
| AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning The paper introduces a multi-agent system to automate data curation and dataset editing for supervised fine-tuning. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Develops a fine-tuning agent system rather than an evaluation benchmark. | |
| HTN-Orchestrate: A Hierarchical Task Network Framework for Multi-Agent Orchestration The paper introduces a hierarchical task network framework that separates multi-agent planning from execution scheduling. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an orchestration architecture, not new LLM evaluation benchmarks. | |
| Decentralized LLM Workflows via Sequential Auctions The paper examines decentralized multi-agent coordination through sequential market auctions under conditions of imperfect competition. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on multi-agent game-theoretic workflows rather than LLM evaluation benchmarks. | |
| TRACE: Topology-Aware Reconnection via Prediction-Adaptive Connectivity Enforcement for Thin-Structure Segmentation The paper introduces an adaptive connectivity objective to improve deep learning segmentation of thin, continuous anatomical or physical structures. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates computer vision segmentation, not LLM benchmarks or evaluation. | |
| GNN-Guided Local Refinement for Efficient Layout Decomposition with DSA-MP The paper proposes a graph neural network method to speed up layout decomposition for advanced semiconductor manufacturing. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on circuit layout optimization rather than language model evaluation benchmarks. | |
| Skill-Decoupled On-Policy Expert Distillation for Unified Vision-Language Models The paper introduces a skill-decoupled on-policy distillation method to merge vision-language experts without conflicting gradients or reward hacking. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates a distillation algorithm rather than proposing a new evaluation benchmark. | |
| OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents The paper presents an end-to-end trained multimodal agent that combines visual graphical interface actions and executable code execution. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents an agent training method rather than an evaluation benchmark. | |
| Training-Free On-Policy Distillation The paper presents a closed-form, training-free approach to merge specialized fine-tuned models into a single student using forward passes. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a training-free distillation method rather than an evaluation benchmark. | |
| Disagreement-Guided Weak-to-Strong Routing The paper develops an unsupervised routing method to escalate tasks from weak language models to stronger models without ground-truth labels. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a routing methodology rather than an evaluation benchmark. | |
| Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries The paper trains language model agents to reorganize unstructured document collections into structured workspaces to speed up future multi-document queries. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a workspace curation method rather than an evaluation benchmark. | |
| LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents The paper proposes a training-free framework that uses uncertainty signals to escalate computer-use tasks from weak models to strong models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents an agent routing architecture rather than benchmark evaluation methodologies. | |
| Asking Smarter Questions: Targeted Active Preference Learning for Underspecified Rewards The paper presents a method for robots to generate targeted preference queries to learn underspecified rewards from humans. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an active preference learning algorithm rather than an evaluation benchmark. | |
| Improving Environment-Agnostic Coding Agents: The Role of Scaffolding The authors evaluate how developer scaffolding and general prompt instructions affect the performance of environment-agnostic coding agents across diverse benchmarks. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on agent scaffolding interventions rather than benchmark design. | |
| EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search The authors introduce EliteReplay, combining reinforcement learning with quality-diversity search replay to retain solution diversity during post-training. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Investigates RL optimization techniques rather than evaluation methodologies. | |
| Towards Continual Learning: Mitigating Forgetting With Subspace Tuning The authors propose Subspace Tuning, a fine-tuning method restricting weight updates to bottom singular subspaces to mitigate catastrophic forgetting in language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a tuning method rather than benchmark evaluation frameworks. | |
| Private Retrieval Augmented Generation via Random Projection The authors present a random projection method to protect sensitive datastore embeddings against extraction attacks in retrieval-augmented generation systems. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates a privacy defense rather than presenting a general benchmark. | |
| Learning to Steer LLM Agents and Multi-Agent Systems The authors propose a learned steering agent that monitors execution context and intervenes to prevent goal drift in tool-using language model systems. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on an agent steering algorithm rather than benchmark design. | |
| A team of Gemini 4 Argon agents beats Minecraft The authors report on a team of autonomous Gemini 4 Argon agents collaborating via chat and shared code to beat Minecraft survival mode. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on an empirical multi-agent demonstration rather than benchmark methodology. | |
| Learning Self-Orchestration in Decentralized Agent Swarms The authors present a market-based post-training framework that allows decentralized agent swarms to learn both action policies and coordination mechanisms without central orchestration. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a multi-agent training methodology rather than a benchmark or evaluation dataset. | |
| Twice Upon a Time: Complementary Representations for Time-Series Reasoning The authors propose combining line plots and normalized delay embeddings to provide complementary visual representations for time-series reasoning in language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a dual-view time-series representation architecture rather than an LLM benchmark. | |
| ACEdit: Stable Multi-Turn Image Editing via Asymmetric Conditioning The authors propose ACEdit, a training and distillation scheme that stabilizes image editing models against accumulating errors across multiple editing turns. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a training scheme for image editing, not an LLM benchmark. | |
| Designing Transformers for Continuous Diffusion Language Models The authors introduce LangDiT, an adapted transformer architecture designed specifically for continuous diffusion language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a model architecture rather than an evaluation framework or benchmark. | |
| KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents The authors propose KnowGPO, a reinforcement learning framework that assigns credit to deep search agents based on acquired knowledge conditions rather than trajectories. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on training policy optimization for search agents rather than evaluation benchmarks. | |
| Dexterity from Supervised Play The authors introduce DSP, a framework where robots learn from brief human play and autonomously collect contact-rich data for dexterous manipulation. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a robotic manipulation framework rather than an LLM evaluation benchmark. | |
| 3DCodeVerse: Exploring 3D Worlds as Code The authors introduce 3DCodeVerse, a dataset and multi-agent framework enabling coding agents to generate, reason over, and verify 3D executable code. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a dataset and harness for 3D code rather than LLM benchmark evaluation methodology. | |
| GeoState: Learning View-Decoupled Geometry States for 3D World Modeling The authors introduce GeoState, a view-decoupled 3D world modeling approach separating scene dynamics from camera motion. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a predictive 3D model rather than a benchmark or evaluation method for LLMs. | |
| MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering The authors propose a post-training method enabling autonomous language model agents to organize experiments and carry out machine learning engineering tasks. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a model and training method for ML engineering rather than an LLM benchmark. | |
| Rethinking Routing Exploration in Modern Mixture-of-Experts The authors propose MirrorMoE to reconcile an exploration dilemma between fine-grained expert segmentation and shared experts in mixture-of-experts models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on mixture-of-experts architectural design rather than LLM evaluation methodologies. | |
| CoDA: Co-Evolution of Data Engines and Action Policies The authors present CoDA, a framework where a simulation data engine and a robot action policy co-evolve using memory of past failures. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes policy training techniques for robots rather than LLM evaluation or benchmarks. | |
| T2-z: Test-Time Optimization of Environment Latents for Robotic World Models The authors propose T2-Z, a method optimizing learnable environment latents at test time to help robotic world models adapt to unobserved physical factors. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a robotics world model rather than LLM benchmarking or evaluation methods. | |
| Discovering state-of-the-art quantum circuits with large language models The authors introduce an evolutionary framework using large language models to design and optimize resource-efficient quantum circuits. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Deals with quantum circuit synthesis rather than evaluating or benchmarking language models. | |
| LIBERO-PeRM: Benchmarking Personalized Robotic Manipulation The authors introduce LIBERO-PeRM, a simulation benchmark for studying how embodied robotic agents learn and follow user preferences in manipulation tasks. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a benchmark for embodied robotics rather than LLMs or language-model-based systems. | |
| BenchMarker: An Education-Inspired Toolkit for Highlighting Flaws in Multiple-Choice Benchmarks - ACL Anthology Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via acl · 15 Oct, 15:07 | Waiting for check | |
| A Technical Survey of Reinforcement Learning Techniques for Large Language Models This survey provides an algorithmic overview and failure mode analysis of reinforcement learning techniques applied to language models. Peer-reviewedOlder paper (2025) Older paper (2025) — discussed this week | LLM evaluation and benchmarks via acm · 15 Oct, 15:07 | Not relevant Surveys RL algorithms rather than LLM evaluation benchmarks or methodologies. | |
| TorchSpin: Differentiable Electron Paramagnetic Resonance Simulation in PyTorch The paper presents an open-source PyTorch library providing differentiable forward simulation of electron paramagnetic resonance spectra for machine-learning workflows. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:07 | Not relevant Focuses on physics simulation in PyTorch, not LLM evaluation. | |
| Delay Complexity Dimension: Learning with Delayed and Censored Labels The paper introduces a sample complexity metric to analyze binary classification learning under randomly delayed and censored labels. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Studies theoretical PAC learning bounds, not LLM benchmark evaluation. | |
| Wong--Zakai Convergence and FNO Learning for Stochastic Volterra Equations The paper proves Wong-Zakai convergence and applies Fourier neural operators to learn solution operators for stochastic Volterra integral equations. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on solving stochastic differential equations, not LLM evaluation benchmarks. | |
| Semi-Amortized Bayesian Inference with Reusable Local Posteriors The paper proposes a semi-amortized Bayesian inference method that trains neural estimators for parameter blocks and alternates with Markov chain updates. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on Bayesian parameter inference, not LLM evaluation benchmarks. | |
| Virtual-Alloy Physics-Aware Semantic System for extrapolative prediction of perovskite solar cells The paper introduces a physics-aware machine learning framework that interpolates crystal graphs to predict performance in perovskite solar cells. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on solar cell property prediction, not LLM benchmarking. | |
| Data Density Scaling Laws for Image Self-Distillation The paper proposes a revised scaling law for image self-distillation that incorporates sample repetition alongside unique sample counts. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Studies image self-distillation scaling laws, not LLM evaluation. | |
| IR-PRM+ with Previous-Utility Prediction: Constant Regret and Best-Iterate Convergence The paper establishes constant regret and convergence guarantees for simultaneous and alternating regret matching variants using previous-utility prediction in matrix games. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Deals with zero-sum game theory algorithms, not LLM benchmarks. | |
| Discrete Diffusion Models are Minimax Optimal Under Sparsity The paper establishes minimax-optimal statistical rates for discrete diffusion models generating text and categorical data under sparsity assumptions. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Presents theoretical rates for diffusion models, not LLM benchmarks. | |
| Optimization for One, Feasibility for Most: Decoupling Optimization from Feasibility in Chance-Constrained Programs The paper proposes a framework that decouples candidate generation from feasibility assessment using structured sampling for chance-constrained stochastic programs. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on chance-constrained mathematical optimization, not LLM evaluation. | |
| How Many Ranks Does a Token Need? A Noise-Floor Squelch for Rank-One LoRA Experts The paper introduces a router-free method that adaptively thresholds rank-one LoRA adapter responses to remove cross-talk noise per token. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Presents a dynamic LoRA pruning method, not an evaluation benchmark. | |
| Lag: A Mean-Field Optimal Policy for Online Assignment with Reusable Resources The paper develops a closed-form index policy for online task assignment with reusable resources, establishing its asymptotic optimality in fluid limits. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on online resource assignment policies, not LLM evaluation. | |
| What Information Buys in Active Inference The paper compares active inference to standard exploration in reinforcement learning and proposes a square-root calibration to prevent sub-optimal actions. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on theoretical active inference regret, not LLM benchmarks. | |
| Deep Generative Models for Phylogenetic Inference with Complex Evolutionary Processes The paper studies deep generative simulation-based inference to estimate phylogenetic tree posteriors from sequences under complex evolutionary models. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on phylogenetic tree inference, not LLM benchmarks. | |
| Conformalized Physics-Informed Weak Regression for Parameter Estimation in Epidemic Models The paper presents a weak regression approach combined with conformal prediction to estimate parameters and uncertainty in epidemic models without numerical derivatives. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Studies parameter estimation for epidemic models, not LLM benchmarking. | |
| Theoretical and Practical Advances in Hierarchical Bayesian Deep Learning Methodology The paper introduces a multilevel particle marginal Metropolis-Hastings method for joint Bayesian inference of neural network parameters and prior hyperparameters. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on MCMC methodology, not LLM evaluation benchmarks. | |
| Backpropagating Verifier Rewards Through Continuous Flow Language Models The paper introduces a post-training framework using intermediate token critics to backpropagate verifier rewards through continuous flow language models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces an RL framework for continuous flow models, not benchmarks. | |
| Contextual Credit Policy Optimization The paper presents a method that weights trajectory peers by context similarity to improve step-level credit assignment for language model agents. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes an RL credit assignment method, not a benchmark. | |
| Neural Supervised Clustering for Learning Individualized Treatment Rules with Many Treatments The paper introduces a neural Q-learning framework that clusters treatments by response similarity to improve individualized treatment rules across many options. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on biostatistical treatment rule estimation, not LLM evaluation. |
1,021–1,080 of 3,345← PreviousNext →