Every paper checked — nothing hidden
All papers
Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.
All3,345In brief3More (on-topic)587Not relevant2,721Not assessed24Older45Could not confirm25Blogs with no paper54All blogs57
| Paper | Topic | Relevance | Feedback |
|---|---|---|---|
| When is a better LLM worth paying for? Stochastic Optimization with Switching Mixed-Fidelity Oracles Investigates cost-aware policies for adaptively switching between language models of varying fidelity in stochastic optimization pipelines. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on optimization theory rather than LLM benchmarking or evaluation frameworks. | |
| CoSTA: Compressed Speech Token Adaptation for Efficient Spoken Dialogue Models Presents a framework that decouples speech generation and synthesis token spaces to speed up spoken dialogue model inference. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Tests speech dialogue models rather than proposing new general LLM evaluation standards. | |
| All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents Introduces a reinforcement learning framework that evaluates memory operations in agents using a shared state and process rewards. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates a memory agent policy rather than proposing new general LLM evaluation benchmarks. | |
| CARE-SD: Explainable Reasoning Architecture for Local SLM Self-Disclosure Analysis in Textual Data Develops a structured explainable reasoning architecture to extract sensitive self-disclosures from cybersecurity interviews using local small language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates a specific text analysis task rather than broader LLM evaluation methodologies. | |
| GRADS: Graph-Regulated Agentic Data Synthesis with Controllable Difficulty Proposes a graph-guided synthetic data generation framework with controlled difficulty levels for training multi-hop research agents. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Designs a data synthesis pipeline rather than establishing LLM evaluation methodologies or benchmarks. | |
| MemScope: Evolving Agent Memory Architectures through Localized Code Repair Introduces a localized code repair framework that evolves language model memory architectures using fine-grained execution feedback. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Uses benchmarks to test an agent memory architecture rather than benchmarking LLMs themselves. | |
| Language Models Share a Readout Bottleneck in Working Memory Analyzes working memory error patterns in language models to show that failures stem from readout interference rather than target retrieval. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Analyzes cognitive working-memory mechanics rather than evaluating general LLM benchmarks. | |
| Mine, Sample, Render: Test-Time Diversity Alignment Of Generative Models Presents a test-time method that extracts structured schemas from corpora to guide generative models toward more diverse outputs. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on diversity sampling rather than benchmarking LLMs. | |
| FedSumUp: User-Auditable Privacy-Aware Representation Sharing for Resource-Constrained Federated Intelligence Proposes a lossy compression framework for federated learning that balances user privacy against model utility on resource-constrained devices. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a federated learning framework, not an evaluation benchmark. | |
| Condition-Contrastive Extrapolation for On-Policy Distillation of Language Models Introduces a condition-contrastive distillation approach to prevent student language models from learning erroneous teacher continuations. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates reasoning distillation rather than proposing evaluation frameworks. | |
| Who May Release an Agent's Promise? Standing over Commitments in Multi-Party Chat Studies whether conversational agents correctly enforce authorization rules when users attempt to cancel prior agent commitments to others. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies commitment tracking rather than general LLM benchmarking. | |
| AutoOnboarder: Autonomously Onboarding LLM Agents Presents an autonomous supervisor agent that iteratively tunes an agent's code, harness, and prompt on few examples. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents an agent optimization framework, not an evaluation standard. | |
| RecSoup: Efficient Model Merging Approach for Controllable Multi-Objective Recommendation Introduces a two-stage model merging method to enable dynamic multi-objective recommendation trade-offs at inference time without retraining. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a model merging technique rather than an evaluation benchmark. | |
| Representation Redistribution in Sparse Autoencoders Analyzes how prompt variations redistribute internal feature activations in sparse autoencoders and traces their impact on downstream model decisions. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on feature sensitivity analysis, not LLM benchmarking. | |
| A Responsibility-Centered Coordination Mechanism for Decentralized Multi-Agent Systems Proposes a decentralized coordination mechanism using commitment tracking to preserve clear accountability across multi-agent delegation chains. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a coordination framework, not a general evaluation benchmark. | |
| Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference Proposes a fractional power-of-two quantization grid to enable efficient multiplier-free integer inference for large language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates existing benchmarks rather than proposing evaluation methodology. | |
| Behavioral Compositionality of Sparse Autoencoder Representations Introduces a metric to measure how non-linearly features learned by sparse autoencoders interact during downstream model behaviors. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies interpretability metrics rather than LLM benchmarking standards. | |
| TRACER: A Multimodal Harness for Patient Care in Hospitals Introduces a multimodal harness and benchmark pairing continuous video with clinical records to assist patient monitoring in hospitals. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates its specific system rather than proposing a general LLM benchmark. | |
| TRACER: Trajectory-Aware Tool Retrieval for LLM Agents Proposes a trajectory-aware retrieval method that indexes tool sequences in a trie to cut token costs and improve selection accuracy. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a tool selection method, not an evaluation benchmark. | |
| Diviner: Agentic RCA with Human-Governed Adaptation for Large-Scale Infrastructure Presents an agentic system with human-governed adaptation to automate root-cause analysis across complex, large-scale infrastructure environments. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on infrastructure incident RCA agents, not benchmark methodology. | |
| Mallea: Building, Steering, and Memorizing Worlds with Coding Agent The paper presents an agent system that generates, edits, and renders interactive virtual worlds from text using executable code rules. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a world simulation system, not benchmark methodology. | |
| The Density of Thought: Predicting and Steering Reasoning in Large Language Models The paper explores whether measuring information density across reasoning steps can predict and guide answer correctness in large language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies chain-of-thought density, not benchmark evaluation design. | |
| Bias Control and Injection for Multi-Agent Aggregation The paper proposes a test-time aggregation method using bias control and injection to resolve disagreements among reasoning language model agents. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an aggregation technique, not benchmark evaluation methodology. | |
| Money-Mule Detection as a Relational Learning Problem: A Study on the RBIH NFPC Dataset The paper applies heterogeneous graph neural networks directly to bank relational schemas to detect money-mule accounts prior to flag dates. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on tabular/graph fraud detection, not LLM evaluation. | |
| Loud and Clear: Making Visual Evidence Count in Multimodal Document RAG The paper examines how increasing visual token budgets and cropping key evidence improves document retrieval-augmented generation in vision-language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies generator visual fidelity, not benchmark evaluation methodology. | |
| Constrained Multi-Agent Reinforcement Learning in Imagination The paper evaluates a constrained multi-agent reinforcement learning method that learns policies inside world models to reduce environment interaction costs. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates classic MARL benchmarks, not LLM evaluation. | |
| DocPilot: Interactive Agent Orchestration and Training for Long Visual Document Understanding The paper presents an agent orchestration framework that trains smaller language models to navigate, inspect, and reason over long multimodal documents. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents an agent architecture and training, not evaluation benchmarking. | |
| Does On-Policy Distillation Need to Complete Every Rollout? The paper introduces a method that accelerates on-policy knowledge distillation by truncating student rollouts and applying inverse-probability weighting. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an optimization algorithm, not LLM evaluation benchmarking. | |
| Quantum Versus Classical Machine Learning for Neuropsychiatric Disorders: A Feature-Matched Multi-Cohort Evaluation The paper compares quantum and classical machine learning models on multi-cohort fMRI datasets to test their utility for neuropsychiatric classification. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on medical ML classification comparisons, not LLM benchmarks. | |
| FedSkillGit: Federated Skill Evolution via Version Control The paper proposes a federated learning framework that uses version control operations instead of full document aggregation to evolve agent skills. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on federated skill learning, not benchmark evaluation. | |
| Hierarchical Graph Convolutional Network Approach for Detecting Low-Quality Documents The paper introduces a hierarchical graph convolutional network and dataset to detect internal inconsistencies and headline mismatches in news documents. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a text classification dataset, not LLM evaluation methods. | |
| LTM: Latent Transition Based In-Context Memory for Vision-Language-Action Models The paper introduces an in-context memory method storing latent transitions between visual observations to assist robot manipulation without extra annotation. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a memory architecture, not benchmark evaluation methodology. | |
| Memorilla: Latent Semantic Memory for LLMs The paper introduces a system that retrieves document information into a fixed set of prompt embeddings for frozen language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates on benchmarks, but does not contribute novel benchmark methodology. | |
| PriorityFEC: Deadline-Aware Streaming FEC for KV-Cache Handoff in Edge Multi-Agent Systems The paper proposes an error-correction method to prioritize and transmit key-value cache tokens across edge-deployed multi-agent systems under latency deadlines. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on edge networking efficiency, not evaluation benchmarks. | |
| Learning to Reason with Selective Internal Feedback The paper introduces a reinforcement learning method that uses internal certainty profiles and selective filtering to train models without external verifiers. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a training algorithm, not evaluation benchmarks. | |
| Self-Guided LLM Reasoning for Error Diagnosis in Vehicle Routing Problems The paper presents a self-guided reasoning framework where language models generate code to detect and diagnose errors in vehicle routing problem instances. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Investigates reasoning for VRP error detection, not benchmark methodology. | |
| Matoi: Selective Revalidation of Consequence Evidence for Agent Authorization The paper introduces an authorization system that reuses validated consequence evidence across agent execution branches while maintaining separate live authority. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on authorization architecture, not LLM benchmark evaluation. | |
| When do quantum views help? Complementary classical-quantum representations in multi-view fusion The paper studies combining classical and quantum representations to determine when multi-view fusion improves machine learning predictions over single representations. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies quantum feature fusion algorithms, not LLM benchmarks or evaluation methods. | |
| Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense The paper analyzes multi-reward reinforcement learning failures where auxiliary objectives degrade primary performance and introduces a trust-anchored gating method. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on training algorithms rather than empirical benchmark development or LLM-as-a-judge. | |
| Jev-LiteMAS: Separating Decisions from Generation in Multi-Agent Workflows The paper proposes a framework that replaces generative model calls with lightweight decision models for routing and reviewing in multi-agent workflows. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on multi-agent workflow architecture rather than evaluating general LLM benchmarks. | |
| Where, How, and When: Query-Guided Grounding for Video Understanding Presents methods for query-guided video grounding across object localization, relationship detection, and temporal moment identification. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents visual grounding architectures, not LLM benchmark design or evaluation analysis. | |
| Do Agents Need a New Kind of Paper? SPRA: Structured Paper Representation for Agents Creates a structured schema that converts research papers into connected scientific entities to improve retrieval-augmented generation for autonomous agents. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an information representation framework rather than general LLM evaluation benchmarks. | |
| Learning from Interaction Experience via Localized Context Distillation Introduces a distillation method that identifies critical sub-spans in student agent trajectories for teacher revision and continual learning. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on post-training distillation methodology rather than LLM benchmarks or evaluation. | |
| V-SHARE: Provable Convergence Guarantees for Advantage-based Algorithms Provides theoretical convergence guarantees and finite-sample bounds for advantage-based value sharing in reinforcement learning algorithms. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on theoretical RL convergence analysis, not LLM benchmark evaluation or judge reliability. | |
| Incentivizing Tool-Use Reinforcement Learning for Post-Arrival Emergency Care Prediction Integrates reinforcement learning and tabular model guidance to help language models select features and predict emergency department clinical risk. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates a predictive clinical agent, not developing novel LLM benchmarks or evaluation methods. | |
| X-Fly: Learning to Reach and Stop in Full-Chunk Aerial Vision-Language Navigation Proposes a training framework enabling aerial vision-language agents to navigate in full action chunks and stop reliably at destinations. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an aerial navigation policy rather than general benchmark evaluation methodologies. | |
| Full-Duplex Interaction with Speech, Expressions and Gestures Presents a full-duplex conversational model that generates real-time speech alongside synchronized facial expressions and body gestures. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a model pipeline and specialized metrics, not general LLM evaluation or benchmark critique. | |
| Can Visual Generation Expand What Unified Multimodal Models Know? Investigates whether training multimodal models on visual generation tasks transfers accessible factual knowledge into text question answering. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Uses probing to analyze representation rather than benchmarking general LLM capability or trustworthiness. | |
| Trajectory-Level Speculation with Semantic Verification for Tool-Using Agents Introduces a framework where a small model drafts tool-use trajectories and a larger model verifies them in single prefill passes. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces an agent framework and evaluates on existing benchmarks, not novel benchmark design. | |
| Ontology Refinement with Embeddings and LLMs: A Case Study on CASTON Combines domain embeddings with language models to detect and repair incorrect subsumption axioms in industrial knowledge ontologies. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Case study on ontology repair, not a benchmark or evaluation method for LLMs. | |
| Tool-Augmented Large Language Models for Reliable Medical Calculation from Patient Notes Uses tool-augmented language models to extract clinical variables from patient notes and reliably execute deterministic medical calculation formulas. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Uses an existing benchmark (MedCalc-Bench) rather than contributing novel LLM evaluation methodology. | |
| AI-Driven Disaster Response and Displacement Monitoring Surveys the application of artificial intelligence methods to humanitarian disaster response and displacement monitoring. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Survey overview, not introducing or analyzing LLM benchmarks or evaluation methods. | |
| FlashAttention with Statistics: Low-Overhead Attention Features for Compact Hallucination Detectors Extracts attention statistics directly during fast attention computation to build lightweight hallucination detectors for language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a detection mechanism rather than evaluating or benchmarking LLM evaluation methods. | |
| MetaExp: Learning to Self-Improve through Experience via Verifiable Cross-Task Environment Scaling Presents a training framework that teaches language model agents to transfer reusable problem-solving experience across structured task families. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a framework for agent learning, not a benchmark. | |
| ISW-Agent: Integrating LLM-Driven Dialogue with n8n Workflow for Internal Solitary Wave Remote Sensing Image Processing Combines language model dialogue with workflow automation to process satellite imagery for internal solitary wave detection and forecasting. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Describes a specific remote sensing pipeline, not general benchmark methodology. | |
| Decomposing Brain-LLM Alignment with Interpretable Semantic Representations Decomposes the alignment between language model activations and human brain scans using neurocognitively grounded semantic property representations. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Examines cognitive neuroscience alignment rather than LLM benchmarking standards. | |
| QREF: Improving DeepSearch Agents by Aligning Search with Reasoning via Query Refinement and Evidence Filtering Improves deep search agents by refining search queries and filtering evidence to align retrieval with multi-step reasoning goals. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a retrieval agent method rather than benchmark evaluation. | |
| SCRIBE: Scalable Contextual Representations of Interpretable Brain-Grounded Semantics Introduces a framework using language models to construct context-sensitive semantic representations that align with human brain activity during language comprehension. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates brain-prediction fidelity rather than LLM benchmarking methods. | |
| Hierarchical Interpretable Multitask Audio Representation Learning: An Analysis of Event-Scene Relationships through the Information Bottleneck Studies a hierarchical conditioning architecture that predicts acoustic scenes directly from intermediate audio event probabilities. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Investigates acoustic scene learning, not LLM evaluation frameworks. | |
| CriticaLLM: Using LLMs as Autonomous Game Master for Tabletop Role-Playing Games Presents an automated framework using language models to manage storytelling, rules, and combat in tabletop role-playing games. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Demonstrates a game application rather than standard benchmark methodology. |
901–960 of 3,345← PreviousNext →