Every paper checked — nothing hidden

All papers

Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.

PaperTopicRelevanceFeedback
When is a better LLM worth paying for? Stochastic Optimization with Switching Mixed-Fidelity Oracles
Investigates cost-aware policies for adaptively switching between language models of varying fidelity in stochastic optimization pipelines.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Focuses on optimization theory rather than LLM benchmarking or evaluation frameworks.
CoSTA: Compressed Speech Token Adaptation for Efficient Spoken Dialogue Models
Presents a framework that decouples speech generation and synthesis token spaces to speed up spoken dialogue model inference.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Tests speech dialogue models rather than proposing new general LLM evaluation standards.
All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents
Introduces a reinforcement learning framework that evaluates memory operations in agents using a shared state and process rewards.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Evaluates a memory agent policy rather than proposing new general LLM evaluation benchmarks.
CARE-SD: Explainable Reasoning Architecture for Local SLM Self-Disclosure Analysis in Textual Data
Develops a structured explainable reasoning architecture to extract sensitive self-disclosures from cybersecurity interviews using local small language models.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Evaluates a specific text analysis task rather than broader LLM evaluation methodologies.
GRADS: Graph-Regulated Agentic Data Synthesis with Controllable Difficulty
Proposes a graph-guided synthetic data generation framework with controlled difficulty levels for training multi-hop research agents.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Designs a data synthesis pipeline rather than establishing LLM evaluation methodologies or benchmarks.
MemScope: Evolving Agent Memory Architectures through Localized Code Repair
Introduces a localized code repair framework that evolves language model memory architectures using fine-grained execution feedback.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Uses benchmarks to test an agent memory architecture rather than benchmarking LLMs themselves.
Language Models Share a Readout Bottleneck in Working Memory
Analyzes working memory error patterns in language models to show that failures stem from readout interference rather than target retrieval.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Analyzes cognitive working-memory mechanics rather than evaluating general LLM benchmarks.
Mine, Sample, Render: Test-Time Diversity Alignment Of Generative Models
Presents a test-time method that extracts structured schemas from corpora to guide generative models toward more diverse outputs.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Focuses on diversity sampling rather than benchmarking LLMs.
FedSumUp: User-Auditable Privacy-Aware Representation Sharing for Resource-Constrained Federated Intelligence
Proposes a lossy compression framework for federated learning that balances user privacy against model utility on resource-constrained devices.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Presents a federated learning framework, not an evaluation benchmark.
Condition-Contrastive Extrapolation for On-Policy Distillation of Language Models
Introduces a condition-contrastive distillation approach to prevent student language models from learning erroneous teacher continuations.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Evaluates reasoning distillation rather than proposing evaluation frameworks.
Who May Release an Agent's Promise? Standing over Commitments in Multi-Party Chat
Studies whether conversational agents correctly enforce authorization rules when users attempt to cancel prior agent commitments to others.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Studies commitment tracking rather than general LLM benchmarking.
AutoOnboarder: Autonomously Onboarding LLM Agents
Presents an autonomous supervisor agent that iteratively tunes an agent's code, harness, and prompt on few examples.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Presents an agent optimization framework, not an evaluation standard.
RecSoup: Efficient Model Merging Approach for Controllable Multi-Objective Recommendation
Introduces a two-stage model merging method to enable dynamic multi-objective recommendation trade-offs at inference time without retraining.
Peer-reviewed
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Proposes a model merging technique rather than an evaluation benchmark.
Representation Redistribution in Sparse Autoencoders
Analyzes how prompt variations redistribute internal feature activations in sparse autoencoders and traces their impact on downstream model decisions.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Focuses on feature sensitivity analysis, not LLM benchmarking.
A Responsibility-Centered Coordination Mechanism for Decentralized Multi-Agent Systems
Proposes a decentralized coordination mechanism using commitment tracking to preserve clear accountability across multi-agent delegation chains.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Presents a coordination framework, not a general evaluation benchmark.
Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference
Proposes a fractional power-of-two quantization grid to enable efficient multiplier-free integer inference for large language models.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Evaluates existing benchmarks rather than proposing evaluation methodology.
Behavioral Compositionality of Sparse Autoencoder Representations
Introduces a metric to measure how non-linearly features learned by sparse autoencoders interact during downstream model behaviors.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Studies interpretability metrics rather than LLM benchmarking standards.
TRACER: A Multimodal Harness for Patient Care in Hospitals
Introduces a multimodal harness and benchmark pairing continuous video with clinical records to assist patient monitoring in hospitals.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Evaluates its specific system rather than proposing a general LLM benchmark.
TRACER: Trajectory-Aware Tool Retrieval for LLM Agents
Proposes a trajectory-aware retrieval method that indexes tool sequences in a trie to cut token costs and improve selection accuracy.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Presents a tool selection method, not an evaluation benchmark.
Diviner: Agentic RCA with Human-Governed Adaptation for Large-Scale Infrastructure
Presents an agentic system with human-governed adaptation to automate root-cause analysis across complex, large-scale infrastructure environments.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Focuses on infrastructure incident RCA agents, not benchmark methodology.
Mallea: Building, Steering, and Memorizing Worlds with Coding Agent
The paper presents an agent system that generates, edits, and renders interactive virtual worlds from text using executable code rules.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Presents a world simulation system, not benchmark methodology.
The Density of Thought: Predicting and Steering Reasoning in Large Language Models
The paper explores whether measuring information density across reasoning steps can predict and guide answer correctness in large language models.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Studies chain-of-thought density, not benchmark evaluation design.
Bias Control and Injection for Multi-Agent Aggregation
The paper proposes a test-time aggregation method using bias control and injection to resolve disagreements among reasoning language model agents.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Proposes an aggregation technique, not benchmark evaluation methodology.
Money-Mule Detection as a Relational Learning Problem: A Study on the RBIH NFPC Dataset
The paper applies heterogeneous graph neural networks directly to bank relational schemas to detect money-mule accounts prior to flag dates.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Focuses on tabular/graph fraud detection, not LLM evaluation.
Loud and Clear: Making Visual Evidence Count in Multimodal Document RAG
The paper examines how increasing visual token budgets and cropping key evidence improves document retrieval-augmented generation in vision-language models.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Studies generator visual fidelity, not benchmark evaluation methodology.
Constrained Multi-Agent Reinforcement Learning in Imagination
The paper evaluates a constrained multi-agent reinforcement learning method that learns policies inside world models to reduce environment interaction costs.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Evaluates classic MARL benchmarks, not LLM evaluation.
DocPilot: Interactive Agent Orchestration and Training for Long Visual Document Understanding
The paper presents an agent orchestration framework that trains smaller language models to navigate, inspect, and reason over long multimodal documents.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Presents an agent architecture and training, not evaluation benchmarking.
Does On-Policy Distillation Need to Complete Every Rollout?
The paper introduces a method that accelerates on-policy knowledge distillation by truncating student rollouts and applying inverse-probability weighting.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Proposes an optimization algorithm, not LLM evaluation benchmarking.
Quantum Versus Classical Machine Learning for Neuropsychiatric Disorders: A Feature-Matched Multi-Cohort Evaluation
The paper compares quantum and classical machine learning models on multi-cohort fMRI datasets to test their utility for neuropsychiatric classification.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Focuses on medical ML classification comparisons, not LLM benchmarks.
FedSkillGit: Federated Skill Evolution via Version Control
The paper proposes a federated learning framework that uses version control operations instead of full document aggregation to evolve agent skills.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Focuses on federated skill learning, not benchmark evaluation.
Hierarchical Graph Convolutional Network Approach for Detecting Low-Quality Documents
The paper introduces a hierarchical graph convolutional network and dataset to detect internal inconsistencies and headline mismatches in news documents.
Peer-reviewed
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Introduces a text classification dataset, not LLM evaluation methods.
LTM: Latent Transition Based In-Context Memory for Vision-Language-Action Models
The paper introduces an in-context memory method storing latent transitions between visual observations to assist robot manipulation without extra annotation.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Proposes a memory architecture, not benchmark evaluation methodology.
Memorilla: Latent Semantic Memory for LLMs
The paper introduces a system that retrieves document information into a fixed set of prompt embeddings for frozen language models.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Evaluates on benchmarks, but does not contribute novel benchmark methodology.
PriorityFEC: Deadline-Aware Streaming FEC for KV-Cache Handoff in Edge Multi-Agent Systems
The paper proposes an error-correction method to prioritize and transmit key-value cache tokens across edge-deployed multi-agent systems under latency deadlines.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Focuses on edge networking efficiency, not evaluation benchmarks.
Learning to Reason with Selective Internal Feedback
The paper introduces a reinforcement learning method that uses internal certainty profiles and selective filtering to train models without external verifiers.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Introduces a training algorithm, not evaluation benchmarks.
Self-Guided LLM Reasoning for Error Diagnosis in Vehicle Routing Problems
The paper presents a self-guided reasoning framework where language models generate code to detect and diagnose errors in vehicle routing problem instances.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Investigates reasoning for VRP error detection, not benchmark methodology.
Matoi: Selective Revalidation of Consequence Evidence for Agent Authorization
The paper introduces an authorization system that reuses validated consequence evidence across agent execution branches while maintaining separate live authority.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Focuses on authorization architecture, not LLM benchmark evaluation.
When do quantum views help? Complementary classical-quantum representations in multi-view fusion
The paper studies combining classical and quantum representations to determine when multi-view fusion improves machine learning predictions over single representations.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Studies quantum feature fusion algorithms, not LLM benchmarks or evaluation methods.
Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense
The paper analyzes multi-reward reinforcement learning failures where auxiliary objectives degrade primary performance and introduces a trust-anchored gating method.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Focuses on training algorithms rather than empirical benchmark development or LLM-as-a-judge.
Jev-LiteMAS: Separating Decisions from Generation in Multi-Agent Workflows
The paper proposes a framework that replaces generative model calls with lightweight decision models for routing and reviewing in multi-agent workflows.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Focuses on multi-agent workflow architecture rather than evaluating general LLM benchmarks.
Where, How, and When: Query-Guided Grounding for Video Understanding
Presents methods for query-guided video grounding across object localization, relationship detection, and temporal moment identification.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Presents visual grounding architectures, not LLM benchmark design or evaluation analysis.
Do Agents Need a New Kind of Paper? SPRA: Structured Paper Representation for Agents
Creates a structured schema that converts research papers into connected scientific entities to improve retrieval-augmented generation for autonomous agents.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Proposes an information representation framework rather than general LLM evaluation benchmarks.
Learning from Interaction Experience via Localized Context Distillation
Introduces a distillation method that identifies critical sub-spans in student agent trajectories for teacher revision and continual learning.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Focuses on post-training distillation methodology rather than LLM benchmarks or evaluation.
V-SHARE: Provable Convergence Guarantees for Advantage-based Algorithms
Provides theoretical convergence guarantees and finite-sample bounds for advantage-based value sharing in reinforcement learning algorithms.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Focuses on theoretical RL convergence analysis, not LLM benchmark evaluation or judge reliability.
Incentivizing Tool-Use Reinforcement Learning for Post-Arrival Emergency Care Prediction
Integrates reinforcement learning and tabular model guidance to help language models select features and predict emergency department clinical risk.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Evaluates a predictive clinical agent, not developing novel LLM benchmarks or evaluation methods.
X-Fly: Learning to Reach and Stop in Full-Chunk Aerial Vision-Language Navigation
Proposes a training framework enabling aerial vision-language agents to navigate in full action chunks and stop reliably at destinations.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Proposes an aerial navigation policy rather than general benchmark evaluation methodologies.
Full-Duplex Interaction with Speech, Expressions and Gestures
Presents a full-duplex conversational model that generates real-time speech alongside synchronized facial expressions and body gestures.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Introduces a model pipeline and specialized metrics, not general LLM evaluation or benchmark critique.
Can Visual Generation Expand What Unified Multimodal Models Know?
Investigates whether training multimodal models on visual generation tasks transfers accessible factual knowledge into text question answering.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Uses probing to analyze representation rather than benchmarking general LLM capability or trustworthiness.
Trajectory-Level Speculation with Semantic Verification for Tool-Using Agents
Introduces a framework where a small model drafts tool-use trajectories and a larger model verifies them in single prefill passes.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Introduces an agent framework and evaluates on existing benchmarks, not novel benchmark design.
Ontology Refinement with Embeddings and LLMs: A Case Study on CASTON
Combines domain embeddings with language models to detect and repair incorrect subsumption axioms in industrial knowledge ontologies.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Case study on ontology repair, not a benchmark or evaluation method for LLMs.
Tool-Augmented Large Language Models for Reliable Medical Calculation from Patient Notes
Uses tool-augmented language models to extract clinical variables from patient notes and reliably execute deterministic medical calculation formulas.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Uses an existing benchmark (MedCalc-Bench) rather than contributing novel LLM evaluation methodology.
AI-Driven Disaster Response and Displacement Monitoring
Surveys the application of artificial intelligence methods to humanitarian disaster response and displacement monitoring.
Peer-reviewed
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Survey overview, not introducing or analyzing LLM benchmarks or evaluation methods.
FlashAttention with Statistics: Low-Overhead Attention Features for Compact Hallucination Detectors
Extracts attention statistics directly during fast attention computation to build lightweight hallucination detectors for language models.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Proposes a detection mechanism rather than evaluating or benchmarking LLM evaluation methods.
MetaExp: Learning to Self-Improve through Experience via Verifiable Cross-Task Environment Scaling
Presents a training framework that teaches language model agents to transfer reusable problem-solving experience across structured task families.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Introduces a framework for agent learning, not a benchmark.
ISW-Agent: Integrating LLM-Driven Dialogue with n8n Workflow for Internal Solitary Wave Remote Sensing Image Processing
Combines language model dialogue with workflow automation to process satellite imagery for internal solitary wave detection and forecasting.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Describes a specific remote sensing pipeline, not general benchmark methodology.
Decomposing Brain-LLM Alignment with Interpretable Semantic Representations
Decomposes the alignment between language model activations and human brain scans using neurocognitively grounded semantic property representations.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Examines cognitive neuroscience alignment rather than LLM benchmarking standards.
QREF: Improving DeepSearch Agents by Aligning Search with Reasoning via Query Refinement and Evidence Filtering
Improves deep search agents by refining search queries and filtering evidence to align retrieval with multi-step reasoning goals.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Proposes a retrieval agent method rather than benchmark evaluation.
SCRIBE: Scalable Contextual Representations of Interpretable Brain-Grounded Semantics
Introduces a framework using language models to construct context-sensitive semantic representations that align with human brain activity during language comprehension.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Evaluates brain-prediction fidelity rather than LLM benchmarking methods.
Hierarchical Interpretable Multitask Audio Representation Learning: An Analysis of Event-Scene Relationships through the Information Bottleneck
Studies a hierarchical conditioning architecture that predicts acoustic scenes directly from intermediate audio event probabilities.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Investigates acoustic scene learning, not LLM evaluation frameworks.
CriticaLLM: Using LLMs as Autonomous Game Master for Tabletop Role-Playing Games
Presents an automated framework using language models to manage storytelling, rules, and combat in tabletop role-playing games.
Preprint
LLM evaluation and benchmarks
via arxiv · 17 Oct, 06:17
Not relevant
Demonstrates a game application rather than standard benchmark methodology.
901–960 of 3,345← PreviousNext →