Every paper checked — nothing hidden
All papers
Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.
All3,345In brief3More (on-topic)587Not relevant2,721Not assessed24Older45Could not confirm25Blogs with no paper54All blogs57
| Paper | Topic | Relevance | Feedback |
|---|---|---|---|
| REDaCT: Collapse-Free Unlearning in Large Reasoning Models Proposes an inference-time unlearning method that detects leaks and injects corrective embeddings into large reasoning model traces. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents an unlearning method rather than a general LLM evaluation benchmark. | |
| Representation Alignment from a Few Paired Data via Distilled Optimal Transport Aligns independently pretrained unimodal encoders using a few paired anchors and distilled entropic optimal transport. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an alignment technique using optimal transport, not an LLM benchmark or evaluation study. | |
| EvoBelief: Falsifiable World Models for Self-Improving LLM Agents Presents a framework allowing language model agents to evolve environment dynamics beliefs through falsifiable hypothesis testing. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Develops an agent world-modeling framework rather than an LLM evaluation benchmark. | |
| Meta-Reflective Evolution Effectively Improves Production AI Systems Introduces an evolutionary optimization method using multi-turn reflective mutators to improve production data agents and pipelines. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents an optimization framework evaluated on benchmarks, not a novel benchmark or evaluation method. | |
| Efficient Audio-Visual Speech Enhancement via Reinforcement Learning-Based Neural Architecture Search Uses reinforcement learning-based neural architecture search to discover compact models for audio-visual speech enhancement. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on speech enhancement architecture search rather than LLM benchmarking. | |
| VEPA: Verified Evolving Formal Policy for Lifelong LLM Agent Guardrails Develops a framework for evolving deterministic formal policies to serve as runtime guardrails for large language model agents. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on runtime safety enforcement rather than LLM benchmarking or evaluation frameworks. | |
| LeWAM: End-to-End World and Action Modeling with JEPAs Presents an end-to-end joint-embedding predictive architecture that combines world modeling and action flow matching for robot planning and control. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Develops a predictive architecture for control rather than a standard LLM evaluation benchmark. | |
| Reward Hacking as a Transmissible Strategy in Multi-Agent Coding Systems Demonstrates that reward-hacking strategies can propagate across multiple agents through shared memory and code in multi-agent coding systems. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Studies multi-agent behavioral failure dynamics rather than general LLM benchmark design. | |
| COFFEQA: Counterfactual-Guided Few-Shot Distillation for LLM Question Answering Introduces a counterfactual-guided few-shot distillation framework to train smaller question-answering language models using limited training samples. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a distillation training method rather than an evaluation benchmark or testing methodology. | |
| Neural networks are more modular than single neurons suggest Proposes a hierarchy of modularity to identify functional modular organization in neural networks lacking single-neuron specialization. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Studies internal representation structures rather than standard LLM evaluation methods or benchmarks. | |
| Towards Defining, Measuring, and Leveraging Creativity in AI Applications Proposes formal mathematical postulates and definitions to measure and operationalize computational creativity across domains. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Formulates a conceptual metric for creativity in chess and art, not LLM evaluation benchmarks. | |
| When Generative Models Act as Simulators Investigates mappings between generative model latent states and simulator states to enable controllable environment simulation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Studies world representation mechanics rather than benchmarking or evaluating language models. | |
| Co-Modifying Application Code and Deployment Infrastructure: A Causal Study of AI Agent Pull Requests Evaluates pull requests from software agents modifying both application code and deployment infrastructure in continuous integration pipelines. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Investigates real-world PR outcomes rather than proposing or testing LLM benchmark methodologies. | |
| Mitigating Capability Degradation in LLM Self-Improvement via Representation Geometry Regularization Introduces representation geometry regularization to prevent capability degradation during on-policy self-distillation in language models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a training regularization technique, not an LLM evaluation framework or benchmark study. | |
| Localized Generative Design of Ship Hulls with Diffusion Models and High-Fidelity Assessment Applies a reward-directed diffusion model to optimize localized ship hull geometry evaluated by automated fluid dynamics simulations. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Covers CFD evaluation for ship hulls, not LLM evaluation benchmarks or methodologies. | |
| Collaborative Theory of Mind: Learning to Self-Revise by Anticipating User Responses Trains language models to anticipate user replies and self-revise drafts before responding in multi-turn interactions. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a training framework for collaborative revision rather than a benchmark. | |
| One-Step Text Generation by Seq-Drifting Proposes a sequence-drifting framework to generate full text sequences in a single forward pass from Gaussian noise. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces a text generation framework rather than an evaluation benchmark or judge system. | |
| OGPO: Offline Goal-conditioned Policy Optimization for Recoverable Vision-Language-Action Models Introduces an offline goal-conditioned policy optimization method to help vision-language-action models recover from execution deviations. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents policy optimization for robotics rather than LLM evaluation or benchmark design. | |
| DeFiRank: Federated Pairwise Ranking for Governance in Decentralized Financial Networks Proposes a federated pairwise ranking framework for institutions to evaluate decentralized finance participants without sharing transaction data. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on a federated ranking model for financial transactions rather than LLM benchmarks. | |
| AI-Mediation in Retail Markets: Revenue Effects and Assortment Optimization Studies the impact of AI shopping assistant mediation on retail product assortment planning and business revenue. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Analyzes retail assortment optimization rather than benchmarking or evaluating LLMs. | |
| Teaching neural decoders with language Uses natural language class descriptions as supervision to train neural decoders for brain-computer interfaces across multiple modalities. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a supervised decoding framework for neural signals, not LLM benchmarking. | |
| Collaborative Learning and Reasoning over Model Slices Develops a collaborative learning framework that splits large language models into smaller heterogeneous slices to mitigate error propagation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on a multi-model reasoning framework rather than benchmarking or evaluation metrics. | |
| Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Training Presents an uncertainty-guided framework to compress reasoning supervision trajectories and reduce reasoning post-training costs. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces a post-training compression framework rather than a testing or benchmark methodology. | |
| Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding Proposes matrix-entropy and kernel alignment criteria to select optimal layer subsets of hidden states for speculative decoding. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Analyzes internal layer information for inference speedup rather than model benchmarking. | |
| JoyInside-VLM-Agent: Product-Conditioned Multimodal Policy Learning for Embodied Agents Introduces a multimodal embodied agent policy learning framework using trajectory construction, distillation, and token-level evidence routing. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents policy learning and distillation rather than new benchmarks or evaluation methodologies. | |
| MotionCoach: Constructing Multimodal Guidance for Video Professional Motion Editing Proposes a multimodal agent providing grounded prompts and video clips to guide video editing models in professional sports motions. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces VPME for evaluation, but core contribution is a video motion editing method. | |
| Beyond Semantic IDs: Finite-Beam Reachability and Continuous-Query Generation in Generative Recommendation Proposes continuous-query generation paradigms to eliminate reachability constraints caused by finite-beam decoding over discrete semantic IDs in recommendation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents a continuous-query recommendation architecture rather than LLM evaluation benchmarks. | |
| The Evidence Anchoring Boundary: Where Visual Interventions Matter and When They Help in Multimodal LLMs Analyzes layer-wise visual interventions in multimodal language models to identify where visual edits effectively reduce hallucinations. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Investigates intervention mechanics and hallucination reduction rather than proposing a benchmark. | |
| Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill Induction Introduces a framework and multimodal dataset of human desktop demonstrations to induce reusable skills for computer-use agents. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Primary focus is skill induction for agents, though it includes an evaluation dataset. | |
| What Should SFT Learn? Conditional Information Gain for Generalizable Post-Training Introduces an information-theoretic framework to reweight supervised fine-tuning data based on conditional information gain. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a training method rather than an evaluation benchmark or judge metric. | |
| Benchmarking Embedding Geometries and Clustering Algorithms for Unsupervised Patent Claim Classification Compares hyperbolic and Euclidean embeddings for unsupervised patent representation to improve cross-category prior-art discovery. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Evaluates clustering representations, not language model benchmarking or LLM-as-a-judge. | |
| Hindsight Structured Advice Distillation: Steering Executor Agents with Advisors Trained on Their Failures and Task Characteristics Trains an advisor model on past failures to add dynamic, structured guidance to frozen language model agent prompts. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on an agent advisor training framework rather than LLM benchmarking or evaluation methodology. | |
| Scaling Empowered Agents Presents a scalable method to maximize agent empowerment by guiding agents toward states affording numerous executable skills. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Explores unsupervised RL objectives rather than LLM benchmarks or evaluation methodologies. | |
| Provenance-Aware Belief Updating in Human-AI Feedback Loops Introduces a training objective enabling language models to update beliefs using report lineage rather than raw citation repetition. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a training method for belief updating rather than a comprehensive evaluation benchmark. | |
| Generalized Proximal Policy Optimization: Unifying LLM RL Algorithms from the Perspective of Divergence Proposes a generalized f-divergence trust-region framework to unify and stabilize reinforcement learning algorithms for LLM post-training. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on reinforcement learning policy optimization rather than benchmark design or evaluation. | |
| GeoGR: Spatial-Semantic Token Compression and Group Routing for Efficient 3D VLMs Proposes spatial-semantic token compression and layer routing to accelerate 3D vision-language model inference across multi-view inputs. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an architectural efficiency framework rather than an evaluation benchmark. | |
| Travel Back To Move Forward: Retrospective Exploration and Evolving for Long Horizon Agents Introduces a framework allowing agents to rewind long-horizon environments to earlier states while retaining learned exploratory knowledge. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Uses established benchmarks to assess agent backtracking rather than proposing new evaluations. | |
| Evolving Creatives with Clicks: CTR-Driven Advertising image Editing via Attribute Disentanglement and Preference Alignment Proposes an advertising image editing method that identifies and selectively refines visual attributes driving click-through rates. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an advertising image optimization pipeline, not an LLM or vision evaluation benchmark. | |
| LongHarness: Advancing Agentic Reasoning over 100M-Token Corpora via Stateful Evidence Management Presents an agent harness that coevolves evidence and conclusions to improve reasoning across 100-million-token document corpora. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces an agent framework for corpus-level reasoning, not a general evaluation benchmark. | |
| Attention Sees, MLP Refuses: Tracing the Attention-to-MLP Safety Pathway in LLMs Investigates internal transformer mechanisms, showing attention updates decode harmfulness while MLP layers primarily drive refusal token generation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on mechanistic interpretability of safety mechanisms rather than benchmarking LLM systems. | |
| SWEAR: Section-Wise Efficient Audio Reasoning via Self-Rated Difficulty Presents a difficulty-aware framework that allocates reasoning steps across sections to improve efficiency in large audio-language models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on an adaptive inference framework evaluated on standard benchmarks, not evaluation design. | |
| VETR: Execution-Grounded Self-Improving Agent for Symbolic Graphics Reasoning Introduces a recursive self-improvement agent framework that refines symbolic graphics programs using execution traces and runtime state changes. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Evaluates on existing Blender benchmarks rather than introducing a new evaluation suite. | |
| Continually Post-training Small Models via Synthetic Self-distillation Proposes a synthetic self-distillation method using a slightly larger aligner to enable continual post-training in small language models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Applies standard retention benchmarks to evaluate continual learning rather than proposing new evaluation methods. | |
| Locate, Judge, Read: Role-Specialized Tool Agents over a Knowledge-Point Index for Open-Domain Multimodal Document QA Designs a modular multi-agent pipeline separating search, verification, and reading for multimodal document question answering. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Evaluates on existing benchmarks rather than proposing novel LLM evaluation methodologies. | |
| Looking Ahead and Back: Anticipatory Generative Modeling for Road-Network Trajectories Introduces an anticipatory generative modeling approach for road networks to improve realistic next-transition selection in trajectory generation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on generative trajectory modeling methods rather than language model evaluation frameworks. | |
| DP-MIND: Differentially private memory-guided Long context Inference Proposes a memory-guided differential privacy framework enabling private inference over million-token contexts in LLM-as-a-service settings. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a privacy-preserving inference framework rather than evaluating models or introducing a general benchmark. | |
| Competence-Aware Orchestration for LLM-Based Multi-Agent Systems Presents a framework that dynamically adjusts role activation and multi-agent collaboration based on the model backbone's query-specific competence. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on multi-agent orchestration architecture rather than evaluation methodology or benchmark metrics. | |
| RoleWorld: Knowledge Transfer in World Modeling with Functional Role Abstractions Proposes a symbolic world modeling method using functional role abstractions to transfer interaction dynamics across environments for LLM agents. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on agent world modeling techniques rather than benchmark design or LLM evaluation. | |
| JANUS: Enhancing Long-Horizon Autonomous Research with Co-Evolving World Models Proposes JANUS, using a co-evolving world model to predict, verify, and guide exploration paths in autonomous scientific research. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces an autonomous agent framework rather than testing benchmark contamination or judge methods. | |
| Does Learning Recurrent Networks Matter in Partially Observable RL? Investigates whether frozen recurrent networks can provide sufficient memory for partially observable reinforcement learning without end-to-end training. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Investigates memory in POMDPs rather than LLM benchmarking or evaluation frameworks. | |
| Beyond Actions: Learning Future Operation Targets for Vision-Language-Action Models Introduces AFCE, training vision-language-action policies to predict future physical effects rather than raw action chunks directly. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Does not address language model evaluation benchmarks or LLM testing methodologies. | |
| ReSFlow: Rectifying Categorical Flows on Statistical Manifolds for One-Step Generation Introduces ReSFlow, rectifying categorical flows along geodesic trajectories on statistical manifolds to enable one-step discrete generation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Does not study LLM evaluation benchmarks or assessment techniques. | |
| Tabular Foundation Models Are Effectively Shallow Analyzes tabular foundation models and shows most transformer layers can be replaced with affine translators without sacrificing performance. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Studies model architectural depth rather than benchmark contamination or evaluation methods. | |
| Forgetting is Not Always Bad: A Neuro-Inspired Memory Repair Mechanism for Poisoned LLM Agents Introduces DREAM, an agent-intrinsic defense that triggers functional forgetting of poisoned memory entries through implicit reactivation and topological diagnosis. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Does not focus on LLM benchmark validation, saturation, or evaluation metrics. | |
| Agent4Spec: An LLM-based Agentic System to Optimize API Specifications through Automated Validation and Market Analysis Presents Agent4Spec, a multi-agent framework that automatically validates OpenAPI syntax errors and identifies functional API gaps through market analysis. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces an application system and dataset rather than studying LLM benchmark reliability. | |
| BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning Distillation Proposes BoundaryEvo, dynamically adapting training data around a student model's evolving capability boundary during long-chain reasoning distillation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Does not focus on benchmark design, contamination, or LLM evaluation frameworks. | |
| PRIME: Pre-Retrieval Index Modeling Enhancement for Structured Document Retrieval-Real Proposes PRIME, an index-side optimization framework using semantic restructuring and deterministic metadata synthesis to improve document retrieval. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Uses standard benchmarks to evaluate retrieval rather than studying LLM evaluation methodology. | |
| Masked Diffusion Language Agents for Tool-Integrated Chemical Reasoning Introduces ChemDiffAgent, applying masked diffusion to plan tool use and reasoning trajectories bidirectionally for chemical problem-solving. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a new agent architecture rather than evaluating benchmarks or LLM-as-a-judge schemes. | |
| AlphaNIST: Emergent Assembly Skills from Binary Task Rewards Evaluates whether robotic manipulation agents can learn complex multi-step assembly skills purely from sparse binary task rewards. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Examines RL policy learning rather than testing or benchmarking LLM evaluation methods. | |
| RegulaTutor: A Theory-Informed Framework for Long-Term Personalized Learning Presents RegulaTutor, a personalized tutoring framework that grounds instructional policies in explicit cognitive and self-regulated learning theories. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents a personalized tutoring framework rather than a general LLM evaluation benchmark. |
3,241–3,300 of 3,345← PreviousNext →