Every paper checked — nothing hidden
All papers
Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.
All3,345In brief3More (on-topic)587Not relevant2,721Not assessed24Older45Could not confirm25Blogs with no paper54All blogs57
| Paper | Topic | Relevance | Feedback |
|---|---|---|---|
| Self-Distillation with Verbal Regularization This paper proposes verbal regularization prompts for on-policy self-distillation to prevent teacher models from taking shortcut reasoning paths. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | Not relevant Introduces a distillation training technique, not evaluation methods or benchmarks. | |
| HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision This paper introduces HexaBee-4B, a translation model trained across six languages using joint multilingual answering and bidirectional supervision. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | Not relevant Presents a trained translation model rather than an evaluation framework or benchmark. | |
| PersonaFactor: Preference Factors with Dynamic Selection for LLM Personalization This paper presents PersonaFactors, a framework that distills reusable preference patterns across tasks for dynamic prompt personalization in language models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | Not relevant Presents a dynamic personalization framework rather than an evaluation benchmark. | |
| CultureBot: A Framework for Retrieval-Augmented Exploration of Cultural Heritage Data This paper introduces an interactive retrieval-augmented framework that combines knowledge graphs and text retrieval to explore digital cultural heritage archives. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | Not relevant Describes an application framework rather than general LLM evaluation benchmarks. | |
| Differential Privacy Protected Preference Alignment on Survey Derived Data This paper presents a label differential privacy framework to protect survey-derived ordinal preference data during language model alignment. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | Not relevant Presents privacy-preserving alignment methods rather than LLM evaluation benchmarks. | |
| The interpreter as developer: An autoethnographic design study of building a computer-assisted interpreting tool through human-LLM co-creation This paper presents a case study of an interpreter co-creating a local computer-assisted interpreting tool using large language models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | Not relevant Presents a qualitative design study rather than standard benchmarks or evaluation methods. | |
| Ontology-Based Post-Coordinated Expression Assembly: from Clinical Narrative to Queryable Knowledge Graphs This paper introduces UNIGRAPHE, a deterministic method to assemble clinical text into queryable knowledge graphs conforming to SNOMED CT grammar. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | Not relevant Presents an information extraction methodology rather than an LLM evaluation benchmark. | |
| Communication, Vocalisation, and Language This position paper argues that large language models perform language as social practices rather than building distributed representations of symbols. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | Not relevant Conceptual perspective piece rather than an empirical benchmark or evaluation methodology. | |
| Intent-Triggered Anchoring for Multi-Token Format Units in Parallel Decoding of Diffusion Language Models This paper introduces a training-free decoding rule that anchors multi-token format units to ensure parseable parallel generation in diffusion language models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | Not relevant Focuses on decoding algorithms rather than benchmarking or evaluating language models. | |
| Support or Magnitude? Characterizing Neuron Relevance for Instruction Data Selection This paper examines neuron relevance characterization methods based on magnitude and distribution support for selecting instruction fine-tuning data. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | Not relevant Proposes data curation metrics rather than benchmarks or evaluation frameworks. | |
| Spectral-Driven Adaptive-Rank Optimizer for Large Language Model Reinforcement Learning with Verifiable Rewards This paper investigates gradient spectral properties during reinforcement learning with verifiable rewards to design an adaptive-rank optimizer for reasoning models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Focuses on optimization methods rather than benchmarking or evaluation protocols. | |
| Know Yourself and Know Your Neighbour: A Syntactically Informed Self-Supervised Compositional Sentence Representation Learning Framework using a Recursive Hypernetwork This paper presents a self-supervised recursive hypernetwork framework using parse trees to learn syntactically informed compositional sentence representations. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Evaluates representations on probing tasks rather than providing an LLM evaluation benchmark. | |
| What Is the "Self" of a Large Language Model? This paper investigates internal model representations that separate assistant actions from externally provided text across conversational layers. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Presents mechanistic interpretability analysis rather than standard benchmarks or evaluation methodologies. | |
| Lost in Contrast: Instruction-Preserving Decoding for Reliable Visual Question Answering This paper introduces an instruction-preserving decoding strategy to prevent visual contrastive decoding from discarding valid abstention responses. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Focuses on a decoding method rather than proposing a general LLM evaluation benchmark. | |
| A Bethe-Ansatz Source of Early Token Correlations: A marginal-preserving construction for parallel proposals This paper constructs joint token probability laws from physics models to improve speculative verification in parallel language model decoding. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Theoretical statistical physics paper, not an evaluation methodology. | |
| TPPO: Tracking-Value PPO For Language Model Reasoning This paper proposes a cumulative value parameterization method for reinforcement learning to improve credit assignment in multi-step language reasoning. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Introduces a training algorithm, not an evaluation method or benchmark. | |
| The Bell Boundary of Shared Representations This paper uses Bell's theorem bounds to analyze capacity limits and task interference in multi-task shared representations. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Investigates representational capacity limits rather than benchmark evaluation frameworks. | |
| Estimating Uncertainty of Omnimodal Large Language Models via External Omnimodal Embeddings This paper develops an uncertainty estimation framework for omnimodal models using external embeddings to score reasoning grounding and coherence. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Focuses on uncertainty estimation techniques rather than benchmarking LLMs. | |
| Event Memory and Narrative Dynamics: A Large Language Model Framework for Multidimensional State Spaces This paper presents a language model framework to extract event representations from autobiographical narratives into geometric state spaces. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Proposes a cognitive modeling framework, not an LLM benchmark or evaluation protocol. | |
| Predicting Mental Health Help-Seeking Behaviour on Reddit: A Survival Analysis of Linguistic and Behavioural Signals This paper uses survival analysis on Reddit text and activity to predict when users will seek mental health support. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Presents a domain-specific predictive modeling study rather than an LLM benchmark. | |
| MENTOR: Procedural Shielding of Tool-Using LLM Agents This paper introduces a system to ensure tool-using language model agents comply with natural language company procedural manuals. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Proposes a runtime shielding framework rather than a general evaluation benchmark. | |
| UZHCL_for_the_win at Touche: Encoder Fine-Tuning, Synthetic Augmentation, and an Underpowered Encoder-LLM Comparison This paper evaluates encoder fine-tuning and synthetic data augmentation for fallacy detection and argument classification on Reddit discussions. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Reports competition participation results rather than a general evaluation benchmark. | |
| Gluing Local Contexts into Global Meaning: A Sheaf-Theoretic Decomposition of Transformer Representations This paper proposes a mathematical framework using cellular sheaves to isolate phrasing-invariant content representations in transformer language models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Presents an interpretability framework rather than benchmark evaluation. | |
| Soft Teacher Forcing The paper proposes combining teacher-forced embeddings with soft tokens to improve supervised fine-tuning on reasoning datasets without changing inference. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Proposes a training method, not a benchmark or evaluation methodology. | |
| Welcome to the Age of Agent Ads: Using On-Policy Attacks to Steer Agentic Decisions The paper investigates on-policy indirect manipulation attacks that steer autonomous agent purchasing and ranking decisions via web retrieval. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Focuses on adversarial steering attacks rather than LLM benchmarking methods. | |
| MLAfication: Efficiently Enabling High-Fidelity MLA for Modern LLMs The paper introduces an attention conversion method enabling multi-head latent attention on modern architectures with reduced memory footprints. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Presents an architecture optimization method rather than an evaluation benchmark. | |
| Nufo: A Benchmark for Ewe Automatic Speech Recognition The paper introduces a 174-hour speech benchmark and evaluates baseline models on Ewe automatic speech recognition. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Focuses on speech recognition benchmarking rather than language model evaluation. | |
| From Sampled Evidence to Radiology Reports: Objectives, Constraints, and Realization The paper evaluates how objective functions and candidate constraints affect decision conversion when generating radiology reports from sampled evidence. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Studies report realization trade-offs rather than general LLM benchmarking. | |
| UMCR: Universal Medical Concept Recognition through Guideline Integration and Agentic Linking The paper introduces a framework combining guidelines and agentic linking to ground medical concepts across diverse text types and ontologies. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Applies existing MedPath benchmark rather than proposing LLM evaluation methods. | |
| PIG: Parallel Inconsistency Guidance for Accelerating Diffusion Language Models The paper introduces a lightweight guidance model to estimate parallel decoding inconsistency and accelerate diffusion language models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Uses standard benchmarks to evaluate speedup rather than creating evaluation methods. | |
| Distilling Text LMs into Speech-to-Speech LMs by Matching Internal Text Predictions The paper proposes a knowledge distillation method that transfers text language model capabilities to speech-to-speech models using internal text interfaces. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Presents a pretraining approach rather than benchmark evaluation methods. | |
| Scaling Inference-Time Compute with Teams of Coding Agents The paper studies compute allocation across multi-agent coding teams to identify Pareto-optimal trade-offs between cost and performance. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Studies team configurations rather than introducing evaluation benchmarks. | |
| A Sense of Proprioception in Language Models The paper trains language models to report internal states through supervised fine-tuning and on-policy learning. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Studies internal state reporting rather than proposing general LLM benchmarks. | |
| Everyone Was Right: Lexical Representations in Language Models Are Both Abstract and Holistic The paper investigates lexical representations in transformer models by decomposing representations into item, class, and interaction components. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Investigates representation theory rather than evaluation frameworks or benchmarks. | |
| EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud Delegation The paper proposes an edge-cloud collaboration framework that selectively delegates complex agent steps from local models to cloud models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Presents an operational framework rather than an LLM evaluation standard. | |
| TDR: Tagalog Diacritic Restoration Dataset The paper introduces an annotated dataset and retrieval approach for Tagalog homograph diacritic restoration. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Focuses on a niche Tagalog NLP corpus rather than general LLM benchmarking. | |
| Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation The paper studies long chain-of-thought distillation and argues improvements stem from structured reasoning patterns rather than factual correctness. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Analyzes chain-of-thought distillation mechanics rather than benchmarking methodology. | |
| StateHarness: Maintaining Evolving Project State for Long-Horizon Coding Agents The paper presents a runtime framework that tracks evolving repository state and dependencies to assist long-horizon coding agents. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Introduces an agent framework rather than a general evaluation benchmark. | |
| DisaggKD: Efficient Knowledge Distillation through Teacher-Student Disaggregation and a Fused Loss The paper proposes disaggregating teacher and student models across separate nodes with a fused loss to accelerate knowledge distillation. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Not centered on evaluation methodology or LLM benchmarks. | |
| Decouple to Forget: Semantic Mediation for Sensitive Unlearning in Large Language Models The paper proposes an unlearning method that removes sensitive facts from model responses while preserving surrounding non-sensitive factual context. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Evaluates unlearning methods rather than proposing new general evaluation frameworks. | |
| The Image Still Has a Say: Final-Answer Repair for Multimodal Chain-of-Thought The paper repairs multimodal chain-of-thought final answers using original image features without altering the generated reasoning trace. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Presents an inference intervention method rather than an evaluation benchmark. | |
| RALI@iKAT 2026: Retrieval-aware User Profile Denoising via Personalized Conversational Dense Retrieval The paper presents a retrieval-aware denoising method for conversational user profiles to improve personalized dense retrieval. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Presents a retrieval method evaluated at a benchmark, not a general evaluation methodology. | |
| Encoding Context into LLM Weights: What Makes It Context? The paper investigates methods for encoding context into model weights so that language models properly apply the temporary context during subsequent inference. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Focuses on context encoding methods rather than benchmark design. | |
| Pushback in a Haystack: Language Models Respond to Factual Errors Ambiguously and Ineffectively The paper evaluates how language models correct user misconceptions across five languages, showing indirect pushback often fails to fix factual errors. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Introduces a labeling framework for pushback but focuses on behavioral sycophancy analysis. | |
| Geometric Structure Predicts Per-Example Neighborhood Loss Under PCA Truncation The paper analyzes geometric properties of text embeddings to predict how many nearest neighbors an individual example loses under dimensionality reduction. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Analyzes geometric stability under PCA truncation rather than establishing LLM benchmarks. | |
| More Reader Data or a Better Teacher? Updating a Model of Crowd Highlights in Late 2026 The paper compares direct prompting, model-distilled supervision, and larger reader datasets for predicting user text highlighting behavior on web pages. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Compares training data scaling versus model teachers rather than proposing LLM benchmarks. | |
| Many Right Answers, One Training Example: Removing Responder Noise from GRPO for LLM Recommenders The paper proposes ReMaRk, a reinforcement learning method that trains recommender models using all available reader replies simultaneously to reduce variance. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Focuses on RL training optimization rather than evaluation methodology or benchmarks. | |
| Full-duplex Spoken Language Modeling with Discrete Diffusion The paper presents Diffuplex, a full-duplex spoken language model that separates speech timing from content generation using discrete diffusion. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Presents a spoken language model rather than an evaluation benchmark. | |
| Where Words Begin and End The paper evaluates audio forced aligners and fine-tunes Whisper to predict human-aligned word boundaries across read, conversational, and telephone speech. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Focuses on fine-tuning speech models rather than general LLM evaluation benchmarks. | |
| BRIDGE: Selective Boundary Repair with Dependency Graph Evidence in Multi-Agent Systems The paper introduces BRIDGE, a runtime intervention framework that detects and repairs broken information handoffs across dependency graphs in multi-agent workflows. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Proposes a multi-agent repair framework rather than an evaluation benchmark. | |
| Perturbation, Not Provenance: Why Activation Steering Does Not Make a Robust Text Watermark The paper tests activation steering as a text watermarking technique, finding that standard logit-based watermarking provides more robust detection against paraphrasing. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Evaluates watermark robustness rather than general LLM capability benchmarks. | |
| Prefilling-dLLM v2: Learned Chunk Routing for Long-Context Diffusion Language Models The paper proposes a training approach that uses learned chunk routing to speed up long-context diffusion language model processing. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Tests on RULER but primary contribution is an efficient context architecture. | |
| Batch-Paced: Fixed-Budget Batch Decoding in Discrete Diffusion Language Models The paper proposes Batch-Paced, a method that dynamically allocates decoding compute across batched requests to improve discrete diffusion language model inference. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Evaluates on benchmarks but main contribution is an efficient serving algorithm. | |
| Agent-Reflex: Teaching Language Agents to Act Reflectively The paper introduces Agent-Reflex, a training framework designed to teach language agents how to reflect and recover from noisy environment feedback. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Presents a training framework rather than evaluation benchmarks. | |
| Fish Audio Drama 3 Technical Report The paper introduces Fish Audio Drama 3, an open-source text-to-speech model designed for multi-character dialogue, long-form synthesis, and iterative voice revision. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Presents a model technical report rather than evaluation benchmarks. | |
| A Benchmark for Bangla Task-Oriented Dialogue Understanding The paper introduces BanglaTOD, a tri-lingual benchmark for task-oriented dialogue understanding covering Bangla, English, and romanized Banglish across nine service domains. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Presents a language-specific dialogue dataset rather than broader LLM evaluation. | |
| From Agents to Edges: Paired Optimization for Multi-Agent Collaboration The paper introduces an optimization method that jointly refines communication instructions between communicating agents to improve multi-agent collaboration. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Focuses on optimization methods rather than benchmark design. | |
| Narrative Lookahead in LLMs: Event-Level Decodability, Predictive Advantage, and Steering The paper probes language model hidden states across generated stories to test whether models anticipate narrative events multiple steps ahead. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Does not focus on benchmarking or LLM evaluation frameworks. | |
| SparPR: Runtime KV-Cache Recovery for Resource-Constrained LLM Inference on Consumer Hardware The paper presents SparPR, a runtime key-value cache recovery mechanism that reduces memory bottlenecks during model inference on consumer hardware. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Does not introduce or study benchmarks for LLM evaluation. | |
| TRACESTEER: Trace Driven Activation Steering for Hallucination Mitigation in Diffusion Language Models Proposes dynamic activation steering using preceding denoising predictions to reduce hallucinations in diffusion language models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | Not relevant Focuses on mitigation methods rather than evaluation benchmarks. |
2,041–2,100 of 3,345← PreviousNext →