Every paper checked — nothing hidden
All papers
Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.
All3,345In brief3More (on-topic)587Not relevant2,721Not assessed24Older45Could not confirm25Blogs with no paper54All blogs57
| Paper | Topic | Relevance | Feedback |
|---|---|---|---|
| JetSpec++: Foresight Parallel Drafting with Self Refinements The paper introduces a speculative decoding system combining self-refinement and pipelined drafting to reduce wall-clock generation time. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces an inference acceleration method, not an evaluation benchmark. | |
| Evolve Together, Fall into Place: How Social Scaffolding Shapes Self-Evolving Multi-Agent Systems The paper investigates how team composition and organizational interdependence affect performance in self-evolving multi-agent systems. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Evaluates multi-agent evolution, not benchmark design. | |
| Delta Tokenizer: Reference-Identical Tokenization of Session Deltas and Segment-Parallelism for Long Prompts on Multicore CPUs The paper introduces a CPU front-end tokenizer that encodes only session deltas to speed up multi-turn inference. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Systems paper on serving speed, not LLM evaluation benchmarks. | |
| EGCForest: Structured Agent Memory Organization for Enterprise Group Chats The paper proposes a multi-view hierarchical memory framework to improve information retrieval from enterprise group chats. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes a retrieval framework, not general benchmark methodology. | |
| MolHop: Scaffold-Hopping Molecular Design with Agentic Reinforcement Learning The paper presents an agentic reinforcement learning framework that navigates organized chemical spaces for cross-scaffold molecular discovery. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Not about general LLM evaluation or benchmarks. | |
| MemDuet: Learning Dual-Process Memory for Long-Context Reasoning The paper introduces a dual-process memory framework coordinating small and large models for efficient long-context reasoning. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces a framework evaluated on existing benchmarks, not benchmark design. | |
| To Reason but Not to Stop: Understanding and Recovering Pruned Large Reasoning Models The paper shows pruned reasoning models fail by looping, proposing an activation steering method to restore proper termination. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Analyzes pruned model behavior, not LLM benchmarking methodology. | |
| Choosing the Number of Attention Heads from Intrinsic Dimension: A Small-Scale Study The paper tests a heuristic to set the number of transformer attention heads using the intrinsic dimension of token representations. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Evaluates a training heuristic, not LLM evaluation benchmarks. | |
| Gray-Box Hallucination Detection in LLMs with Token Identities The paper introduces a lightweight gray-box hallucination detector combining token probabilities with vector representations of token identities. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on hallucination detection algorithms, not evaluation benchmarks. | |
| FACES: Fine-grained Gated Semantic Alignment for Multimodal Emotional Support Conversation The paper introduces a gated semantic alignment framework with multi-source rewards to improve multimodal emotional support conversations. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces a conversational framework, not general benchmark evaluation. | |
| Language Exposure Matters: Distribution and Order in Multilingual BabyLM Pre-training The paper evaluates how language exposure distribution and ordering affect the performance of multilingual models trained with limited data. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on pre-training regimens, not benchmark evaluation methodology. | |
| Why Generate a Click? Plug-in Decision Heads for Web Agents This paper introduces plug-in decision heads that read action choices directly from visual agent hidden states rather than generating text tokens. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Evaluates web agent performance, but focus is on architecture. | |
| Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility This paper surveys 138 studies on sparse autoencoders in NLP, examining dictionary design, feature recoverability, and practical utility. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Surveys feature interpretability, not standard LLM evaluation benchmarks. | |
| Gap Detection: Mitigating Homogenization through Demand and Defensibility This paper introduces Gap Detection, an optimization method that identifies unoccupied semantic directions to reduce homogenization in open-ended model answers. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on generation diversity rather than benchmark evaluation. | |
| RM-OPD: On-Policy Distillation for Generative Reward Models This paper presents RM-OPD, a framework that trains domain-specialized reward models via on-policy distillation into a generalist model. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Develops generative reward models, not general evaluation benchmarks. | |
| TreeSpark: Learning to Draft the Trees You Verify for Speculative Decoding This paper introduces TreeSpark, a framework training semi-autoregressive drafters to generate candidate trees aligned with speculative verification. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on inference acceleration rather than evaluation metrics. | |
| When Occupation-derived Skills Don't Help: On the Limits of Skill-Augmented Career Path Prediction This paper analyzes career path prediction systems, reporting that explicit skill representations provide marginal improvement over text-only job descriptions. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on career trajectory tasks rather than LLM benchmarks. | |
| Scaling Multilingual Mixture-of-Experts: How Language Coverage Changes Capacity and Compute Allocation This paper proposes a joint scaling law evaluating how adding experts in multilingual mixture-of-experts models affects language capacity and loss penalties. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Investigates scaling laws rather than benchmark evaluation reliability. | |
| Repairing Speech Access After Text Updates with Frozen Affine Fields This paper introduces AFFIRM, using affine fields on frozen speech models to repair spoken access to recently updated text knowledge. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces modality repair layers, not benchmark methodology. | |
| Anuvani-LID: Robust and Calibrated Spoken Language Identification for Indian Languages This paper introduces Anuvani LID, a dedicated 115M parameter model for spoken language identification across 22 Indian languages. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Evaluates spoken language ID models, not broad LLM benchmarks. | |
| Neural POS Tagging for the Low-Resource Saraiki This paper introduces an annotated part-of-speech tagging dataset and evaluates pretrained transformer encoders for the low-resource Saraiki language. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on low-resource NLP labeling rather than LLM benchmarks. | |
| Taking the PULSE of Medical Reasoning: Step-Level Retrieval Timing for Knowledge-Grounded Inference This paper proposes PUlSE, using internal layer representations to detect factual errors and trigger step-level retrieval during medical reasoning. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces a retrieval mechanism, not benchmark validation or scoring. | |
| MMLA-RSI: Recursive Self-Improvement through RTT and RTMA This paper proposes MMLA-RSI, a recursive self-improvement framework using reasoning-time policy training and authoritative memory adaptation. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes agent architectures, not benchmark analysis or judge models. | |
| Anuvani-Diarization: Zero Shot Speaker Diarization and Annotation Conventions Across 22 Languages This paper presents Anuvani-Diarization, evaluating zero-shot speaker diarization across 22 Indian languages with uncertainty-aware silver supervision. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Targets speech diarization benchmarking, not general LLM evaluation. | |
| Do AI Agents Gossip? Evaluative Talk in Human and AI-Agent Online Communities This paper investigates whether gossip discourse naturally emerges in autonomous AI agent online communities compared to human discussion boards. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Analyzes agent social discourse, not standard LLM evaluation benchmarks. | |
| Lexical Persistence Without Semantic Entrenchment in AI Agent Communities This paper measures semantic drift and stability across AI agent communities, finding higher drift and lower temporal continuity than human forums. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Investigates linguistic dynamics across platforms rather than proposing an LLM benchmark. | |
| Cascaded Retrieval with Cross-Modal Query Refinement for Efficient Visual Document Search This paper proposes SCOUT, a cascaded two-track framework combining visual candidate filtering, offline semantic summaries, and query refinement for document retrieval. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Presents a retrieval framework evaluated on existing benchmarks, not a benchmark itself. | |
| FOLD: Position-Free Far Memory with Local Order for Long-Context Speculative Decoding This paper introduces FOLD, a training-free speculative decoding approach treating distant tokens as position-free far memory for long-context generation. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes an inference speedup technique rather than an evaluation benchmark. | |
| The COUNT: Counting N-grams Unlocks Scaling on Host Memory and Training-Free Adaptation This paper introduces COUNT, scaling language model memory without training parameters by storing precomputed n-gram counts directly in host memory. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces a pretraining method evaluated on loss, not an evaluation benchmark. | |
| MolLean: A Process-Verifiable Language for Chemical Reasoning This paper introduces MolLean, a verifiable language and tool interface that grounds chemical reasoning in formal state transitions and explicit checks. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Evaluates chemical reasoning against direct prompting rather than creating a general LLM benchmark. | |
| AdaRSI: On-Demand Recursive Self-Improvement for Embodied Agents This paper introduces AdaRSI, an on-demand recursive self-improvement framework for embodied agents that isolates and fine-tunes only persistent sub-step failures. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces a self-improvement framework rather than a general LLM evaluation benchmark. | |
| ReaDy: Recurrent States as Dynamic Context for Speculative Drafting in Hybrid Linear Attention Large Language Models This paper presents ReaDy, a speculative decoding method using hybrid linear attention recurrent states to provide position-specific context during drafting. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces an efficient decoding method rather than a general LLM evaluation framework. | |
| Latent Compaction with Test-Time Training Compresses long-context key-value caches using test-time training on generic document prompts without task-specific labels. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces a compression technique evaluated on existing benchmarks, not an evaluation benchmark. | |
| Personalising Deformable Image Registration with Clinical Context and Deformation History for Adaptive Radiotherapy Integrates clinical text and deformation history into medical image registration models for adaptive radiotherapy planning. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces a medical imaging technique rather than a language model benchmark. | |
| The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery Presents an end-to-end distributed system orchestrating coding agents to execute computational research experiments alongside human scientists. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Describes an agentic scientific system rather than an LLM evaluation benchmark. | |
| Non-Abelian quantum Hall states at strong Landau level mixing: finding anyons with first-principles AI Uses a neural-network variational wave function to model non-Abelian quantum Hall states with realistic Landau level mixing. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on quantum Hall physics rather than language model evaluation benchmarks. | |
| Discovering Undisclosed Concepts in LoRA Adapters for Diffusion-Based AIGC Models Proposes an optimization method to discover undisclosed or harmful generation concepts embedded within diffusion model adaptation weights. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes an optimization method to inspect adapters rather than an evaluation benchmark. | |
| Scaling Cheap Synthetic Pretraining Improves Indoor Radio-Map Prediction Improves indoor radio-map prediction by pretraining models on diverse synthetic floorplans and an open-source ray tracer. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Evaluates synthetic radio pathloss modeling rather than language model benchmarks. | |
| Who Should We Listen to More? Welfare-Aware Preference Acquisition for Pluralistic Alignment Introduces a welfare-aware objective to allocate limited human feedback across diverse individuals when training pluralistic alignment models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces an alignment data acquisition framework rather than an evaluation benchmark. | |
| Loss Curvature as a Sampling-Free Proxy for Memorization in Diffusion Models Proposes input loss curvature as a sampling-free metric to detect the onset of training data memorization in diffusion models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Investigates diffusion memorization metrics rather than language model benchmarking. | |
| Graph Coarsening Enables Escaping Local Optima in Compute-Constrained Nonconvex MAX-CUT Combines spectral graph coarsening with continuous relaxation to find high-quality solutions for compute-constrained MAX-CUT problems. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Evaluates a mathematical solver rather than language model evaluation frameworks. | |
| RLPE: Teaching LLMs to Prompt with Reinforcement Learning Uses reinforcement learning to train an auxiliary model that inspects execution traces and generates targeted prompts for another system. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces a prompt-learning method rather than evaluating benchmark integrity or LLM-as-a-judge. | |
| How Inner Optimizers Shape Replica Scaling Studies how the choice of local optimizers affects model performance degradation when scaling distributed training replicas under fixed budgets. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Studies optimizer scaling dynamics during training rather than post-training model evaluation. | |
| Revisiting Generative vs. Dense Retrieval: A Static Value Prior Reverses the Gap Shows that incorporating static item value into dense retrieval outperforms generative retrieval in e-commerce recommendation settings. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on retrieval algorithms rather than language model evaluation frameworks or leaderboards. | |
| Plan or Retrieve First? A Scale Study of Tool Discovery Architectures Compares retrieve-then-plan and plan-then-retrieve strategies for tool discovery across varying tool registry complexities. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on agent tool discovery architectures rather than general benchmark validity or evaluation. | |
| DexMimic: A Generalist Dexterous Hand-Object Tracking Policy via Value-Guided Reset Allocation Introduces a generalist policy that tracks diverse dexterous hand-object interactions using value-guided reset allocation during training. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on reinforcement learning for robotics rather than testing language models. | |
| GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning Presents a large graphical user interface grounding dataset and a combined distillation and reinforcement learning training method for agents. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Introduces a dataset and model for GUI grounding rather than evaluating LLM benchmark reliability. | |
| Genesis Steering: Better Activation Steering Through Training Data Steers language model activations using vectors derived directly from the model's own training data or inferred proxy data. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Develops an activation steering technique rather than an evaluation framework or benchmark. | |
| Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming The paper proposes a training-free structured pruning method that trims multilayer perceptron layers while preserving attention mechanisms for reasoning tasks. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Proposes a pruning method rather than a novel benchmark or evaluation technique. | |
| Learning in Orbit: Onboard Training and Closed-Loop Attitude Control with a Thermal Segmentation Model on a Spacecraft The paper demonstrates onboard thermal segmentation training and closed-loop attitude control on a satellite in low Earth orbit. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Demonstrates an onboard robotics deployment rather than proposing LLM evaluation frameworks. | |
| PairIT: Autoregressive Transformers for Low-Data Molecule Optimization The paper introduces a transformer framework trained on matched molecular pairs for prompt-guided, multi-property molecular optimization in low-data regimes. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on domain-specific molecule optimization rather than general LLM evaluation methodology. | |
| Imagining Goals with Geometric Horizon Models for Policy Generalization The paper proposes sampling from a geometric horizon world model to imagine novel outcomes for zero-shot policy generalization. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Evaluates RL policy generalization on existing benchmarks rather than proposing LLM benchmarks. | |
| Plant Memory: Technical Language Processing and a Verified Maintenance Ontology for Agent-Accessible Failure Knowledge from SAP PM Work Orders The paper presents a system using technical language processing to transform industrial maintenance work order records into an ontology. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on ontology building rather than general LLM evaluation benchmarks or methods. | |
| CAFE: Confining Feature Distillation to Language-Routed Adapters for Compact Multilingual Automatic Speech Recognition The paper proposes confining feature distillation loss to language-routed adapters to balance performance in compact multilingual speech recognition models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Applies existing speech leaderboards rather than proposing LLM benchmarks or evaluation methods. | |
| Graph-Based Orchestration of Heterogeneous AI Models: A Control Node Approach to Composite Intelligence The paper presents a graph-based framework using control nodes and behavior trees to coordinate heterogeneous artificial intelligence models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Presents a system orchestration architecture rather than an evaluation benchmark or methodology. | |
| Compiling Expert Procedures into Verified Programs: When a Small Model Is Enough The paper compiles teacher model reasoning traces into verified programs that guide small models through complex procedural tool-use tasks. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Focuses on a procedural execution system rather than benchmarking methodologies. | |
| Verification Turns Capability Gaps into Routing Decisions: Small Language Models for Clinical Documentation The paper studies using an agentic verification architecture to enable small on-device models to generate factual clinical documentation. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Applies a verifier architecture to clinical note generation rather than benchmarking LLMs. | |
| Learning from Self-Generated Responses: Reliability--Uncertainty Self-Distillation for Reasoning The paper proposes a self-distillation method weighting loss by judged reliability and student predictive uncertainty on open-ended reasoning tasks. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Evaluates a training method across existing benchmarks rather than benchmarking LLM systems. | |
| Multi-System Cognitive Steering for Generative Communication The paper presents a framework to steer language models using multimodal context to expand abbreviated text inputs while preserving user intent. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Evaluates simulated abbreviation expansion rather than introducing a general LLM benchmark. | |
| Diagnosing Low-Confidence Patient Profiles via Localized Harmful Covariate Shift Detection The paper introduces a diagnostic pipeline to identify low-confidence patient subgroups and detect harmful covariate shifts in clinical prediction models. Preprint | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | Not relevant Deals with subgroup covariate shift detection in clinical predictive models, not LLMs. |
1,261–1,320 of 3,345← PreviousNext →