Every paper checked — nothing hidden
All papers
Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.
All3,345In brief3More (on-topic)587Not relevant2,721Not assessed24Older45Could not confirm25Blogs with no paper54All blogs57
| Paper | Topic | Relevance | Feedback |
|---|---|---|---|
| Project Greenhouse: Progress Toward Fully Open and Sovereign Agentic Search The paper describes training a competitive decoder-only search reranker from scratch using modest computational resources and common datasets. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents a trained reranker family rather than an evaluation framework or benchmark study. | |
| MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement The report presents an omni-modal model family trained by scaling reinforcement learning throughput, environments, and agentic reward compute. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces a model and RL training methodology rather than a benchmark or evaluation method. | |
| ILM: An AI-Powered Storytelling Educational Tool The paper introduces an educational tool combining Arabic knowledge graphs and passage retrieval to generate narrative-based questions. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Uses LLM-as-a-judge as a minor component rather than studying LLM evaluation methodology. | |
| Rehearse Everything, Remember Nothing: Attic-KV Rehearses What Will Be Read The paper shows that whole-context KV cache rehearsal harms compression under tight budgets, proposing targeted rehearsal to preserve answer-relevant entries. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a KV cache compression algorithm rather than a general evaluation benchmark. | |
| When KL Regularization Misfires in Group Policy Optimization Analyzes interaction failures between KL regularization and rewards in group policy optimization and proposes a calibrated surrogate objective. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces an optimization algorithm rather than an evaluation benchmark or dataset. | |
| SciTBERT: A family of chronologically consistent language models for scientific and technological language processing Releases BERT models pre-trained with yearly chronological cutoffs to reduce lookahead bias in scientific and patent text mining. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Develops a domain-specific encoder family rather than a general LLM evaluation benchmark. | |
| DiffuPlex: Accelerating Full-Duplex Spoken Dialog Models via Rolling Masked Diffusion Presents a rolling masked diffusion method that accelerates full-duplex spoken dialog models by forecasting multiple future interaction frames. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents an inference acceleration framework rather than an evaluation benchmark. | |
| Language Models as AI Research World Models Evaluates whether language models can act as world models to predict empirical experimental outcomes across AI research workflows. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on world models for experiment prediction rather than LLM benchmarking methods. | |
| TokenRouter: Efficient Serving System for Token-Level LLM Routing Designs an inference serving system optimized for dynamic, token-level routing across multiple language models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an inference system architecture rather than a benchmarking framework. | |
| NativeScope: Relation-Localized Retrieval over Native Topology with a Correct Anchor Proposes a scope-then-rank retrieval method using document topology and relational structure rather than pure semantic similarity. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Evaluates a specific retrieval algorithm rather than establishing a broad LLM benchmark. | |
| EgoVoice: Proactive Spoken Assistance from Egocentric Multimodal Streams Develops a framework for egocentric multimodal assistants to decide both when to speak and what guidance to provide. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a model framework rather than general LLM evaluation benchmarks. | |
| HarnessSQL: Harness-Native Training for SQL Agents in Realistic Database Environments Introduces a post-training framework that trains text-to-SQL agents directly inside interactive database execution environments. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents a training framework rather than proposing a general LLM benchmark. | |
| SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference Identifies distribution shift during autoregressive decoding and proposes a decoding-aware pruning method to accelerate model inference. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an inference pruning technique rather than an evaluation benchmark. | |
| VFold: Symmetry-Aware Cross-Layer Value Cache Compression Presents a symmetry-aware value cache merging method to compress key-value memory during long-context decoding without architectural changes. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents a systems optimization technique rather than an evaluation benchmark. | |
| Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model-Skill Co-Evolution Proposes a method to select a compact subset of procedural skills that provide effective training signals during model distillation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces a distillation method rather than a general LLM evaluation benchmark. | |
| Latent Core Tokenizer: Compress, but Meaningfully Introduces a multilingual tokenizer that uses structural discovery and linguistic constraints to improve downstream model representations. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces a tokenization method rather than an LLM evaluation methodology or benchmark. | |
| Long Text to Predictive Features: LLM-Guided Blockwise Feature Engineering via Executable Program Search Presents a framework that uses language models offline to generate executable feature-engineering code for tabular risk models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents a feature engineering method rather than an LLM evaluation framework or benchmark. | |
| ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills Proposes a framework that encodes visual trajectories into reusable skill cards to reinforce vision-language agent planning and rewards. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an agent framework and training method rather than an evaluation benchmark. | |
| Contextual Correction of Parallel Draftsfor Speculative Decoding Introduces a lightweight correction mechanism to restore token dependencies during parallel draft generation in speculative decoding. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Focuses on speculative decoding speedup rather than novel evaluation methodologies. | |
| CLUE: A Contrastive Learning Framework for Unsupervised Event Type Induction and Annotation Proposes an unsupervised framework using contrastive learning and clustering to induce, name, and annotate event types from text. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an unsupervised learning method, not a benchmark or evaluation framework. | |
| One Model to Loop Them All: Elastic Latent Recurrence for Language Models Presents a training method for latent recurrent language models that avoids performance penalties at single loops while scaling with compute. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces an architecture rather than a benchmarking or evaluation methodology. | |
| U4RASD at KnowledgeGraphEval-2026: Ontology-Constrained Encoder and LLM Systems for Arabic Relation Extraction Describes an ontology-constrained ensemble system of encoders and fine-tuned language models for directed Arabic relation extraction. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Describes a competition submission system rather than proposing a new evaluation methodology or benchmark. | |
| Large Language Models, Machine-Assisted Cultural Analytics, and the Agentic Turn Examines methodological and conceptual implications of using large language models and agentic workflows for machine-assisted cultural analytics. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Discusses humanities methodology rather than proposing technical LLM evaluation benchmarks. | |
| Time Delay System for Sequence Modeling Proposes a sequence modeling framework using dynamic nonlinear time delays based on semantic similarity between current and past inputs. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a sequence modeling method rather than an evaluation benchmark. | |
| Temporal Evidence Forests for Long-Horizon Multi-Party Conversational Memory Presents a temporal memory architecture using hierarchical tree structures to trace evolving multi-party conversational histories for agents. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a memory framework rather than an evaluation benchmark. | |
| Rethinking the Roles of Diffusion and Decoding: Predictive-State Flows for Parallel Language Generation Proposes separating continuous diffusion language generation into high-level flow-generated predictive states and block-level autoregressive token decoding. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a new generation model architecture, not an evaluation benchmark. | |
| SEEDS: Preview-Guided Evidence Admission for Search Agents Introduces a preview-guided admission framework allowing search agents to selectively incorporate retrieved candidates into the ongoing reasoning context. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents a framework for selective search ingestion, not an evaluation benchmark. | |
| Language Models Have Interpretable Narrative Mechanisms Identifies internal attention heads, neurons, and sparse autoencoder features that causally influence narrative representation in language models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Investigates mechanistic interpretability rather than providing a benchmark or evaluation method. | |
| Evo$^2$: Evolving the Evolver through Weight-Harness Co-Evolution Proposes a co-evolution framework that coordinates agent harness prompt modifications with model weight updates using self-generated experience. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an agent training framework, not an evaluation benchmark. | |
| RAMP: Recurrent Adaptive Mixed Precision for Efficient Prefill in Recurrent-Depth Language Models Analyzes quantization error propagation in recurrent-depth language models and proposes adaptive mixed-precision schedules for faster prefill inference. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an inference speedup technique, not a benchmark. | |
| CliQ-RRG: Clinical-Knowledge Guided Disease-aware Visual-Textual Alignment for QA-Style Radiology Report Generation Presents a two-stage visual-textual alignment and retrieval-augmented framework for generating structured radiology reports from medical scans. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a domain-specific model framework rather than an evaluation benchmark. | |
| KV-Repurpose: A Testbed for Evaluating Non-Prefix KV-Cache Reuse Develops an open-source testbed to evaluate segment-wise key-value cache reuse beyond exact prompt prefixes during language model serving. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a serving system testbed for cache hits, not a standard LLM evaluation benchmark. | |
| From 2.1M Hours of Raw Broadcast Audio to Finnish Speech SSL: Confidence Filtering and Architectural Analysis at Scale Evaluates speech self-supervised learning models on Finnish broadcast audio using ASR confidence filtering and controlled architectural comparisons. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents an audio pretraining filtering method, not an LLM benchmark. | |
| BenCoDiC: A Large-Scale Contextual Cyberbullying Corpus for Bengali Political News Discourse FULL VERSION Presents a multi-dimensional annotated corpus of Bengali political YouTube discussions to support research on contextual cyberbullying detection. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces an annotated dataset for cyberbullying detection, not general LLM benchmarking. | |
| Neurograf v1: An End-to-End Romanian-First Language Model Family Introduces an end-to-end open pipeline, pretraining corpus, and family of decoder-only models trained from scratch for Romanian. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces a model family and dataset rather than a novel LLM evaluation methodology. | |
| Les Misérables Dialogue: A film-based network representation of social communication Constructs a temporal and multilayer social dialogue network dataset based on the film adaptation of Les Misérables. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents a social network dataset from a movie, not an LLM benchmark. | |
| SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents Proposes a reinforcement learning framework that tracks external versus internal knowledge sources to reduce search agent fabrication. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an RL training framework rather than an evaluation benchmark. | |
| Scaling Agentic Data for Long-Horizon Terminal Intelligence Introduces a synthetic data generation framework to produce long-horizon task trajectories for terminal agents. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a data synthesis engine for training, not an evaluation benchmark. | |
| PreSTO: Predictive Subtree Prefetching for Fast LLM Power Sampling Presents a predictive subtree prefetching method to accelerate power sampling for test-time language model reasoning. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an accelerated sampling method, not an evaluation benchmark or diagnostic. | |
| S$^3$R: Fast and Faithful RL Rollouts via Self-Sparse Speculative Decoding Introduces speculative decoding using self-sparse attention to speed up rollout generation during reinforcement learning. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces a decoding method for RL, not an evaluation framework or benchmark. | |
| SharP: Soft and Hard Prompt-Guided Augmentation with LLM for Low Resource Fake News Detection Proposes a prompt-guided text augmentation framework to assist fake news detection in low-resource settings. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a data augmentation method rather than a novel evaluation methodology. | |
| Vision Transformers Learn Word Meaning from Images Alone Tests whether self-supervised vision transformers represent semantic meanings of rendered words without paired text data. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Studies representational semantics rather than LLM benchmarking methodology. | |
| CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation Proposes a chunk-level credit assignment method for on-policy distillation across language models with different tokenizers. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents a distillation method rather than an evaluation benchmark. | |
| Beyond the Final Embedding: Retrieving Memories from Token-Level Evidence Introduces a retrieval method combining token-level matching and document-level embeddings for memory-augmented language models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents a retrieval method rather than a novel LLM evaluation benchmark. | |
| Host-Aware Web Crawling for LLM Pretraining Data Studies host-level budget allocation and crawling policies to collect pretraining web data for language models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Studies data collection strategies rather than LLM benchmarking or evaluation. | |
| Taking Tokenization off the Host for Agentic LLM Serving Develops a GPU-accelerated tokenizer using CUDA graphs to reduce time-to-first-token in agentic language model serving. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Does not propose evaluation frameworks or benchmarks. | |
| Exploring Attention Sinks in Text-to-Image Transformers from Diffusion and Autoregressive Models Analyzes the emergence and behavior of attention sinks in text-to-image diffusion and autoregressive Transformers. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Investigates attention mechanisms rather than LLM benchmarking techniques. | |
| Test-Time Memory Construction Presents a framework that defers memory construction to test time to answer queries across long interaction histories. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents a memory retrieval architecture rather than LLM benchmarking methods. | |
| Overcoming Data Imbalances with Expert Selection Biases Proposes category-conditioned expert routing biases to mitigate data imbalance in mixture-of-experts language models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an architectural routing bias rather than evaluation methodologies. | |
| MoPrO-SD: Modular Prompt Optimization for Stance Detection Presents a modular prompt optimization framework and benchmark targeting stance detection in text. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes a prompt optimization method rather than general benchmark evaluation frameworks. | |
| Towards the Roof of the World: NLP Progress in Tibetan Languages Surveys twenty-two years of Tibetan natural language processing literature across corpora, speech processing, and language models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents a broad literature review across Tibetan NLP rather than LLM evaluation methods. | |
| Does Financial Speech Need Continual Learning Regularization? A Study of EWC and mini-KFAC on Earnings Call ASR Investigates continual learning regularization techniques for speech recognition during domain adaptation on financial earnings calls. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Studies continual learning regularization techniques rather than LLM benchmark evaluation. | |
| Oído: Open-Vocabulary Speech Recognition on a $5 Microcontroller without a Neural Accelerator Implements open-vocabulary speech recognition on a low-cost ESP32-S3 microcontroller without dedicated neural accelerators. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Presents an on-device inference system rather than LLM evaluation methodology. | |
| Towards Enriching Design-Time Models with Evidence from Simulation and Operational Activities Proposes a workflow to enrich design-time cyber-physical system models using empirical evidence collected from simulations and operational runs. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Pertains to system engineering simulation rather than LLM evaluation benchmarks. | |
| Proactive Agentic Orchestration Introduces POrchestra, an agentic orchestration framework enabling sub-agents to proactively communicate with the orchestrator during dynamic execution. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes an orchestration method, not a novel evaluation methodology or benchmark. | |
| S-Space: Exploring Spatial Workspace in Multimodal Language Models The paper identifies a linear spatial workspace subspace in multimodal language models that encodes continuous 3D coordinate representations. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Investigates mechanistic spatial representations rather than proposing new benchmark suites. | |
| zip2zip++: Towards Inference-Time Token Compression with Minimal Quality Loss The paper proposes compression-layer modifications to reduce accuracy loss when using dynamic LZW vocabularies during language model inference. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Proposes token compression methods rather than novel benchmark suites or evaluation protocols. | |
| Multi-Stream Attention Residuals The paper extends attention residuals across model depth by introducing multi-stream depth attention with causal convolutions. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Introduces a model architecture rather than an evaluation framework or benchmark suite. | |
| Retractions Revisited: A Replication, Extension, and Mechanistic Evaluation of Linguistic Obfuscation Effects in Fraudulent Science The paper evaluates linguistic obfuscation patterns and reality-monitoring mechanisms across fraudulent and genuine scientific publications. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Examines psychological linguistics in fraudulent papers rather than LLM evaluation benchmarks. | |
| Verifiable Retrieval for Legal Reasoning through Structure-driven Knowledge Graph The paper evaluates knowledge-graph-informed chunking and retrieval strategies over German legal code to support verifiable retrieval-augmented generation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | Not relevant Develops a KG-guided RAG approach rather than an LLM evaluation benchmark. |
3,061–3,120 of 3,345← PreviousNext →