Every paper checked — nothing hidden
All papers
Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.
All3,345In brief3More (on-topic)587Not relevant2,721Not assessed24Older45Could not confirm25Blogs with no paper54All blogs57
| Paper | Topic | Relevance | Feedback |
|---|---|---|---|
| Efficient Agnostic Learning of Univariate Densities Under Local Differential Privacy Proves theoretical sample bounds and presents algorithms for learning univariate probability distributions under local differential privacy. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Theoretical statistical work on density estimation, not language model evaluation. | |
| Clean Data Can Still Carry Backdoors: Support-Persistent Backdoors for Model Reuse Introduces a clean-data backdoor method that persists through standard knowledge distillation by targeting naturally occurring feature subspaces. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on creating backdoor attacks resistant to distillation, not on LLM evaluation benchmarks. | |
| Federated Art Chase: Privacy Leakage and Data Poisoning in Federated Fine-Tuning Studies label privacy leakage and data poisoning risks during head-only federated fine-tuning of vision models across distributed clients. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies FL privacy attacks rather than LLM evaluation benchmarks. | |
| CascadeLedger: Counterfactual Auditing of Information and Resource Integrity in LLM Cascades Introduces an auditing framework that checks counterfactual execution traces to ensure language model routing uses only legitimate information. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Audits cascade routing integrity rather than standard LLM benchmark evaluation. | |
| TurboMamba: Faster Inference, Faster Training, and Better Prediction with Lexical Memory Proposes a hybrid architecture combining global state-space modeling with local transformer attention and lexical memory for efficient inference. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on architecture efficiency rather than LLM benchmarking methodologies. | |
| PRISM: Transfer and Limits of Activation Interventions in Tool-Calling Language Models Studies the transferability of internal activation interventions to reduce unsafe tool execution across model precisions and languages. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an alignment framework rather than general LLM evaluation benchmarks. | |
| Gibbs-Gap Policy Optimization for Off-Policy RL with Verifiable Rewards Introduces an off-policy reinforcement learning algorithm for language models using verifiable rewards without importance sampling weights. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a training algorithm rather than an LLM evaluation benchmark. | |
| Does Morphological Correctness Matter for Tokenization? A Scrambled Control Study in Tagalog Tests whether linguistically accurate morpheme boundaries in tokenizers improve translation performance compared to scrambled control segmentations. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on linguistic tokenization, not LLM benchmark evaluation. | |
| CogDrift-4D: Hierarchical Multimodal Modeling of Temporal Behavioral Decay for MCI Detection Proposes a hierarchical multimodal transformer to model long-term behavioral changes across multi-session interactions for cognitive impairment detection. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Does not focus on general LLM evaluation or benchmarks. | |
| Task-Centric Multimodal Learning for Cognitive Impairment Detection Evaluates a multimodal screening method using voice, video, and text from voice-assistant interactions to detect cognitive impairment. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Not focused on general LLM evaluation benchmarks or methodologies. | |
| Prune the Summary, Not the History: Source-Judged Compaction for Long-Horizon Agents Introduces a method to filter unsupported compressor inferences from long-horizon agent summaries to avoid compounding execution errors. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a compaction technique rather than an LLM evaluation benchmark. | |
| Learning When to Think: Utility-Guided Adaptive Reasoning for Efficient Text Retrieval Proposes an adaptive reasoning framework that supervises intermediate retrieval states and stops reasoning steps dynamically to cut latency. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a retrieval method rather than a general evaluation benchmark. | |
| SHIP: Structured and Hierarchical Insight with Privacy The paper proposes SHIP, an architecture for extracting structured differentially private representations from text for analytical queries. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a data synthesis method rather than an LLM evaluation benchmark. | |
| Multimodal AI Detection for Math-Heavy Scientific Text The paper examines AI-generated text detection in scientific papers by combining visual formatting, PDF text, and LaTeX representations. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Investigates AI-generated content detectors rather than broad LLM capability benchmarks. | |
| Compression Does Not End at Selection: Post-Selection Positional Geometry for Sparse KV Caches The paper proposes RoPERelay to reorganize positional geometry and rotary embeddings after sparse KV cache token eviction. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents an inference optimization technique rather than an evaluation benchmark. | |
| From Observation to Intervention: Distilling Self-Generated In-Context Demonstrations via Executable Causal Intervention The paper introduces a causal intervention framework to distill transferable self-generated in-context learning demonstrations. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on prompt distillation methods rather than LLM benchmarking standards. | |
| Stop Training Against Soft Users: Mentalized Adversarial Self-Play for Proactive Dialogue The paper introduces an adversarial self-play framework modeling user mental states to improve proactive dialogue agent planning. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a self-play training framework rather than an evaluation benchmark. | |
| Automatic Sentence Completion Using Bengali Audio Speech with a Transformer-Based Early Speech Recognition System The paper presents an automated speech recognition and sentence completion system to assist dementia patients speaking Bengali. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an assistive application rather than LLM benchmarking methodologies. | |
| From Long Context to Selective Memory:A Survey of Memory-Centric Compression for Large Language Models The paper surveys memory-centric compression techniques for managing inference resources and context windows in large language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant A survey of memory compression techniques rather than an evaluation benchmark. | |
| FGSS-BO: Feedback-Guided Hypothesis Management for LLM-Based Bayesian Optimization The paper proposes FGSS-BO, adding structured hypothesis management to language-model-guided Bayesian optimization. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes an optimization method rather than a general LLM evaluation benchmark. | |
| LLM-Based Weak Annotation for Cybersecurity Tweet Classification: A Teacher-Student Framework The paper presents a teacher-student framework using language models for weak annotation to classify cybersecurity tweets. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Deals with dataset annotation rather than standardized LLM evaluation frameworks. | |
| TimeCap: A Scalable Open-Source Pipeline for Captioning Wearable Health Time-Series Data The paper introduces TimeCap, an open deterministic pipeline for generating signal-grounded captions from wearable health time-series data. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a dataset creation pipeline rather than an LLM evaluation benchmark. | |
| Making every stage of medical device document assessment with large language models deterministic The paper evaluates output non-determinism in commercial language model deployments during medical device regulatory compliance assessments. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Addresses production pipeline determinism rather than standard LLM benchmark evaluations. | |
| PocketTSLM: Deploying an On-Device Soft-Prompt Time-Series Language Model for Privacy-Preserving Smartphone Reasoning The paper presents PocketTSLM, deploying an on-device soft-prompt time-series language model on smartphones for personal health data. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents an on-device deployment system rather than an LLM evaluation benchmark. | |
| ZoomCap: Evidence-Routed Visual Zooming for Recoverable Visual Information Transfer The paper introduces ZoomCap, an adaptive visual zooming and evidence-routing framework to improve fine-grained image captioning supervision. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces FactCycle for caption scoring, but primary focus is multimodal caption generation. | |
| Hybrid Generative-Tagging Decoding for Efficient Product Attribute-Value Extraction The paper introduces a hybrid generative-tagging architecture to improve efficiency and coverage in product attribute and value extraction. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates extraction accuracy rather than general LLM evaluation methodologies. | |
| Ranking and Partitioning of Attention Heads for Sparse and Continual Reinforcement Learning Fine-tuning The paper studies ranking and partitioning attention heads by gradient norm for sparse and continual reinforcement learning fine-tuning. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies continual RL training methods rather than LLM benchmark designs. | |
| OpCredit: Operation-Aware Policy Learning for Agentic Knowledge Graph Question Answering The paper proposes OpCredit, a policy learning framework that assigns individual operation credits for agentic knowledge graph question answering. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates question answering methods rather than proposing new evaluation benchmarks. | |
| Emotion Is Not External Knowledge: Internal Attention Signals for Affective State Reasoning The paper presents an internal attention boosting method to improve how language models infer implicit conversational affective states. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on attention boosting techniques rather than LLM evaluation methodology. | |
| RenoLayout: Unified Floorplan Renovation and Furniture Layout for Constraint-Aware Residential Design The paper introduces RenoLayout to generate residential renovation floorplans and furniture layouts under structural and physical constraints. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a floorplan generation framework rather than an LLM evaluation benchmark. | |
| DPA-MoE: A Runtime Precision Control Framework for MoE Inference Introduces a dynamic precision control framework that allocates weight chunks to mixture-of-experts models during inference. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates quantization precision rather than general LLM benchmarking methodologies. | |
| Calibration Language Shapes Block-Influence Pruning in Multilingual Language Models Analyzes how calibration language choices affect block-influence depth pruning rankings in multilingual language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Analyzes pruning calibration transfer rather than LLM benchmarking or evaluation frameworks. | |
| Beyond Invariance: Factor-wise Selective Treatment of Shifts for Multimodal Learning Proposes a factor-wise representation framework to selectively treat different distribution shifts in multimodal machine learning. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies robust representation learning rather than benchmarking methodologies. | |
| Identifying Effective Strategies in Prompt Evolution for Stress and Emotion Prediction Evaluates prompt optimization methods and identifies effective prompting strategies for social media stress and emotion classification. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies prompt tuning for emotion detection rather than general LLM evaluation frameworks. | |
| HyCo: Hybrid Drafting for Cost-Efficient Mobile-Cloud Speculative Decoding Introduces a hybrid speculative decoding framework splitting draft tasks between mobile neural processing units and cloud servers. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates inference performance on benchmarks rather than benchmarking methodologies. | |
| Translation Circuit Discovery and Targeted Optimization for Low-Resource Multilingual Translation Identifies translation circuits in multilingual models and optimizes fewer than five percent of parameters for low-resource translation. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on translation optimization rather than LLM evaluation methodology or benchmark flaws. | |
| OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents Uses multi-scope reminders and on-policy distillation to help coding agents maintain constraints over long software engineering interactions. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Uses coding benchmarks for evaluation rather than proposing new LLM benchmark methodologies. | |
| Writability and the Pathway: A Controlled Study of Knowledge Injection into Addressed Memory Conducts a controlled evaluation of direct knowledge injection into addressed conditional memory modules across small and production models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Evaluates parametric memory writing techniques rather than general LLM benchmark reliability. | |
| Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation Gates reinforcement learning policy-shift distillation using intermediate checkpoints to ensure student models only learn consistent teacher shifts. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a distillation algorithm rather than an evaluation benchmark. | |
| ContextLift: Breaking the Spurious Correlation Behind Conditional Emergent Misalignment Introduces a data augmentation method that breaks spurious contextual correlations causing emergent misalignment during domain finetuning. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on fine-tuning alignment defense rather than evaluation benchmarking. | |
| Spurious Attention Heads in Vision Transformers: Mechanistic Identification of Annotation-Driven Bias in Medical Chest X-Ray Analysis Identifies a specific vision transformer attention head that spuriously attends to medical image corner text instead of lung pathology. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on mechanistic interpretability rather than LLM benchmarking. | |
| Enhancing Multimodal RLVR with Pairwise Token Contrast and Adaptive Trajectory Confidence Improves reinforcement learning credit assignment in multimodal reasoning using pairwise token contrast and adaptive trajectory confidence. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Develops an RLVR training technique rather than benchmarking methodology. | |
| Diffusion Language Models as Open Quantum Systems Frames diffusion language models as open quantum systems to enable text generation conditioned on superpositions of prompts and thoughts. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a novel text generation model, not an evaluation benchmark. | |
| SceneWeave: Audience-Guided Scene Exploration for Dramatic Adaptation Introduces a system that generates and compares alternative scene drafts to guide full dramatic script adaptations from prose. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Develops a creative writing system rather than an LLM evaluation benchmark. | |
| Bias Mitigation in Audio-Enabled LLMs through Adversarial Training and Distillation Methods Applies adversarial training and distillation to remove unwanted gender bias in audio-enabled language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on bias mitigation techniques rather than evaluation benchmarks. | |
| Content associations with answer selection and engagement in supply chain community question answering Analyzes how specific nouns and semantic clusters relate to answer selection and user engagement in logistics question-answering forums. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Analyzes human forum dynamics rather than evaluating language models. | |
| N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models Proposes a compressed representation using Gaussian trellises to quantize conditional memory lookup tables down to two bits. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a compression method rather than benchmark evaluation frameworks. | |
| Dynamic ordering in sequence generation in Transformers: experiments on low-resource Tagging scenarios Introduces a non-autoregressive transformer model that dynamically decides tag generation order at inference time for low-resource sequence labeling. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a decoding method rather than benchmarking or evaluation protocols. | |
| TEM: Temporal State Tracking Memory for Large Language Model Agents Presents a training-free framework that tracks changing conversational facts as versioned records to improve agent memory retrieval. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a memory system rather than addressing LLM benchmarking methodology. | |
| Learning Speech Attribute Representations via Multi-Teacher Distillation Distills multiple speech attributes from four teacher models into a single shared speech encoder. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents a representation distillation technique rather than a benchmarking framework. | |
| P2TPO: Structure-Aligned Page-to-Token Credit Assignment for Full-Page Document Parsing Proposes a reinforcement learning credit-assignment method for document parsing models to align rewards with structured page elements. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a training credit assignment method, not an evaluation framework. | |
| Documenting Trofinetide Treatment Phenotypes in Rett Syndrome with a Dual-LLM Consensus Pipeline Applies a dual language model consensus pipeline to extract clinical phenotypes from electronic health records for patients with Rett syndrome. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on a specific clinical application rather than general LLM evaluation. | |
| The Right Byte First: How UTF-16 Helps Multilingual Boundary Prediction Studies how using UTF-16 byte order improves boundary prediction and representation efficiency in multilingual byte-level language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Studies byte encoding schemes rather than model benchmarking. | |
| Patch where the text is new: a compression view of where byte-level models should spend compute Introduces a compression-based patching strategy for byte-level language models that allocates compute where upcoming bytes are unlikely to repeat. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on patch allocation methods rather than evaluation benchmarking. | |
| Unfragmenting Nawatl NMT: Single-Variant Specialization for Masewalkopa Specializes a translation model to a specific indigenous Nawatl variety using a curated dataset and finite-state morphological analysis. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Presents translation models rather than LLM evaluation benchmarks. | |
| Intern-Decision: Efficient Multimodal Decision Making with Autoregressive Masked Language Modeling Develops small multimodal models that output structured choices using masked language modeling to reduce inference latency in agent tasks. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a model architecture rather than an evaluation benchmark. | |
| CLUE: Uncovering Social Context for Pragmatic Understanding Develops a two-stage training approach combining evidence reasoning and reinforcement learning to improve pragmatic social reasoning in conversational models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Proposes a training pipeline rather than a standardized evaluation framework. | |
| Lexical Reference Frames: Interpreting LM Residual Streams in Native Semantics Introduces lexical reference frames to interpret intermediate hidden states in language models using the model's own remaining layers. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on mechanistic interpretability rather than evaluation benchmarks. | |
| EVA-Cap: Optimizing Audiovisual Video Captioning via Event-Centric Alignment Proposes an event-centric framework to align audio and video features when generating detailed video descriptions with language models. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Focuses on a captioning method rather than general benchmarking. | |
| Hy-Memory-30B-A3B: Attributed Reinforcement Learning for Long-Term Memory Writing Introduces a compact language model trained with distillation and reinforcement learning to manage long-term memory across multiple external frameworks. Preprint | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | Not relevant Introduces a memory model rather than an evaluation standard. |
781–840 of 3,345← PreviousNext →