Every paper checked — nothing hidden
All papers
Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.
All3,345In brief3More (on-topic)587Not relevant2,721Not assessed24Older45Could not confirm25Blogs with no paper54All blogs57
| Paper | Topic | Relevance | Feedback |
|---|---|---|---|
| MEmoS2S-Bench: An Open-Ended Benchmark for ... Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via openreview · 14 Oct, 06:18 | Waiting for check | |
| WORLD DISTILLATION FOR SCAL Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via openreview · 14 Oct, 06:18 | Waiting for check | |
| EARTHARENA: BENCHMARKING AGENTS ON REAL Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via openreview · 14 Oct, 06:18 | Waiting for check | |
| Unit Test Generation using Generative AI : A Comparative Performance Analysis of Autogeneration Tools This paper compares the quality and coverage of unit test cases generated by ChatGPT against an automated generation tool. Peer-reviewedOlder paper (2023) Older paper (2023) — discussed this week | LLM evaluation and benchmarks via acm · 14 Oct, 06:18 | Not relevant Examines software testing generation rather than general LLM evaluation benchmarks. | |
| MAPS: Multi-Fidelity AI-Augmented Photonic Simulation and Inverse Design Infrastructure This paper introduces an open-source infrastructure and benchmark for multi-fidelity AI-assisted photonic simulation and inverse design. Peer-reviewedOlder paper (2025) Older paper (2025) — discussed this week | LLM evaluation and benchmarks via ieee · 14 Oct, 06:18 | Not relevant Focuses on photonic design benchmarks rather than general LLM evaluation. | |
| Defeating Prompt Injections by Design The paper introduces a system layer that separates control flow from untrusted data to protect language model agents against prompt injection attacks. Peer-reviewedOlder paper (2025) Older paper (2025) — discussed this week | LLM evaluation and benchmarks via ieee · 13 Oct, 06:19 | Not relevant Focuses on prompt injection defense architecture rather than new evaluation benchmarks. | |
| Debate, Deliberate, Decide (D3): A Cost-Aware Adversarial Framework for Reliable and Interpretable LLM Evaluation - ACL Anthology Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via acl · 13 Oct, 06:18 | Waiting for check | |
| QUANTITATIVEFINANCE-BENCH: BENCHMARKING AI ... Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via openreview · 12 Oct, 18:41 | Waiting for check | |
| PARALLAX:ATWO-PHASE DEFENSE FOR MEMORY- ... Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via openreview · 12 Oct, 18:41 | Waiting for check | |
| MLP-BENCH: TOWARDS END-TO-END FORMAL ... Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via openreview · 12 Oct, 18:41 | Waiting for check | |
| TRUST, BUT VERIFY: A FRONTIER MODEL BENCH- ... Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via openreview · 12 Oct, 18:41 | Waiting for check | |
| Proceedings of the 1st International Workshop on Harness Engineering for Generative UI: Everything Evolves, Something Endures The publication compiles the proceedings of the first international workshop on harness engineering for generative user interfaces. Not a research paperPeer-reviewed | LLM evaluation and benchmarks via acm · 12 Oct, 18:41 | Not relevant Summary is too thin to evaluate. | |
| SELECTION IS NOT LEARNING: CHOOSING MODEL ... Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via openreview · 12 Oct, 18:41 | Waiting for check | |
| ADAPTIVE SELF-DISTILLATION FOR LLM REASON- ING Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via openreview · 12 Oct, 18:41 | Waiting for check | |
| LLM-Based Pairwise Judgment for Math Item Parameter ... Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via acl · 12 Oct, 18:08 | Waiting for check | |
| How Long Reasoning Chains Influence LLMs’ Judgment of Answer Factuality - ACL Anthology Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via acl · 12 Oct, 18:08 | Waiting for check | |
| All Prompts Are Created Equal? Evaluating Robustness of LLM Judges Against Non-Adversarial Prompt Variations - ACL Anthology Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via acl · 12 Oct, 18:08 | Waiting for check | |
| Prior Beliefs Prejudice LLM-as-Judge: Evidence from Persuasion Evaluation - ACL Anthology Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via acl · 12 Oct, 18:08 | Waiting for check | |
| Can LLMs Judge Pedagogy? Assessing Conversational AI ... Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via acl · 12 Oct, 18:08 | Waiting for check | |
| HOW LLMS ADAPT TO HIDDEN EVALUATION RULES Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via openreview · 12 Oct, 18:08 | Waiting for check | |
| Aggregate vs. Personalized Judges in Business Idea Evaluation: Evidence from Expert Disagreement - ACL Anthology Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via acl · 12 Oct, 18:08 | Waiting for check | |
| PROBE: PROcess-Based BEnchmark for Hallucination Detection - ACL Anthology Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via acl · 12 Oct, 18:08 | Waiting for check | |
| TripTide: A Benchmark for Adaptive Travel Planning under Disruptions - ACL Anthology Summary pending (next check). Could not confirmPreprint | LLM evaluation and benchmarks via acl · 12 Oct, 18:08 | Waiting for check | |
| Proceedings of the 1st International Workshop on PromptOps and Vibe Coding The paper outlines a vision for trace-guided automated program repair to address context bloat and reasoning limitations. Peer-reviewed | LLM evaluation and benchmarks via acm · 12 Oct, 18:08 | Not relevant Focuses on automated program repair rather than LLM benchmarks or evaluation methods. | |
| Retrieval-Augmented Generation for Large Language Models: A Survey The paper reviews retrieval-augmented generation paradigms and examines frameworks across retrieval, generation, and modular components. PreprintOlder paper (2023) Older paper (2023) — discussed this week | LLM evaluation and benchmarks via missed · 12 Oct, 17:43 | Not relevant Surveys RAG methods generally rather than proposing LLM benchmarks. | |
| SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models The paper presents an 8-bit weight and activation post-training quantization technique that migrates activation outliers to weights. Peer-reviewedOlder paper (2022) Older paper (2022) — discussed this week | LLM evaluation and benchmarks via blog · 12 Oct, 16:38 | Not relevant Proposes an inference acceleration algorithm rather than benchmark creation or evaluation methodology. | |
| AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generation The paper introduces a post-training mixed-precision quantization method using GPU-compatible 2D weight tiles for efficient inference. PreprintOlder paper (2026) Older paper (2026) — discussed this week | LLM evaluation and benchmarks via blog · 12 Oct, 16:38 | Not relevant Presents a compression algorithm rather than benchmarking methodology or LLM evaluation frameworks. | |
| TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones The paper presents an open-source multimodal language model combining a small language backbone with vision encoders for efficient inference. PreprintOlder paper (2023) Older paper (2023) — discussed this week | LLM evaluation and benchmarks via blog · 12 Oct, 16:38 | Not relevant Presents a new model architecture rather than an evaluation framework or benchmark study. | |
| AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges The paper provides a conceptual taxonomy and review distinguishing task-specific AI agents from collaborative agentic AI systems. Peer-reviewedOlder paper (2025) Older paper (2025) — discussed this week | LLM evaluation and benchmarks via blog · 12 Oct, 16:38 | Not relevant Focuses on architectural and conceptual classification of agents rather than evaluation benchmarks or methodology. | |
| FadeMem: Biologically-Inspired Forgetting for Efficient Agent Memory Proposes an agent memory architecture that uses biologically inspired adaptive decay functions to selectively retain and forget context. Peer-reviewedOlder paper (2026) Older paper (2026) — discussed this week | LLM evaluation and benchmarks via ieee · 12 Oct, 16:30 | Not relevant Introduces a memory architecture rather than a new general evaluation benchmark or evaluation protocol. | |
| MoPD: Mixture-of-Prompts Distillation for Vision-Language Models Introduces a mixture-of-prompts distillation method to transfer knowledge from hard prompts to soft prompts, improving generalization on unseen classes. Peer-reviewedOlder paper (2024) Older paper (2024) — discussed this week | LLM evaluation and benchmarks via ieee · 12 Oct, 16:30 | Not relevant Presents a model adaptation technique rather than an evaluation benchmark or LLM-as-a-judge framework. | |
| SkillCraft: LLM Powered Interview Preparation Platform Presents a web platform using GPT-4 and Whisper to deliver interactive interview practice and automated feedback. Peer-reviewedOlder paper (2026) Older paper (2026) — discussed this week | LLM evaluation and benchmarks via ieee · 12 Oct, 16:30 | Not relevant Describes an end-user interview coaching app, not a methodology or benchmark for evaluating LLMs. | |
| Exploration of LLM Workload Reliability Based on di/dt Effects and Voltage Droops Profiles power oscillations and voltage droops in cloud GPUs caused by LLM inference workloads and introduces a stressmark framework. Peer-reviewedOlder paper (2026) Older paper (2026) — discussed this week | LLM evaluation and benchmarks via ieee · 12 Oct, 16:30 | Not relevant Investigates physical GPU hardware power delivery reliability and voltage droops, not LLM benchmark evaluation. | |
| CTC-Assisted LLM-Based Contextual ASR Uses coarse CTC decoding to filter relevant hotwords into LLM prompts for improved contextual speech recognition on rare words. Peer-reviewedOlder paper (2024) Older paper (2024) — discussed this week | LLM evaluation and benchmarks via ieee · 12 Oct, 16:30 | Not relevant Evaluates an ASR model on Librispeech, not a study of LLM benchmark methodology. | |
| LLM-Driven Badminton Action Evaluation Framework Based on Monocular 3D Pose Estimation Integrates monocular 3D pose estimation, Transformer-based stroke classification, and an LLM to evaluate badminton player postures and provide feedback. Peer-reviewedOlder paper (2026) Older paper (2026) — discussed this week | LLM evaluation and benchmarks via ieee · 12 Oct, 16:30 | Not relevant Evaluates human sports posture and movement, not the capabilities or reliability of LLMs. | |
| LLM-Simulated Users for Rapid Usability Evaluation of Decision Support Tools The paper studies using persona-conditioned synthetic LLM users to conduct usability evaluations of complex decision support tools. Peer-reviewedOlder paper (2026) Older paper (2026) — discussed this week | LLM evaluation and benchmarks via ieee · 12 Oct, 16:30 | Not relevant Focuses on evaluating software usability via synthetic persona simulation, not benchmarking LLMs. | |
| SAEC: Scene-Aware Enhanced Edge-Cloud Collaborative Industrial Vision Inspection with Multimodal LLM The paper proposes an edge-cloud collaborative inspection framework using multimodal LLMs dynamically scheduled based on scene complexity. Peer-reviewedOlder paper (2025) Older paper (2025) — discussed this week | LLM evaluation and benchmarks via ieee · 12 Oct, 16:29 | Not relevant Focuses on industrial inspection deployment rather than creating new benchmarks or LLM evaluation methodology. | |
| What is the goal of unsupervised machine learning? Argues conceptually that unsupervised machine learning serves four distinct goals rather than a single unifying objective. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Conceptual position paper on unsupervised learning goals, not about evaluating or benchmarking LLMs. | |
| Does an Illumination Prior Help Face-Swap Detection? A Controlled Study of Temporal Self-Blended Images Evaluates whether temporal illumination inconsistency priors improve face-swap deepfake detection performance across diverse video datasets. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Evaluates training techniques for face-swap detectors, not language model evaluation or general AI benchmarks. | |
| Addressing Overcommitment in the Reasoning of Gendered Economic Memes under Multimodal Ambiguity Proposes a multimodal reasoning framework to avoid biased role attributions in underspecified or ambiguous social memes. Peer-reviewed | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Presents a meme reasoning framework and dataset, not a general LLM evaluation methodology or benchmark study. | |
| Spectral Weight Decay: Inducing Low-Rank Structure in Neural Network Weights Introduces a spectral weight decay update inducing low-rank weight matrices to improve SVD-based post-training model compression. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Introduces a weight decay regularizer, not an LLM benchmark, leaderboard, or evaluation framework. | |
| Correlational Training of Morphological Neural Networks Presents a correlation-based multiplicative weight update method for training morphological neural network layers without standard gradient backpropagation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Presents an alternative update rule for morphological layers, not an evaluation framework or LLM benchmark. | |
| TraceRelay: Attention-Aligned Recurrence over Rolling Traces Introduces an attention-aligned recurrent architecture that maintains persistent state representations across sequences using rolling low-dimensional traces. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Proposes a recurrent neural architecture, not an evaluation framework or benchmark for language models. | |
| The Ball and the Box: Two Geometries of Computation in Superposition Derives mathematical dimension thresholds required to compute Boolean gates using neural representations stored in superposition. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Theoretical analysis of Boolean computation in superposition, not LLM evaluation benchmarks or evaluation protocols. | |
| SR-TTA: Spatial-Redundancy Test-Time Adaptation for Interference-Robust Respiration Sensing Proposes label-free test-time adaptation using spatial redundancy to improve respiration sensing on massive MIMO base stations against interference. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Presents wireless signal processing for vital sign sensing, not LLM evaluation or benchmarking methods. | |
| Beyond QAOA: A Review of AI and Quantum Computing for Adaptive Combinatorial Optimization Reviews literature on adaptive combinatorial optimization combining machine learning policies with quantum computing workflows. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Reviews quantum computing and combinatorial optimization, not LLM evaluation or benchmark methodologies. | |
| RAGenome: Scaling Retrieval-Based Genomic Language Models to Long Contexts Presents a retrieval-augmented genomic language model scaling pretraining context length to capture long-range biological sequence interactions. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Introduces a retrieval-based genomic model, not an LLM evaluation dataset, benchmark, or evaluation methodology. | |
| Optimal random quantisers for spherically symmetric distributions Analyzes optimal random quantisers for high-dimensional spherically symmetric distributions to reduce distortion with moderate sample sizes. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Focuses on mathematical quantization theory for probability distributions, not LLM benchmarks or evaluation methods. | |
| In-Ride Alcohol-Impairment Detection in E-Scooterists with False-Alarm Control Develops an onboard inertial and throttle sensor detector with false-alarm controls to identify alcohol-impaired e-scooter riders during trips. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Presents a sensor-based classifier for e-scooters, not a benchmark or framework for LLM evaluation. | |
| Compile the Table: Query-Calibrated Operator Compression for Tabular In-Context Learning Compresses tabular in-context learning key-value caches into query-calibrated prototypes to improve throughput without sacrificing accuracy. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Proposes a KV cache compression algorithm for tabular learning, not an LLM evaluation benchmark. | |
| Softmax Attention on Gaussian Mixtures: Linear When It Can, Selective When It Must Analyzes the infinite-prompt limit of softmax attention on Gaussian mixtures to explain when attention behaves linearly or selectively. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Theoretical analysis of attention mechanisms on Gaussian mixtures, not LLM benchmark evaluation or testing. | |
| PRAXIS: Learning Dynamics of Self-Improving Models with Symbolic Archives Models self-improving systems with symbolic archives as dynamical processes, establishing theoretical optimization bounds and empirical stability. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Studies optimization dynamics in self-improving systems, not general LLM evaluation frameworks or benchmark integrity. | |
| Self-Supervised Speech Representations for Cross-Speaker Dysarthria Detection During Awake Craniotomy Evaluates a pipeline combining speaker diarization and self-supervised speech representations to detect dysarthria during awake craniotomy procedures. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Evaluates a specialized acoustic clinical pipeline, not general LLM evaluation benchmarks or evaluation methodologies. | |
| Recovery Guarantees for Posterior Sampling of One-Bit Compressed Sensing Establishes theoretical sample complexity bounds for posterior sampling in noisy one-bit compressed sensing under prior distribution mismatch. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Theoretical study on sample complexity in compressed sensing, not LLM evaluation or benchmark design. | |
| Structural and Representational Agency in LLM-Driven Timbre Generation and Interpolation Examines the disparity between reachability and self-reported parameters in an LLM-driven audio interface for timbre generation. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Analyzes user interfaces and synthesizer parameter accuracy in audio generation, not general LLM benchmarking methods. | |
| DADP: Dynamic Activity-Dependent Pruning, A Reverse Hebbian-Inspired Structural Pruning Method Proposes a dynamic structural pruning method using biologically inspired plasticity to reduce neural network parameters during training. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Proposes a neural network pruning technique rather than an LLM evaluation benchmark or evaluation framework. | |
| Conditional Kernel Stein Discrepancy Develops a kernel Stein discrepancy framework using operator-valued kernels to assess conditional goodness-of-fit for unnormalized models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Presents statistical goodness-of-fit testing theory using kernel Stein discrepancies, not LLM benchmarking or evaluation methodology. | |
| Understanding Latent-Dimension Scaling in Dynamical-System Learning through Spectral Reliability Analyzes how latent-dimension scaling impacts dynamical-system learning and rollout error using Koopman operator spectral reliability metrics. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Focuses on theoretical dynamical systems and Koopman operator spectral analysis, not LLM evaluation benchmarks. | |
| Learning structured linear dynamical systems from missing observations Establishes non-asymptotic statistical error bounds and optimization convergence for learning structured linear dynamical systems from highly subsampled observations. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Focuses on linear dynamical systems with missing observations, not LLM evaluation. | |
| Automated Assembly Instruction Generation from CAD Models Using Grounded Large Language Models: A Human-in-the-Loop Framework Builds a human-in-the-loop framework using grounded language models to automatically generate natural-language assembly instructions from CAD models. Preprint | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | Not relevant Presents an instruction generation framework, not an LLM benchmark. |
2,761–2,820 of 3,345← PreviousNext →