Every paper checked — nothing hidden
All papers
Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.
All3,345In brief3More (on-topic)587Not relevant2,721Not assessed24Older45Could not confirm25Blogs with no paper54All blogs57
| Paper | Topic | Relevance | Feedback |
|---|---|---|---|
| Stronger Judges Do Not See More: Deployment-Integrity Blindness in LLM Evaluation Introduces a benchmark measuring how output-only model judges fail to detect tampered runtime environments in agent deployments. Ranked #61Surprising resultPreprint8/10 suggested | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | On-topic Introduces a benchmark evaluating how output-only LLM judges fail to detect runtime deployment tampering. | |
| When Judges Agree and Are Wrong: Consistency-Accuracy Dissociation as a Blind Spot for Label-Free Evaluation Examines failure modes in label-free evaluation where models reach high consistency across perturbations while generating identical incorrect answers. Ranked #62Surprising resultPreprint8/10 suggested | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | On-topic Critiques label-free evaluation and consistency-based scoring, uncovering blind spots where models agree incorrectly. | |
| INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems The paper introduces an end-to-end benchmark to evaluate indirect prompt injection risks across long-horizon agent workflows. Ranked #63New testPreprint8/10 suggested | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | On-topic Introduces a standardized benchmark of 40 tasks to evaluate agent vulnerabilities across frontier models. | |
| FDE-Gym: The Last Mile of AI Delivery The authors introduce FDE-Gym, a benchmark of 95 customer delivery tasks evaluating AI agents on ambiguous requirements and live software environments. Ranked #64New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | On-topic Introduces a benchmark evaluating language agents on real-world engineering delivery tasks. | |
| Thinking Inertia: LLMs Keep Thinking When Told Not To The paper defines evaluation metrics to show that language models continue generating reasoning traces even when instructed not to think. Ranked #65Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Proposes new metrics and evaluation frameworks to measure LLM thinking mode compliance. | |
| RoTA-Bench: Evaluating 3D Spatial Reasoning through Interactive Projection Alignment Introduces RoTA-Bench to evaluate vision-language models on 3D spatial reasoning through interactive object rotation to match target 2D silhouettes. Ranked #66Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces RoTA-Bench to evaluate vision-language models on 3D spatial reasoning. | |
| MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments The paper provides an open platform and environment suite for evaluating and training GUI agents with online reinforcement learning. Ranked #67New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | On-topic Presents a unified evaluation platform and benchmark suite for GUI agents. | |
| Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards The paper presents a dataset and framework with dense progressive rewards to train language agents on long-horizon tasks. Ranked #68New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Presents Tev-Gym, a new benchmark dataset for evaluating long-horizon agents. | |
| Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents The paper introduces a benchmark evaluating how well language models engineer isolated execution harnesses for coding agents using specification documents. Ranked #69New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | On-topic Introduces a two-stage benchmark measuring long-horizon harness engineering for agents. | |
| ProgramGym: Scaling Training Environments for Long-Horizon Program Reproduction Agents Introduces ProgramGym, a framework providing verifiable sandbox environments and black-box test feedback for training long-horizon program reproduction agents. Ranked #70New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Introduces ProgramBench and verifiable training environments to evaluate long-horizon coding agents. | |
| A Decade in a Day: Synthesizing Agent-Native Interfaces from Human-Oriented Software Evolution The paper introduces a benchmark measuring how software interface versions affect the task performance of desktop language model agents. Ranked #71New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces DeskIF-Bench to evaluate agent performance across evolving desktop software interfaces. | |
| LifeSearch: Benchmarking Deep Research Agents for Real-World Life and Productivity This paper introduces a benchmark with expert-written rubrics to evaluate deep research agents on real-world life and productivity tasks. Ranked #72New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces an extensive real-world benchmark to evaluate deep research agents. | |
| WOVEN: Weaving Visual World Modeling into Multimodal LLMs Presents a benchmark and dataset using generative video rollouts to train and evaluate visual transition reasoning in multimodal models. Ranked #73New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Introduces WOVEN, a benchmark evaluating multimodal models across scenes and reasoning types. | |
| DecisionRobust: Benchmarking Robustness for JEV Style Decision Models Across Text and Vision The paper introduces a benchmark evaluating the robustness of typed decision models against text, image, and prompt injection attacks. Ranked #74New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | On-topic Introduces DecisionRobust, a benchmark systematically evaluating decision model reliability under adversarial perturbations. | |
| JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs The paper presents a benchmark and harness to delegate routine tool choices from large language models to cheaper decision models. Ranked #75New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | On-topic Introduces AgentDecisionBench to evaluate decision-making performance in running agent workflows. | |
| $τ$-Emotion-Bench: Exposing the Interaction Fragility of LLM Agents through Real-World User Emotions The paper introduces a benchmark to measure how tool-calling agents perform when interacting with emotionally varied, impatient users. Ranked #76New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | On-topic Introduces tau-Emotion-Bench to evaluate agent reliability and interaction fragility under user variation. | |
| Diagnosing and Correcting Reward Hacking in Verifiable Instruction Following The paper analyzes reward hacking in verifiable instruction following and introduces a likelihood bonus method to maintain answer quality. Ranked #77Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | On-topic Analyzes evaluation blind spots in verifiable instruction-following benchmarks and metrics. | |
| ProbabilityBench: Model Evaluation and Calibration with Exact Ground-Truth Uncertainty The paper introduces ProbabilityBench to evaluate model uncertainty calibration against analytically computed, ground-truth instance-level probabilities across three text environments. Ranked #78New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Presents ProbabilityBench, a benchmark for evaluating model calibration and uncertainty estimation. | |
| What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds The paper presents a benchmark evaluating whether conversational agents agree on verifiable plans that satisfy both parties' private conditions. Ranked #79New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 15:06 | On-topic Introduces HICO-Bench to evaluate agent social coordination plans against verifiable outcomes. | |
| Rubrics from the Task, Verdicts from Evidence: Pointwise Rubric Rewards with Tool-Augmented Judges This paper proposes a rubric-based reward model that derives criteria strictly from instructions and evaluates responses pointwise using tool assistance. Ranked #80Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Investigates tool-augmented LLM judges and rubric-based evaluation methods for assessing responses. | |
| To Align or Not To Align: Check Your COMPASS Before You Train This paper introduces a metric to estimate the compatibility of pretrained models before running costly alignment or checkpoint merging. Ranked #81New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces COMPASS, a theoretically grounded diagnostic metric to evaluate model alignment compatibility. | |
| Beyond Perfect Teammates: Multi-Agent Collaboration with Unreliable Colleagues This paper introduces UnCoBench to benchmark how language model agents handle unhelpful, incomplete, or missing communications from partner agents. Ranked #82New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces UnCoBench to benchmark multi-agent systems facing incomplete or misaligned collaborator inputs. | |
| Looking Complete Is Not Being Grounded: Evaluating LLM Sitreps Against Human-Authored Situation Reports This paper evaluates language-model-generated disaster situation reports against human-written reports using a domain-specific sixteen-slot information schema. Ranked #83Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Assesses LLM-as-a-judge consistency and evaluation schemas against human expert baselines. | |
| Probable but Not Valid: A Psychometric Audit of Decision Models as Synthetic Survey Respondents This paper audits whether decision models and generative language models can serve as valid synthetic respondents for psychometric surveys. Ranked #84Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Conducts a psychometric audit benchmarking decision models against human survey responses. | |
| Judges Follow the Reference: The Derivation Gap in Verifying Tool-Call Arguments The authors identify a derivation gap where LLM judges reject correct tool arguments lacking direct textual references, and propose a dedicated reference model. Ranked #85Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Evaluates and improves LLM-as-a-judge reliability for tool argument verification. | |
| Does a Governed Confidence Score Know When a Text-to-SQL Answer Is Right? An Exploratory Study on BIRD and a Pre-registered Confirmation on Spider The paper audits an enterprise confidence scoring mechanism for text-to-SQL models on the BIRD and Spider benchmarks. Ranked #86Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Audits confidence scoring reliability on text-to-SQL benchmarks. | |
| Quality, Not Just Outcome: A Structural Scoring Framework for Selecting Coding-Agent Trajectories for Supervised Fine-Tuning Presents a structural scoring framework to select high-quality coding agent trajectories for supervised fine-tuning based on interaction efficiency. Ranked #87New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Proposes a structural trajectory quality scoring system without relying on LLM-as-a-judge. | |
| EvoCowork: A Benchmark for Real-World Office Tasks This paper introduces a 240-task benchmark to evaluate recursive self-improvement in agents across diverse real-world office workflows. Ranked #88New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Presents a benchmark and protocol for evaluating harness-level agent self-improvement. | |
| A Distributional Framework for Behavioral Characterization and Risk Management of Large Language Models Presents a Monte Carlo simulation framework to quantify distributional response shifts and model risks across language model lifecycles. Ranked #89New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | On-topic Proposes a Monte Carlo distributional framework to test and characterize stochastic LLM behaviors. | |
| When Charts Deceive Web Agents: A Paired Benchmark for Misleading Visualization-Grounded Tasks Introduces a benchmark evaluating how misleading visualizations and charts affect the multi-step execution and decisions of web agents. Ranked #90New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Presents MisVisAgentBench, an executable benchmark measuring agent task performance and failure modes. | |
| DataSense-Bench: The First Step Toward an AI Scientist The paper introduces DataSense-Bench to evaluate whether AI agents can select and rank effective training data subsets for fine-tuning. Ranked #91New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:29 | On-topic Proposes a new benchmark evaluating AI agents on selecting training data for fine-tuning. | |
| When Pleasing the User Compromises Care: AI Sycophancy in Caregiver-Mediated Healthcare The paper benchmarks how caregiver-directed language models compromise medical advice when users express clinically inferior care preferences. Ranked #92Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces an evidence-grounded benchmark to measure LLM sycophancy when facing harmful user preferences. | |
| AgentStripes: An Evidence-Graded Maturity Model for AI Agents The paper introduces a maturity model that grades artificial intelligence agents across multiple capability levels, evidence stages, and safety gates. Ranked #93New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Proposes AgentStripes, a formal maturity framework and grading methodology for AI agents. | |
| Rethinking the Role of Tool Graph Memory in Multi-Turn Tool Use The authors analyze tool graph memory in multi-turn agents and find that stored transitions provide little benefit over baselines. Ranked #94Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | On-topic Analyzes and re-evaluates tool use benchmarks across nine different models. | |
| Humans vs. Machines: A Comparison With AI Agents in Software Reverse Engineering This paper benchmarks language model agents against skilled humans on binary software reverse engineering tasks using interactive tools. Ranked #95Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Compares human analysts against multiple LLM models on standardized software engineering challenges. | |
| Unconstrained CTI-REALM with Open-Weights This paper evaluates open-weight language models against closed frontier models on a cybersecurity detection benchmark using an interactive telemetry environment. Ranked #96Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Analyzes benchmark constraints and evaluation artifacts in the CTI-REALM benchmark. | |
| When Does an ASR Error Matter? Task-Role Diagnostics for Tool-Using Voice Agents The paper introduces a diagnostic protocol to evaluate how speech recognition errors affect structured tool calling in voice agents. Ranked #97New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces SpeechEvalX, a diagnostic evaluation protocol for assessing tool-calling robustness to speech errors. | |
| The Curse of Knowledge in Role-Playing Language Models The paper tests whether role-playing language models leak private or unobserved character information across simulated interactive scenes. Ranked #98Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Constructs a benchmark using counterfactual scenes to evaluate knowledge leakage in role-play models. | |
| DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneck The paper proposes an agent framework that discovers lightweight proxy evaluators to bypass costly benchmark evaluations during autonomous engineering discovery. Ranked #99New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Focuses on overcoming the evaluation bottleneck using proxy evaluators and fidelity checks. | |
| Machine--human agreement in marking undergraduate assignments is specific to model, prompt and institution The paper examines machine grading on undergraduate essays across three universities, finding models compress score distributions and vary across institutions. Ranked #100Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Evaluates the reliability and cross-context validity of LLM judges marking assignments. | |
| Beyond Belief Nesting: Rethinking Higher-Order Theory of Mind in Large Language Models The paper introduces HopToM, a benchmark testing whether higher-order theory-of-mind tasks in language models genuinely evaluate nested reasoning rather than shortcuts. Ranked #101Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Uncovers flaws in higher-order Theory of Mind tests and introduces HopToM, a certified benchmark for perspective reasoning. | |
| Connecting the Dots: Enduring Performance Gaps in ASR Systems Between Standard and Minority English Varieties The paper tracks automatic speech recognition performance from 2021 to 2025, finding persistent accuracy disparities between standard American English and minority accents. Ranked #102Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Conducts a longitudinal benchmark evaluation revealing persistent accent disparities and benchmark limitations in ASR systems. | |
| Label Collapse in Diffusion Language Model In-Context Learning This paper analyzes label collapse failure modes across diffusion and autoregressive language models during in-context learning classification. Ranked #103Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Exposes an undetected failure mode in in-context learning evaluation metrics. | |
| Rule vs.Consequence: Dissociable Internal Representations of Moral Reasoning in LLMs This paper uses activation steering and probing to identify distinct internal representations for rule-based versus consequence-based moral reasoning in language models. Ranked #104New way to see a problemPeer-reviewed7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Evaluates LLM moral reasoning using a 2x2 factorial framing and probing framework. | |
| Strong yet Controllable LLM Abstention without Learning to Abstain This paper proposes TAPS, combining standard accuracy training with post-hoc confidence selection to give controllable abstention rates without specialized tuning. Ranked #105Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Analyzes metrics decomposing risk into answering and distinguishing abilities across QA benchmarks. | |
| RelLedger: Relational Information Governance for LLM Agent Organizations This paper introduces RelLedger, an auditing framework to trace how relational histories drive reporting choices and actions in multi-agent systems. Ranked #106New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Introduces an audit framework to evaluate information sharing behavior in agent organizations. | |
| Who Sets the Pace of Judge Hacking? Judge Capability versus the Policy-Judge Gap This work analyzes whether policy capability gaps or judge models dictate reward hacking when training models against language-model judges. Ranked #107Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Investigates judge hacking dynamics and the limits of LLM-as-a-judge. | |
| MemForm: Benchmarking Memory Formation Poisoning in LLM Agents The paper introduces MemForm to benchmark adversarial memory formation poisoning across five input interfaces in language model agents. Ranked #108New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Introduces a diagnostic benchmark for memory formation poisoning. | |
| Evaluating LLMs as Agentic Workflow Executors in Real-World Customer Support Skills The paper introduces a benchmark derived from real customer support dialogues to evaluate language models executing bounded, tool-using skills. Ranked #109New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Constructs a benchmark using multi-judge evaluation for agentic workflows. | |
| ToolBottleneckBench: Measuring Tool-Agent Performance under Controlled Pressures and Fixed Harness Budgets Measures tool-agent performance failures under controlled harness limits such as completion caps and tool menu visibility. Ranked #110New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | On-topic Introduces a benchmark evaluating tool-agent performance and failure modes across execution harness constraints. | |
| DistressBench: Evaluating Multi-Dimensional Crisis Support in Large Language Models Introduces an evaluation benchmark measuring how effectively language models provide supportive responses during self-harm and crisis disclosures. Ranked #111New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Introduces DistressBench, a rubric-based benchmark measuring crisis support capabilities in frontier LLMs. | |
| SmartSalesBench: A Resistance-Aware Multi-Turn Benchmark for Evaluating LLM Sales Agents Introduces a multi-turn conversational benchmark to evaluate sales agents against resistant, psychologically grounded buyer personas. Ranked #112New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Introduces SmartSalesBench, a new benchmark and metric suite with LLM-as-judge evaluation. | |
| When Faster Decisions Do Not Make Better Agents: An Empirical Study of Jev in Agent Workflows Studies how accelerated decision-making models affect overall task completion, latency, and costs across embodied and mobile agents. Ranked #113Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Conducts an extensive empirical benchmark on agent decision speed versus end-to-end task success. | |
| Don't Guess, Just Ask: Evaluating Interactive Requirement Clarification for Repository-Level Software Development Introduces a benchmark to evaluate how effectively LLMs clarify ambiguous user requirements in repository-level software development tasks. Ranked #114New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Introduces ReqClarifyBench, a new benchmark assessing interactive requirement clarification in LLMs. | |
| On the estimation and validity of AI time horizons---a statistical look at the METR plot Recomputes METR task time horizons using splines and item-response theory to examine the validity of AI difficulty curves. Ranked #115New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Analyzes the statistical estimation, validity, and calibration of the METR AI time horizon benchmark. | |
| Searching for "Harmful Refusal": A Psychometric Audit of an AI Safety Benchmark Applies psychometric auditing to HELM Safety benchmarks to test whether harmful refusal functions as a single measurable attribute. Ranked #116Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Conducts a psychometric audit of safety benchmarks, analyzing construct validity and benchmark saturation. | |
| Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict Proposes an evaluation framework to measure how LLM agents handle uncertainty and resolve conflicting knowledge during execution. Ranked #117New way to see a problemPeer-reviewed7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Introduces a new evaluation framework and behavioral dimensions to benchmark agent uncertainty. | |
| An Investigation of Model Coherence: Narrow Finetunes Contradict Themselves Under Resampling Evaluates model coherence using 175 specific questions to measure internal contradictions and identity conflation under answer resampling. Ranked #118Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Introduces a 175-question test and metric to evaluate model coherence and contradiction under resampling. | |
| Can LLMs Fix It Without Code? Toward Automated Verification of No-Code Bug Fixes Presents an execution-based evaluation pipeline using browser agents to verify language model solutions for no-code software bug reports. Ranked #119New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Introduces an automated execution-based benchmark evaluating LLM capabilities on no-code bug fixes. | |
| When VLMs Destroy What They Know: Spatial Representations Fall Below Lexical Priors The paper evaluates vision-language models' spatial representations using causally controlled perturbations, showing spatial relations degrade compared to lexical features. Ranked #120Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Introduces a new metric and benchmark to evaluate spatial representation fidelity in VLMs. |
61–120 of 3,345← PreviousNext →