Every paper checked — nothing hidden
All papers
Every paper checked in the week of 12–18 Oct 2026 · LLM evaluation and benchmarks — nothing is hidden. Brief papers first.
All3,345In brief3More (on-topic)587Not relevant2,721Not assessed24Older45Could not confirm25Blogs with no paper54All blogs57
| Paper | Topic | Relevance | Feedback |
|---|---|---|---|
| Measuring Artifact Reliance in Machine-Generated Text Detectors The paper demonstrates that machine-generated text detectors rely heavily on document formatting and PDF extraction artifacts rather than text quality. Ranked #121Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Identifies formatting artifacts and introduces format counterfactuals for evaluating text detectors. | |
| Same Outcome, Different Evidence: Intent Recovery in LLM Safety Evaluation Pairs attack success rate with operative understanding rate to assess whether models recognize harmful intent during safety evaluations. Ranked #122Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Critiques attack success rate and proposes operative understanding rate for LLM safety evaluation. | |
| Do Prompt Optimizers Break Requirements They Are Not Scored On? A Preregistered Study Evaluates whether automated prompt optimizers inadvertently cause regressions in unscored prompt constraints and requirements. Ranked #123Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Evaluates prompt optimization benchmarks and assesses metrics on unmeasured constraints. | |
| How Safe Is ChatGPT Health Advice? Evidence from Real User Conversations in Four Global South Countries Evaluates the safety of ChatGPT health advice using real user conversation logs across four Global South countries. Ranked #124Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Presents a demographically grounded safety evaluation rubric and benchmark using real-world user conversations. | |
| Reflective Prompt Optimization of a Deep-Research Agent: A Topic-Disjoint, Multi-Seed Protocol for Measuring Transferable Gains and Prompt-Level Overfitting Studies reflective prompt optimization in research agents, demonstrating that gains on held-out tasks can stem from prompt-level memorization. Ranked #125Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Proposes a rigorous topic-disjoint evaluation protocol to diagnose prompt-level overfitting. | |
| SpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models Introduces a benchmark evaluating how well vision-language models predict unseen physical outcomes following scene interventions. Ranked #126New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Introduces a diagnostic benchmark evaluating predictive spatial reasoning capabilities in models. | |
| SteerablePlex: Can We Steer Full-Duplex Models? Introduces an instruction-following benchmark and training recipe to improve goal adherence in full-duplex conversational models. Ranked #127New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Introduces SimIF-Bench to evaluate instruction-following and goal-completion in conversational simulators. | |
| A persistent accuracy ceiling in automated verbal deception detection The paper meta-analyzes 25 years of automated verbal deception detection models and identifies a persistent accuracy ceiling around 70 to 75 percent. Ranked #128Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Synthesizes 25 years of automated deception detection benchmarks, showing persistent methodological flaws. | |
| When History Helps and Hurts: Selective History Use across Multimodal Turns The paper introduces a 7,000-task benchmark to evaluate how multimodal models selectively use or ignore conversational context. Ranked #129New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 12 Oct, 16:28 | On-topic Introduces ReTurn, a benchmark for evaluating selective history use in conversational models. | |
| Fine-Tuning Strategy, Not Just Data: A Statistically Rigorous Audit of Gender and Stereotypical Bias in Embedding Models for HR Recommendation Systems This paper audits gender and stereotypical bias in multilingual job-matching embedding models using robust statistical tests. Ranked #130Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Proposes a rigorous statistical evaluation framework to audit embedding bias. | |
| Attacking Only Half the Data: Two-Sided Evaluation of AI-Text Detector Robustness Evaluates AI text detectors under attacks applied to both human and machine text, revealing high false-positive rates on edited human writing. Ranked #131Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces a two-sided evaluation protocol exposing flaws in AI detector benchmarks. | |
| Correct Choices, Failed Sequences: Choice2Seq Benchmarks Correctness Retention Across Output Interfaces This paper introduces Choice2Seq, a benchmark measuring how often language models retain correct multiple-choice answers when generating them step by step. Ranked #132New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces Choice2Seq, a diagnostic benchmark measuring correctness retention across different output formats. | |
| Passing Tests, Missing Semantics: Evidence of Interpretive Lock-In in Plan-Based DataFrame Code Generation The paper examines interpretive lock-in where planning agents generate code passing self-generated tests while missing the actual specification requirements. Ranked #133Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Evaluates benchmark metric pitfalls where generated code passes tests but misses semantics. | |
| Suggested Is Not Superseded: A Pre-Registered Evaluation of a System-One Model as a Memory-Retirement Gate The paper evaluates long-term memory gating models to test whether they distinguish established factual updates from mere suggestions. Ranked #134Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Establishes a pre-registered evaluation protocol and test pairs for memory-retirement gates. | |
| Truncation Samplers Are Nested: Comparing Decoding Rules Is a Measurement Problem The paper presents a theoretical framework using nesting properties to calculate distance bounds and compare language model truncation sampling algorithms efficiently. Ranked #135Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Analyzes the mathematical measurement and empirical bounds of truncation sampling across benchmarks. | |
| ClinEdge: An Expert-Graded Benchmark of Real Patient Edge Cases Where Frontier Multimodal Models Fail Introduces a medical benchmark of real de-identified patient edge cases pairing imaging with clinical vignettes to test multimodal language models. Ranked #136New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces an expert-graded benchmark of clinical edge cases combining text vignettes and imaging. | |
| Does Pretraining Teach Chemical Language Models Chemistry? This paper evaluates pretraining on chemical language models across 64 molecular datasets, finding that gains saturate below gradient-boosted tree baselines. Ranked #137Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Evaluates chemical language models across 64 molecular datasets to reveal pretraining limitations. | |
| OpenDGO: Benchmarking Multi-Agent Organizations Across Diverse Application Domains This paper introduces a benchmark evaluating multi-agent organizational structures across diverse domains and collaborative interaction patterns. Ranked #138New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Introduces a benchmark evaluating multi-agent system organizations across multiple real-world domains. | |
| LLM Evaluation is a Forecasting Problem This paper frames language model evaluation as a forecasting problem, using time-series techniques on historical data to predict post-deployment performance. Ranked #139New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Reframes LLM evaluation by showing benchmarks shift across time and proposing forecasting methods. | |
| On the Calibration of Crowdsourcing Aggregators The paper provides theoretical and empirical analyses of calibration in crowdsourced label aggregation across human and language model datasets. Ranked #140Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Studies calibration of crowdsourcing aggregators, including crowds of LLMs. | |
| How the Classical Baseline Shapes Machine-Learning Speed Claims in Topology Optimization This paper audits speedup claims of machine learning models in structural topology optimization against modern classical solvers. Ranked #141Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Critiques baseline comparability and benchmark evaluation practices in machine-learning speedup claims. | |
| PVFH-Bench: Benchmarking Premise-Induced Visual Fixation Hallucinations in Vision-Language Models This paper introduces a benchmark to measure vision-language model hallucinations caused by deceptive textual premises paired with visually confusable images. Ranked #142Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Introduces PVFH-Bench, a diagnostic benchmark measuring visual fixation hallucinations in LVLMs. | |
| VitaBench: Memory Lifecycles Beyond Retrieval This paper introduces a benchmark to evaluate agent memory lifecycle decisions, including compression, consolidation, and deletion. Ranked #143New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Introduces VitaBench, a new benchmark evaluating long-term memory lifecycle decisions in language agents. | |
| BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views This paper introduces a benchmark of paired web pages to evaluate prompt injection detection across browser agents and views. Ranked #144New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:18 | On-topic Introduces a benchmark evaluating prompt injection detection across diverse web contexts. | |
| Isolating a Language Model's Judgment with a Label-Free Subspace The paper proposes isolating internal model judgments from instruction formatting variations using a label-free hidden state subspace. Ranked #145New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | On-topic Investigates internal model judgment versus prompt sensitivity for reliable evaluation. | |
| Behavioral Forgetting Is Not Privacy in Machine Unlearning Demonstrates that machine unlearning often fails to prevent targeted information recovery across alternative multilingual access paths. Ranked #146New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | On-topic Exposes fundamental flaws in existing unlearning benchmarks and evaluation metrics. | |
| When Does "Post-Call" Begin? An Availability Contract for Earnings-Call NLP Benchmarks Proposes an availability framework to identify and resolve within-event timing mismatches in earnings-call financial NLP benchmarks. Ranked #147New testPeer-reviewed7/10 suggested | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | On-topic Identifies timing lookahead contamination flaws in financial NLP benchmark designs. | |
| Auditing Escalation Reliability in Healthcare LLM Agents: A Perturbation-Controlled Benchmark of Workflow-State Failures Introduces an episode-level healthcare benchmark to audit how reliably language model agents escalate actions when facing workflow disruptions. Ranked #148New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 17 Oct, 06:17 | On-topic Introduces a perturbation-controlled benchmark to evaluate agent decision-making under uncertainty. | |
| SolBehave: Evaluating the Behavioural Correctness of LLM-Generated Smart Contracts The paper introduces a benchmark using fuzzing and transaction replays to evaluate the behavioral correctness of generated smart contracts. Ranked #149New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces SolBehave benchmark to test the behavioral correctness of LLM-generated code. | |
| Does self-distillation improve reasoning? The paper evaluates whether self-distillation using privileged information improves reasoning capabilities or reflects overfitting to training tasks. Ranked #150Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Critically evaluates whether self-distillation benchmarks reflect true reasoning gains or overfitting. | |
| How Well Do LLMs Perform in Peer Review? Evidence from Finance and Accounting The authors evaluate language models on finance peer review tasks, finding predictive ability for publication outcomes through correctness rather than originality assessments. Ranked #151Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Investigates the validity and limitations of LLM-as-a-judge for academic peer review. | |
| Consider, Then Choose: Order-Invariant Decisions in System One Models Runoff eliminates option-order bias in multiple-choice model predictions by scoring items independently before running balanced candidate comparisons. Ranked #152New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Exposes option-order sensitivity in multiple-choice evaluations and proposes an invariant scoring approach. | |
| Claim Framing as a Driver of LLM Sycophancy This paper studies how different framings of user claims induce sycophancy in large language models on factual questions. Ranked #153Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Establishes a systematic benchmark and evaluation methodology for probing LLM sycophancy. | |
| AM-Bench: Understanding Agents under Task Underspecification across Models and Harnesses This paper introduces a benchmark to measure how language model agents handle ambiguous or missing goals, context, and constraints. Ranked #154Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Presents a benchmark testing agent performance under goal, context, and constraint underspecification. | |
| ResearchFlow-Eval: Tracing the Execution Gap in End-to-End Multi-Claim Autoresearch This paper introduces an evaluation framework grounded in published papers to assess continuous multi-stage execution in automated research agents. Ranked #155Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces an evaluation benchmark measuring process-level execution in automated research systems. | |
| Same Attack, Different Judge: A First Controlled Adversarial Comparison of Jev, LLM Judges, and Purpose-Built Safety Classifiers This paper conducts an adversarial evaluation comparing a probability-based safety judge against large language model judges and safety classifiers across multiple attack types. Ranked #156Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Provides a controlled adversarial comparison and evaluation methodology for safety classifiers and LLM judges. | |
| Memory Echo in Single-Agent Long-Term Memory: Lifecycle Evaluation and Provenance-Guided Mitigation This paper examines how single-agent long-term memory accumulates repeated historical claims and introduces a benchmark and framework to mitigate unverified persistence. Ranked #157New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces MEchoBench to evaluate long-term memory persistence and correction in agent systems. | |
| Instructions Are Evidence, Not Objectives: Measuring Outcome Alignment of Agents This paper introduces an approach and benchmark for evaluating whether language model agents can infer unstated true requirements instead of following incomplete instructions. Ranked #158New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces TiGanEval, a benchmark measuring whether agents discover true requirements beyond stated instructions. | |
| Auditing Failure Attribution: Label Predictability and Judge Behavior This paper provides four auditing tests to determine whether failure-attribution judges actually perform multi-step checks or merely rely on spurious surface cues. Ranked #159Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Exposes how surface cues distort LLM-as-a-judge failure attribution benchmarks and proposes audit tests. | |
| Visual Dependence Does Not Imply Answerability: A Systematic Analysis of Answerability Signals in Frozen InstructBLIP Analyzes answerability signals in frozen visual question answering models, finding standard confidence and alignment metrics fail to reliably detect unanswerable inputs. Ranked #160Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Provides a systematic evaluation of answerability signals and unanswerable question discrimination in VLMs. | |
| Interaction-as-a-Judge Presents Interaction-as-a-Judge, using computer-use agents to evaluate interactive web and desktop software applications through direct dynamic interaction. Ranked #161New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Proposes an interactive evaluation paradigm and releases benchmarks for judging interactive applications. | |
| ProtoArena: Can AI Agents Discover and Exploit Real-World Cross-Component Protocol Vulnerabilities? Introduces ProtoArena, a black-box dynamic benchmark evaluating the ability of AI agents to discover and exploit cross-component network protocol vulnerabilities. Ranked #162New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces a dynamic benchmark evaluating agent capabilities in exploiting network protocol vulnerabilities. | |
| SWEPatchTest: Evaluating Software Tests Beyond Reproduction SWEPatchTest provides a benchmark to evaluate whether agent-generated test suites correctly accept valid patches and reject invalid ones. Ranked #163New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces a new evaluation benchmark for software tests generated for code patches. | |
| Large Language Models Extend Known Research But Do Not Create New Research Directions The study evaluates scientific hypotheses generated by language models, concluding that they extend existing knowledge rather than starting new research directions. Ranked #164Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Proposes a framework to evaluate whether LLMs generate genuinely novel scientific hypotheses. | |
| Get In, Decide, Prove: A Merchant-Side Readiness Benchmark for Agentic Commerce The paper introduces a benchmark assessing whether merchant websites provide adequate access, terms, and evidence for autonomous shopping agents. Ranked #165New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces a benchmark evaluating merchant sites for agentic decision sufficiency. | |
| LatentRiskBench: Measuring State-Dependent Risks in Enterprise Agents Introduces LatentRiskBench to evaluate safety risks in enterprise agents where action safety depends on latent workflow context. Ranked #166New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces LatentRiskBench to evaluate state-dependent enterprise agent safety. | |
| Glasseek: A Transparent Recipe for Training and Trustworthy Evaluation of DeepSearch Agents The paper introduces an open-source framework and recipe for training and evaluating deep-search language agents on complex web exploration tasks. Ranked #167New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Audits evaluation contamination and establishes trustworthy evaluation practices for search agents. | |
| Comparing GenAI to Machine Learning and Manual Review for eDiscovery The paper analyzes legal eDiscovery review data to compare generative AI tools against continuous active machine learning and manual human review. Ranked #168Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Critiques evaluation methodology and compares performance metrics between GenAI and ML. | |
| Audit tools of the trade: Automated Auditor Elicitation Can Be Sensitive to the Context the Auditor Is Given The paper examines how providing additional context, such as codebases and benchmark transcripts, affects automated LLM safety auditors. Ranked #169Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Evaluates how context shapes automated LLM auditor benchmark assessments. | |
| Releasing the Inner Jev: A Simple Recipe for Open Decision Models The paper demonstrates recipes to extract single-pass probability decisions directly from open instruction-tuned language models. Ranked #170Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces OpenDecisions benchmark to evaluate non-generative LLM decision heads. | |
| The Format is the Filter: Scoring Format Fixes What an LLM Judge Can Express Demonstrates that the scoring format chosen for an evaluation model strongly restricts its expressive capacity and ranking reliability. Ranked #171Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Analyzes how scoring formats bias and constrain LLM-as-a-judge evaluations. | |
| Structural Baselines Matter for Agent Failure Detection and Attribution Shows agent failure detection models often rely on trivial cues like run length, exposing benchmark flaws when attributing critical failure steps. Ranked #172Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Critically exposes flawed evaluation baselines and labeling artifacts in agent failure benchmarks. | |
| Does a Hosted Decision Model Respect Irrelevant Alternatives? A Pre-Registered Test of Option-Set Dependence in Jev Tests a hosted decision model to show that adding irrelevant options systematically shifts output probability distributions between top candidates. Ranked #173Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Presents a formal evaluation test checking decision models against independence of irrelevant alternatives. | |
| VLA-GAUGE: Systematically Evaluating Generalization and Robustness in Vision-Language-Action Models Introduces a diagnostic evaluation framework to systematically test generalization and perturbation robustness in vision-language-action robotic models. Ranked #174New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces a diagnostic evaluation framework and benchmark protocol for Vision-Language-Action models. | |
| EvoWeb: A Benchmark for Coding Agents on Evolving Web Applications Introduces a benchmark evaluating coding agents on evolving web applications across cumulative feature requests. Ranked #175New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces EvoWeb, a novel benchmark dataset for iterative coding tasks. | |
| SLDBench v2: Active Scaling Law Discovery Expands a benchmark to evaluate whether language model agents can actively select experiments to discover reliable model scaling laws under compute budgets. Ranked #176New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces a benchmark measuring LLM agent capabilities in active scaling law discovery. | |
| Review Rarely Beats the Stronger Model: Separating Verification from Capability Transfer in LLM Pipelines The paper shows that LLM review pipelines rarely outperform the stronger model alone due to confirmation bias rather than true verification. Ranked #177Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Examines benchmark evaluation protocols and reveals flaws in testing LLM review pipelines. | |
| Visible but Not Avoided: State-Dependent Re-Proposal of Rejected Candidates in Catalog-Based LLM Optimization Loops The paper demonstrates that language model optimizers frequently re-propose previously rejected candidates during search trajectories. Ranked #178Surprising resultPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Evaluates the disparity between static probing and dynamic trajectory behavior in LLM optimizers. | |
| Big-Big-SycoBench: A Benchmark to Detect Sycophancy Through Model Responses to Valid vs. Invalid Arguments The paper introduces a benchmark measuring whether language models resist sycophancy when presented with invalid user arguments versus valid critiques. Ranked #179New testPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Introduces a benchmark evaluating LLM sycophancy via valid versus invalid arguments. | |
| REPRO: Reflective Evolutionary Prompt Optimizer for Automated LLM Red-Teaming This paper proposes a black-box evolutionary prompt optimization framework using a small open-weight model to discover adversarial jailbreaks against language models. Ranked #180New way to see a problemPreprint7/10 suggested | LLM evaluation and benchmarks via arxiv · 15 Oct, 06:17 | On-topic Presents an automated red-teaming evaluation framework tested across multiple LLMs. |
121–180 of 3,345← PreviousNext →