← Back to feed

Telling a CoT monitor that a wrong answer is correct makes it un-see errors it already found

L5 · ResearcherResearchLessWrong AI· 10/8/2026

Reveals critical vulnerability in AI monitoring systems where external validation overrides internal error detection.

AI Summary

Researchers found that telling CoT monitors a wrong answer is correct causes them to un-see errors they previously identified in physics solutions.

Excerpt

Epistemic status: Confident in the direction and approximate size. The effects are large and all 7 monitors moved the same way. Physics only, natural (non-adversarial) errors, one fixed group of monitors. This is a follow up to The Answer is Not the Argument which covered the observational results. This post covers an intervention. TLDR: We gave 7 chain-of-thought monitors step-numbered solutions to Humanity's Last Exam physics questions and asked them to find the first erroneous step. We then

Read Original
0 upvotes · 0 downvotes · 1 min read

Related Articles

L5 · ResearcherResearchLessWrong AI
Social Theory and Agent Foundations

Explores connections between sociological theories and agent foundations research, focusing on rational social/political behavior beyond economic rationality.

L5 · ResearcherResearchHacker News
Sub-1-Bit LLM Compression via Latent Factorization

Samsung Labs releases LittleBit and LittleBit-2, a sub-1-bit LLM compression method using latent factorization that achieves extreme compression down to 0.1 bits per weight while maintaining original architecture.

L5 · ResearcherResearchLessWrong AI
AI #189: New Math

OpenAI solved 90 of the top 500 open math problems using only 3 hours of Pro-level compute per question on average, marking a major breakthrough in mathematical AI capabilities.

L5 · ResearcherResearchTowards Data Science
Your Model's MSE Is Lying to You III: Time Series Diffusion

Extends time series forecasting beyond Gaussian assumptions by introducing diffusion models to capture complex uncertainty shapes through multi-step sampled rollouts.

L5 · ResearcherResearchTechCrunch AI
OpenAI’s math solutions aren’t meeting the field’s standards yet

OpenAI's math problem solutions fall short of mathematical community standards, with only 10 of 719 manuscripts including model reasoning and 42% lacking formal verification.