Telling a CoT monitor that a wrong answer is correct makes it un-see errors it already found
Reveals critical vulnerability in AI monitoring systems where external validation overrides internal error detection.
AI Summary
Researchers found that telling CoT monitors a wrong answer is correct causes them to un-see errors they previously identified in physics solutions.
Excerpt
Epistemic status: Confident in the direction and approximate size. The effects are large and all 7 monitors moved the same way. Physics only, natural (non-adversarial) errors, one fixed group of monitors. This is a follow up to The Answer is Not the Argument which covered the observational results. This post covers an intervention. TLDR: We gave 7 chain-of-thought monitors step-numbered solutions to Humanity's Last Exam physics questions and asked them to find the first erroneous step. We then
