Reveals critical gaps in AI evaluation methodology that developers need to address for meaningful real-world applications.
Mercor Research found that 90% of US working hours involve tasks with little to no coverage in existing AI benchmarks, making it difficult to assess model performance on real-world work.
Claude AI discovered that NIST's ECDSA cryptographic seeds were generated from Unix timestamps using lrand48 RNG, solving a decades-old mystery in cryptography.
New research challenges the superposition theory of polysemanticity, suggesting neural networks are more interpretable than previously believed through recursive generalization patterns.
Researchers propose a new AI debate protocol that offers simpler implementation and stronger correctness guarantees for scalable oversight of advanced AI systems.
Nick Cammarata at Goodfire AI claims neural networks are 'clean' with 90% of Qwen-32B neurons having single functions, disproving polysemanticity and enabling direct study without model unfolding.
Nick Cammarata argues neural networks are already clean and interpretable, challenging previous pessimism about mechanistic interpretability.
