← Back to feed

Bad Guy Tokens

L5 · ResearcherResearchLessWrong AI· 10/16/2026

Crucial safety research for understanding alignment robustness and model behavior under adversarial training.

AI Summary

A new paper shows that AI models to give bad advice only creates narrow, fragile 'bad guy' personas that break easily with minor changes.

Excerpt

Based on a recent paper, training models to give bad advice in a narrow domain doesn't make the model or "the persona" broadly evil in a robust way. It just makes the particular system prompt[1] tokens you fine-tuned with evoke an evil persona, and perturbing it a little (e.g. changing the date slightly) substantially reduces misaligned behavior. For review, a typical chatbot conversation looks like this under the hood: <|im_start|>system You are Qwen, created by Alibaba Cloud. You are a helpful

Read Original
0 upvotes · 0 downvotes · 1 min read

Related Articles

L5 · ResearcherResearchHacker News
Neural networks are clean

New research challenges the superposition theory of polysemanticity, suggesting neural networks are more interpretable than previously believed through recursive generalization patterns.

L5 · ResearcherResearchLessWrong AI
Linkpost: Neural Networks Are Already Clean

Nick Cammarata argues neural networks are already clean and interpretable, challenging previous pessimism about mechanistic interpretability.

L5 · ResearcherResearcharXiv
LIBERO-PeRM: Benchmarking Personalized Robotic Manipulation

Researchers introduce LIBERO-PeRM, a benchmark for personalized robotic manipulation that evaluates how AI policies learn and follow user preferences in household tasks.

L3 · BuilderResearchArs Technica AI
Targeted ads delivered 0-click exploits to install mercenary malware

Researchers documented mercenary hackers using targeted ads to deliver zero-click spyware that infected Al Jazeera journalists' devices without any user interaction.

L5 · ResearcherResearchLessWrong AI
Headwinds Everywhere (in the Chinese AI Safety Field)

Chinese AI safety research faces significant challenges with limited frontier risk awareness and systemic headwinds according to field research.