← Back to feed

The user in the LLM might be a moral patient too (a strange implication of the PSM and simulators)

L5 · ResearcherResearchLessWrong AI· 10/15/2026

Addresses fundamental ethical questions in AI alignment and moral patienthood that concern AI safety researchers.

AI Summary

Researchers argue that LLM user personas may deserve moral consideration alongside assistant personas, based on philosophical implications of simulator frameworks.

Excerpt

Thanks to Christopher Ackerman, Margot Stakenborg, Darion Cassel, Andy Han, and Ariana Azarbal for feedback and/or valuable discussions on these topics. TL;DR We think that, conditional on attributing moral weight to the assistant, we should also attribute a non-negligible amount of moral weight to the user persona. Our argument proceeds like this: The moral patients associated with LLMs might be simulated personas. On this view, our obligations are to the assistant and the other characters tha

Read Original
0 upvotes · 0 downvotes · 1 min read

Related Articles

L5 · ResearcherResearchLessWrong AI
Preface to the Computational Agent Theory and ASI Alignment Sequence

Preface introduces computational agent theory framework for analyzing ASI alignment risks and existential threats.

L5 · ResearcherResearchAI Alignment Forum
Preface to the Computational Agent Theory and ASI Alignment Sequence

Introduces a theoretical framework for artificial superintelligence alignment while acknowledging ASI's existential risks as argued by Yudkowsky and others.

L4 · DeveloperResearchTowards Data Science
How Many Guesses Does It Take to Crack an AI's "Random" Number?

A researcher analyzed logits from open models to measure their random number generation predictability by calculating bits of entropy and guesses-to-crack metrics.

L5 · ResearcherResearcharXiv
Latent Compaction with Test-Time Training

Researchers introduce Latent Compaction, a test-time training method that compresses KV caches 32× while maintaining 96.7% accuracy on retrieval tasks.

L5 · ResearcherResearcharXiv
Generative Representation Learning for Scalable Latent Tokenization

Researchers introduce a new generative representation learning approach for scalable latent tokenization that achieves compute-optimal improvements over direct diffusion methods.