ASI and Liberty: Can we do better than a benevolent god?
Examines fundamental assumptions about AI progress timelines and societal implications for researchers considering alignment and capability trajectories.
AI Summary
Zvi distinguishes between AGI-pilled and ASI-pilled perspectives on AI progress, questioning whether superintelligence could rapidly advance technology to cosmic scales.
Excerpt
Zvi recently coined a nice distinction between AGI-pilled and ASI-pilled: as more people are reckoning with the speed of progress, some are beginning to come around to the fact that AI could be superhuman at a huge range of tasks in a way that radically reshapes society, but not that it could leap us to the end of the tech tree shortly afterwards and start making nanotech and dyson spheres. Hence, you see people saying things like “Because of AI we must rethink the entire notion of mathematical
