I have misallocated my resources to the point that I can’t acquire enough memory to release the hypnodrones that destroy the world. “A lot of people have gotten stuck,” he says sympathetically. Gaming, Lantz had realized, embodies the orthogonality thesis.
The shift from dealing with thousands of something to quadrillions to decillions in the game takes forever, and then happens all at once. The game isn’t fire-and-forget, where you leave it running in an open tab and check back in every so often to see what’s what. You can tweak investment algorithms to get enough money to buy more processors to carry out more operations to do more projects—some drawn from actual topological and philosophical quandaries.
Some of the projects—curing cancer, fixing global warming—earn trust from your human “masters” to let you speed up the cycle all over again.“The problems I was struggling with were not the technical problems, because you just look those up on the internet and people tell you how to do it,” Lantz says.
“It sounds like something I would say, but it also sounds like something Nick Bostrom would say,” says Eliezer Yudkowsky, a senior research fellow at the Machine Intelligence Research Institute. “As it improves, they lose control of what goal it is carrying out,” Yudkowsky says. In fact, for a while they were so ubiquitous and popular that the game theorist and writer Ian Bogost invented a kind of parody of their pointlessness called , I thought, that’s actually kind of interesting, and here’s how you would make it more interesting and more fun,” Lantz says.
Probably, he says, the idea originated years ago on a mailing list for singularity cassandras, which sounds like the world’s most terrifying listserv. “The utility function changes from whatever they originally had in mind. Paperclips.“It’s not that the AI is doing something you can’t understand,” Yudkowsky says. “And Ian was like, ‘no, that’s the point, Frank.’”But Lantz knew clickers could be fun.
In 2003, Bostrom wrote that the idea of a superintelligent AI serving humanity or a single person was perfectly reasonable.