PAPERCLIPS, A NEW game from designer Frank Lantz, starts simply. The top left of the screen gets a bit of text, probably in Times New Roman, and a couple of clickable buttons: Make a paperclip. You click, and a counter turns over. One.
The game ends—big, significant spoiler here—with the destruction of the universe.
In between, Lantz, the director of the New York University Games Center, manages to incept the player with a new appreciation for the narrative potential of addictive clicker games, exponential growth curves, and artificial intelligencerun amok.
“I started it as an exercise in teaching myself Javascript. And then it just took over my brain,” Lantz says. “I thought, in a game like this, where the whole point is that you’re in pursuit of maximizing a particular arbitrary quantity, it would be so funny if you were an AI and making paperclips. That game would design itself, I thought.”
Lantz figured it would take him a weekend to build.
It took him nine months.
And then it went viral.
THE IDEA OF a paperclip-making AI didn’t originate with Lantz. Most people ascribe it to Nick Bostrom, a philosopher at Oxford University and the author of the book Superintelligence. The New Yorker (owned by Condé Nast, which also owns Wired) called Bostrom “the philosopher of doomsday,” because he writes and thinks deeply about what would happen if a computer got really, really smart. Not, like, “wow, Alexa can understand me when I ask it to play NPR” smart, but like really smart.
In 2003, Bostrom wrote that the idea of a superintelligent AI serving humanity or a single person was perfectly reasonable. But, he added, “It also seems perfectly possible to have a superintelligence whose sole goal is something completely arbitrary, such as to manufacture as many paperclips as possible, and who would resist with all its might any attempt to alter this goal.” The result? “It starts transforming first all of earth and then increasing portions of space into paperclip manufacturing facilities.”
Bostrom declined to comment, but his assistant did send this email back when I pinged him: “Oh, this is regarding the paper clipping game,” she wrote. “He has looked at the game but due to the overwhelming number of requests, he hasn’t been sharing quotes on it.”
One of Bostrom’s fellow doomsayers did agree to explain the origin of paperclips as the End of All Things. “It sounds like something I would say, but it also sounds like something Nick Bostrom would say,” says Eliezer Yudkowsky, a senior research fellow at the Machine Intelligence Research Institute. Probably, he says, the idea originated years ago on a mailing list for singularity cassandras, which sounds like the world’s most terrifying listserv. “The idea isn’t that a paperclip factory is likely to have the most advanced research AI in the world. The idea is to express the orthogonality thesis, which is that you can have arbitrarily great intelligence hooked up to any goal,” Yudkowsky says.
So that’s good, right? A paperclip maximizer! Maximize a goal! That’s what an AI’s creators want, right? “As it improves, they lose control of what goal it is carrying out,” Yudkowsky says. “The utility function changes from whatever they originally had in mind. The weird, random thing that best fulfills this utility function is little molecular shapes that happen to look like paperclips.”
So … bad, because as the AI dedicates more and more intelligence and resources to making paperclips against all other possible outcomes … well, maybe at first it does stuff that looks helpful to humanity, but in the end, it’s just going to turn us into paperclips. And then all the matter on Earth. And then everything else. Everything. Is. Paperclips.
“It’s not that the AI is doing something you can’t understand,” Yudkowsky says. “You have a genuine disagreement on values.”
OK, OK, THAT doesn’t make the game sound fun. But I promise it is. See, Lantz is an ace at taking a denigrated game genre—the “clicker” or “incremental”—and making it more than it is.
You’ve seen these, maybe even played them. Remember Farmville? A clicker. In fact, for a while they were so ubiquitous and popular that the game theorist and writer Ian Bogost invented a kind of parody of their pointlessness called Cow Clicker, which, as my colleague Jason Tanz wrote about so elegantly in 2011, itself became wildly, unironically popular.
Bogost and Lantz are friends, of course. “When I first looked at Cow Clicker, I thought, that’s actually kind of interesting, and here’s how you would make it more interesting and more fun,” Lantz says. “And Ian was like, ‘no, that’s the point, Frank.’”
But Lantz knew clickers could be fun. To him, clickers are to big-budget, perfectly rendered, massively hyped AAA games as punk was to prog rock. Clickers can be sort of passive, more about immersing in the underlying dynamics of a system than mashing buttons. They have rhythms. “What they all have in common is a radical simplicity, a minimalism in an age where video games are often sort of over-the-top, baroque confections of overwhelming multimedia immersion,” Lantz says. “I really like that clicker games are considered garbage. That appeals to me.
For inspiration, Lantz turned to games like Kittens, a seemingly simple exercise in building villages full of kittens that spirals outward into an exploration of how societies are structured. (“I think stuff like this forges some deep, subtle bond that makes people play it for months and even years,” says the designer of Kittens, a software engineer who uses the alias Alma and designs games as a hobby . “AAA games usually try to operate on the same dopamine reinforcement cycle, but they never attempt to make you truly happy.”)
Lantz had been hanging around the philosophy web site Less Wrong, a hub for epic handwringing about singularities. He’d read Superintelligence, so he was familiar with the paperclip conjecture. And he realized that some really wild math underpinned it.
Unfortunately, Lantz is not very good at math. He asked his wife, who is, to help him translate the kind of exponential growth curves he wanted to convey into equations—so that, like, once you had 1,000 paperclip automated paperclip factories spitting out enough paperclips to create thousands more paperclip factories, the numbers would skyrocket. The shift from dealing with thousands of something to quadrillions to decillions in the game takes forever, and then happens all at once.
Source:-.wired
