1y ago

OK doomer

www.newyorker.com Among the A.I. Doomsayers

Some people think machine intelligence will transform humanity for the better. Others fear it may destroy us. Who will decide our fate?

The New Yorker has a piece on the Bay Area AI doomer and e/acc scenes.

Excerpts:

[Katja] Grace used to work for Eliezer Yudkowsky, a bearded guy with a fedora, a petulant demeanor, and a p(doom) of ninety-nine per cent. Raised in Chicago as an Orthodox Jew, he dropped out of school after eighth grade, taught himself calculus and atheism, started blogging, and, in the early two-thousands, made his way to the Bay Area. His best-known works include “Harry Potter and the Methods of Rationality,” a piece of fan fiction running to more than six hundred thousand words, and “The Sequences,” a gargantuan series of essays about how to sharpen one’s thinking.

[...]

A guest brought up Scott Alexander, one of the scene’s microcelebrities, who is often invoked mononymically. “I assume you read Scott’s post yesterday?” the guest asked [Katja] Grace, referring to an essay about “major AI safety advances,” among other things. “He was truly in top form.”

Grace looked sheepish. “Scott and I are dating,” she said—intermittently, nonexclusively—“but that doesn’t mean I always remember to read his stuff.”

[...]

“The same people cycle between selling AGI utopia and doom,” Timnit Gebru, a former Google computer scientist and now a critic of the industry, told me. “They are all endowed and funded by the tech billionaires who build all the systems we’re supposed to be worried about making us extinct.”

36 comments

This was such a chore to read, it's basically quirk-washing TREACLES. This is like a major publication deciding to take an uncritical look at scientology focusing on the positive vibes and the camaraderie, while stark in the middle of operation snow white, which in fact I bet happened a lot at the time.
The doomer scene may or may not be a delusional bubble—we’ll find out in a few years
Fuck off.
The doomers are aware that some of their beliefs sound weird, but mere weirdness, to a rationalist, is neither here nor there. MacAskill, the Oxford philosopher, encourages his followers to be “moral weirdos,” people who may be spurned by their contemporaries but vindicated by future historians. Many of the A.I. doomers I met described themselves, neutrally or positively, as “weirdos,” “nerds,” or “weird nerds.” Some of them, true to form, have tried to reduce their own weirdness to an equation. “You have a set amount of ‘weirdness points,’ ” a canonical post advises. “Spend them wisely.”
The weirdness is eugenics and the repugnant conclusion, and abusing bayes rule to sidestep context and take epistimological shortcuts to cuckoo conclusions while fortifying a bubble of accepted truths that are strangely amenable to allowing rich people to do whatever the hell they want.
Writing a 7-8000 word insider expose on TREACLES without mentioning eugenics even once throughout should be all but impossible, yet here we are.
- Inside the Strange World of the Uwu Smol Beans: An Exposé of a Quirky Community with No Racists Whatsoever
- quirk-washing TREACLES
  I can’t wait to be quirk-washed, I’m ready to hang up my pick-me hat and let the new yorker do the work for me
  
  speaking of, saw this this morning: https://www.vox.com/future-perfect/2024/2/13/24070864/samotsvety-forecasting-superforecasters-tetlock
  2 within a handful of days trying to reputation-wash after they got their filthy little selves exposed last year as the shitgremlins they are. hopefully it's just a coincidence in timing, but guess we'll have to see
  
  you gotta be white cis and loathsome or they won't do it
- God I always forget about the repugnant conclusion. It's baffling that it's being taken as anything but a fatal indictment of utilitarianism.

Yet another news story that omits how the science in HPMOR, the Sequences and the flagship e/acc blog is just wrong. Like, failing junior-high biology wrong.
The A.I., trying to access a Web site, was blocked by a Captcha, a visual test to keep out bots. So it used a work-around: it hired a human on Taskrabbit to solve the Captcha on its behalf.
Wait, didn't that turn out to be bullshit?
- Yeah, a lot of these TESCREAL exposés seem to lean on the perceived quirkiness while completely failing to convey how deeply unserious their purported scientific and philosophical footing is, like virgin tzatziki with impossible gyros unserious.

Wat
[Grace's] grandfather, a British scientist at GlaxoSmithKline, found that poppy seeds yielded less opium when they grew in the English rain, so he set up an industrial poppy farm in sunny Australia and brought his family there.
To grow opium???
(OK I guess for medicinal purposes but maybe point that out)
- We should have known the English rain was trouble when it started giving people tans
- I wonder how much of that family fortune has found its way into EA coffers by now.
  
  In another part of the article, it states that Grace grew up "semi-feral", so perhaps the fortune was smoked away in the Tasmanian opium dens (those exist, right?)

Oh, good, ex-incel Scott is in a polycule now, the wonders of the cult lifestyle.
- Wasn't he supposed to be a romantic asexual at some point?
  
  After all I’ve heard I believe that was a bald faced lie.

I have a bad feeling these people are going to waltz into even more power.
- I dunno. At least in the US, these people are decidedly outside the mainstream, at least in the US. Their views on religion and sexual mores preclude any popular appeal, and they are handicapped in a similar way were they to try to infiltrate existing power structures.
  Basically their only hope is that an AI under their control takes over the world.
  
  Basically their only hope is that an AI under their control takes over the world.
  They are pretty dominant in the LLM space and are already having their people fast tracked into positions of influence, while sinking tons of cash into normalizing their views and enforcing their terminology.
  Even though they aren't trying to pander to religious americans explicitly, their millenialism with the serial numbers filed off worldview will probably feel familiar and cozy to them.
  
  Come on, you’re talking about America, when did mainstream popular appeal ever limit anyone with money?

he dropped out of school after eighth grade, taught himself calculus
Lmaou, gonna need a citation on this one chief. This the same guy who said we need people monitoring for 'sudden drops' in the loss function? I'm supposed to believe this poser understands what a derivative is now?

Non-paywall link.

"Socialists think we’re sociopathic Randroid money-obsessed Silicon Valley hypercapitalists."
No, Scott, we just think you're a coward and a racist

Life is weird when you're living in a racist polycule

36 comments