istewart @ istewart @awful.systems

Posts

0

Comments

267

Joined

11 mo. ago

I picked up a modern Fortran book from Manning out of curiosity, and hoo boy are they even worse in terms of trend-riding. Not only can you find all the AI content you can handle, there's a nice fat back catalog full of blockchain integration, smart-contract coding... I guess they can afford that if they expect the majority of their sales to be ebooks.

I'm mildly surprised that music of any kind is what's getting uploaded to something called Deezer

ChatGPT's got what intelligence craves... it's got neurons

As I noted on the YouTube video, this is doubly heinous as a lot of CA community college instructors are "freeway flyers" - working at multiple campuses, sometimes almost 100 miles apart, just to cobble together a full-time work schedule for themselves. Online, self-paced, forum-based class formats were already becoming popular even before the pandemic, and I've been in such classes where the professor indicated that I was one of maybe 3 or 4 students who bothered to show up to in-person office hours. I have to wonder if that will end up being a hard requirement at some point. The bottom rung on the higher-education ladder is already the most vulnerable, and this just makes it worse.

I have to agree. There are already at least two notable and high-profile failure stories with consequences that are going to stick around for years.

1. The Israeli military's use of "AI" targeting systems as an accountability sink in service of a predetermined policy of ethnic cleansing.
2. The DOGE creeps wanting to rewrite bedrock federal payment systems with AI assistance.

And sadly more to come. The first story is likely to continue to get a hands-off treatment in most US media for a few more years yet, but the second one is almost certainly going to generate Tacoma Narrows Bridge-level legends of failure and necessary restructuring once professionals are back in command. The kind of thing that is put into college engineering textbooks as a dire warning of what not to do.

Of course, it's up to us to keep these failures in the public spotlight and framed appropriately. The appropriate question is not, "how did the AI fail?" The appropriate question is, "how did someone abusively misapply stochastic algorithms?"

Would you invest in commercial real estate, knowing there was a non-zero chance your tenants might come in one day to discover a thoroughly intoxicated JD Vance in a compromising position with the break-room furniture?

Mesa-optimization... that must be when you rail some crushed-up Adderall XRs, boof some modafinil for good measure, and spend the night making sure your kitchen table surface is perfectly flat with no defects abrasions deviations contusions...

couldn't help myself, there are seldom more perfect opportunities to use this one

I actually think it's part-and-parcel of Yarvin's personality. As much as he rails against "the Cathedral," PMCs, whatever, he himself is a perfect example of a pathological middle manager. Somebody who wants power without having to shoulder ultimate responsibility. He craves the childishly simplified social environment of a medieval-fantasy king's court, but he doesn't want to be the king himself. He wants to be (and has been, up until now) the scheming vizier who can run his manipulation games in the background, deciding who gets in front of the king but not having to take the heat if the king makes a bad decision. (And the "kings" he works for have made plenty of bad decisions, but consequences have only just begun to catch up.)

I suspect this newfound mainstream attention is far more uncomfortable than it is validating for him. Perhaps the NYT profile was a burst of exhilaration, but the shine has worn off quickly. This correlates with the story last year about him coming back to Urbit as a "wartime CEO." If Urbit is so damn important for building his ridiculous vision, why wasn't he running it the whole time? He doesn't actually want to be CEO of anything. Power without responsibility.

He will never stop to reflect that his "philosophy," such as it is, is explicitly tailored for avaricious power-hungry narcissists, soooooo

Obvious joke is obvious, but

The essay brims with false dichotomies, logical inconsistencies, half-baked metaphors, and allusions to genocide. It careens from Romanian tractor factories to Harvard being turned “into dust. Into quarks” with the coherence of a meth-addled squirrel.

Harvard isn't already full of Quarks?

Another thread worth pulling is that biotechnology and synthetic biology have turned out to be substantially harder to master than anticipated, and it didn't seem like it was ever the primary area of expertise for a lot of these people anyway. I don't have a copy of any of Kurzweil's books at hand to look at his predicted timelines for that stuff, but they're surely way off.

Faulty assumptions about the biological equivalence of digital neural network algorithms have done a lot of unexamined heavy lifting in driving the current AI bubble, and keeping the harder stuff on the fringes of the conversation. That said, I don't doubt that a few refugees from the bubble-burst will attempt to inflate the next bubble on the back of speculative biotech, and I've seen a couple of signs of that already.

For my money, 2015/16 Adams trying to sell Trump as a "master persuader" while also desperately pretending not to be an explicit Trump supporter himself was probably the most entertaining he's ever been. Once he switched from skimmable text blogging to livestreaming, though, he wanted to waste too much of my time to be interesting anymore.

"This Is What Yudkowsky Actually Believes" seems like a subtitle that would get heavy use in a future episode of South Park about Cartman dropping out after one semester at community college.

Yes, Kurzweil desperately trying to create some kind of a scientific argument, as well as people with university affiliations like Singer and MacAskill pushing EA, are what give this stuff institutional strength. Yudkowsky and LW are by no means less influential, but they're at best a student club that only aspires to be a proper curriculum. It's surely no coincidence that they're anchored in Berkeley, adjacent to the university's famous student-led DeCal program.

FWIW, my capsule summary of TPOT/"post-rationalists" is that they're people who thought that advanced degrees and/or adjacency to VC money would yield more remuneration and influence than they actually did. Equally burned out, just further along the same path.

I've been contemplating this, and I agree with most everyone else about leaning heavily into the cult angle and explaining it as a mutant hybrid between Scientology-style UFO religions and Christian dispensationalist Book of Revelation eschatology. The latter may be especially useful in explaining it to USians. My mom (who works in an SV-adjacent job) sent me this Vanity Fair article the other day about Garry Tan grifting his way into non-denominational prosperity gospel Christianity: https://www.vanityfair.com/news/story/christianity-was-borderline-illegal-in-silicon-valley-now-its-the-new-religion She was wondering if it was "just another fad for these people," and I had to explain no, not really, it is because their AI bullshit is so outlandish that some of them feel the need to pivot back towards something more mainstream to keep growing their following.

I also prefer to highlight Kurzweil's obsession with perpetual exponential growth curves as a central point. That's often what I start with when I'm explaining it all to somebody. It provides the foundation for the bullshit towers that Yudkowsky and friends have erected. And I also think that long-term, the historiography of this stuff will lean more heavily on Kurzweil as a source than Yudkowsky, because Kurzweil is better-organized and professionally published. It'll most likely be the main source in the lower-division undergraduate/AP high school history texts that highlight this stuff as a background trend in the 2010s/2020s. Right now, we live in the peak days of the LessWrong bullshit volcano plume, but ultimately, it will probably be interpreted by the specialized upper-division texts that grow out of peoples' PhD theses.

awful.systems

Huh, 2 paradigm shifts is about what it takes to get my old Beetle up to freeway speed, maybe big Yud is onto something

It is what happened to look good in the valley between the Adderall comedown and yesterday evening's edible really starting to hit

Just had a video labeled "auto-dubbed" pop up in my YouTube feed for the first time. Not sure if it was chosen by the author or not. Too bad, it looks like a fascinating problem to see explained, but I don't think I'm going to trust an AI feature that I just saw for the first time to explain it. (And perhaps more crucially, I'm a bit afraid of what anime fans will have to say about this.)

And the photos from a previous event are an ocean of whiteness. Hard to argue that they're not, uh, cultivating a certain demographic...