ShakingMyHead @ ShakingMyHead @awful.systems

Posts

0

Comments

117

Joined

1 yr. ago

TBF it's probably just a "me" thing. I don't know how much stock you want to put in that.

At the risk of being critical of Zitron, I have some comments. This is probably just nitpicking but regardless.

[...] a technology called Large Language Models (LLMs), which can also be used to generate images, video and computer code.

LLMs cannot be used to generate images or videos. Diffusion models can create images, but that's not a text-generation model. I guess you could use an LLM to prompt an image or video generation model, but I'm not sure if that's what he meant or not.

Large Language Models require entire clusters of servers connected with high-speed networking, all containing this thing called a GPU — graphics processing units.

Sort of, but not really. GPT-5 with its (presumably) trillions of parameters and its (apparently) hundreds of millions of users per month needs a lot of throughput to cater to that, but there's nothing about LLMs that inherently requires massive GPU clusters with high-speed networking.

Here's a LLM running on a Raspberry Pi

Of course, the amount of people running LLMs on Raspberry Pis is effectively that guy in the video to show a LLM running on a Raspberry Pi, and it's not like it's particularily fast without adding a GPU (and at the end of the day it's still LLM output, so), so perhaps he's just using "Large Language Models" as in "The LLMs that the vast majority of people actually use."

He's not wrong about training, however.

IMO it's not a particularly good start to his newsletter. Because an easy counter to his statement is that not all LLMs require massive amounts of compute to run, but a counter against that counter is that training even smaller LLMs still require vast amounts of compute that the average person doesn't have, in addition to the copyrighted material needed to train on, even with the win that Anthropic got meaning that any LLM trained in the future is going to require vast amounts of capitol for just the training data alone. The problem is that he doesn't state any of that. Maybe he does know about that and decided to omit it for brevity. If he did, then, personally, I think that's a mistake. Or maybe I'm just not reading it properly.

The first paragraph immediately conflating all of generative AI with LLMs doesn't particularly help his case either, even though stating that there are multiple types of generative AI wouldn't really harm his thesis that this entire thing is a massive bubble. Again, perhaps he's doing it for a reason that I'm not getting.

“We believe that in the near future half the people on the planet will be AI, and we are the company that’s bringing those people to life”

This quote is just... something.

Is the plan to literally create 8 billion podcasts in the near future? This company doesn't think that might be a tad excessive?

From what I've read (granted from other reviews), the limit would be the equivalent performance of 8 4090s. Which means (assuming we believe Nvidia's claims of 3352 AI TOPS for the 5090 vs the 1321 AI TOPS for the 4090) that you couldn't possess more than the equivalent of 3 5090s. Then that keeps going, so.

Same here, I've never actually seen the term "clanker" be used in reference to a person using the AI, but the AI itself. Which to me was analogous to going to an expensive bakery and accusing the bread of ripping you off instead of the baker (or whoever was setting prices, which wouldn't be the bread).

If there was any sort of op going on (which I don't think there is), I'd guess it would be from the AI doomers who want people to think of these things as things with enough self-awareness that something like "clanker" would actually insult them (but, again, probably not, IMO).

Also, all this would do is change the processing from GPU to CPU. Microsoft commissions AMD, Nvidia or Intel to create a technically-not-a-GPU CPU and just have a computer that uses GDDR instead of the standard DDR.

Thanks.

TBF I can't say I'm sold on the notion of watching Yudkowsky eat spicy wings while also arguing with someone.

Forgive my stupidity, but I don't know what this means. Or it's referencing something I don't know.

Pretty sure I've heard similar things about AI "art" vs artists as well.

I think I might be missing some context here. Granted without context I'm pretty sure that strawman is still the right word.

Pretty sure that's a strawman.

Kind of like saying that humans are car-driving machines because we drive cars.

LLMs and humans are both sentence-producing machines, but they were shaped by different processes to do different work

Except not really. We're not sentence-producing machines, we're "machines" (so to speak) that can produce sentences. Not the same thing.

Once this is in place, they say, nations must be prepared to enforce these restrictions by bombing unregistered data centres, even if this risks nuclear war, “because datacenters can kill more people than nuclear weapons” (emphasis theirs).

So the plan is still to kill everyone to death to prevent GPT-5 6 7 8 ...

OpenAI’s tools also lower the cost of entry, allowing more people to make creative content, he said.

So, even working under the assumption that this somehow works, they still needed two animation studios, professional writers, and 30 million to get this film off the ground.

According to the article, it could be higher than 1.5 billion, though by how much they don't really say. But they're estimating about $3000 per book. For a class action that actually seems extremely high.

Anthropic to pay 1.5 billion to authors

That's... quite a bit. I wonder if we'll start to see more complaints about people reaching their token limit "prematurely" or if Anthropic will release a $1000/month "unlimited" plan or something like that.

This was written in 2015.