Zuckerberg wants to build artificial general intelligence with 350K Nvidia H100 GPUs

www.theregister.com

Zuckerberg wants to build AGI with 350,000 Nvidia H100 GPUs

LocalLLaMA @sh.itjust.works

ylai @lemmy.ml

2y ago

Zuckerberg wants to build artificial general intelligence with 350K Nvidia H100 GPUs

www.theregister.com /2024/01/20/metas_ai_plans/

31 comments

He should build a sub to go to the bottom of the ocean.
- And take Elon Musk with him
  
  they'll never do that, but maybe a spaceship shaped like a giant penis could do the trick
- And skimp on materials, build it out of carbon fiber instead of titanium or steel.
- He should test out those new nitrogen chambers.
Building a metaverse that people want to actually engage with was too hard, so he's decided to scale back his ambitions and tackle something less difficult: AGI.
- He just want some virtual friends to hang out with in the metaverse, since humans weren't interested.
- That whole laughable expedition stinks of financial fraud
Is Zuckerberg an idiot? Or does he have an actual plan with this?
Seems to me it's completely useless like Metaverse.
If the LLM is so stupid it can't figure out the sides of an equal sign can be reversed as simple as in 2+2=4 <=> 4=2+2. He will never achieve general intelligence by just throwing more compute power at it.
As powerful as LLM is, it's still astoundingly stupid when it hits its limitations.
- Humans are astoundigly stupid when they hit their limitations.
  
  Spoken like a true AI!!!! ;-)
  
  The difference is that we can go beyond that limitation. Even self-coding AI will either solve a problem, or compound its own inefficiencies before asking an operator to help out.
- I don't know much, but from what I know, we still haven't reach a point of diminishing returns, so more power = more better.
  
  Not necessarily since you also need better techniques. A competitor could easily surpass you with less by being smarter about how the AI is trained.
- Trying to achieve AGI by throwing more compute at LLMs is like trying to reach the moon by building a more powerful hot air balloon.
  
  Assuming that "not compute" should be "more compute" I totally agree. That's a very apt analogy.
Zuckerberg: Why are my pupils vertical slits? Why am I always cold? Why do people find me so repellent?
AI: Sir, I can answer all three with one response, but you won't like it.
Sure, we could point to thousands of years of really smart people trying and utterly failing to build mathematical models for innovation and thought, but it also does make a certain amount of sense that, if you pile up enough transistors and wish really hard, that your investment will Frosty the Snowman itself into being your friend, right...?
I wonder if he really thinks that AGI is just AI with more parameters and gpus thrown into the mix.
Iirc the h100s are $30k per gpu at this time.
- I’m sure he’d get a hefty discount on 350k of them
- Retail price of $10,500,000,000. That's nuts.
- Still less than Musk paid for Twitter. It's totally reasonable from a biggest-billionaire-toy entry.
- This is why they can't stop harvesting your data.

31 comments