1mo ago

AI will replace us all

story: https://www.pcmag.com/news/chatgpt-gets-absolutely-wrecked-in-chess-match-with-1978-atari

35 comments

This picture is AI generated
Edit: OP removed the picture from the post
- I know right, the Atari is really pulling it's weight.
Can the Atari answer my questions incorrectly with confidence?
Checkmate.
- Check
Everyone here saying "no shit, LLMs were not even designed to play chess" are not the people who this is directed at.
Multiple times at my job I have had to explain, often to upper management, that LLMs are not AGIs.
Stories like these help an under informed general public to wrap their heads around the idea that a "computer that can talk" =/= a "computer that can truly think/reason".
"Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid." attributed to Einstein but I read he didn't say it.
- I'm not sure I've ever read an actual quote by Einstein at this point.
  Or Thomas Jefferson for that matter
  I do like the quote regardless
To be fair, an Atari from 1977 would also outplay me in chess 😅
What exactly is it that makes the image generating AI use the ugliest colors for backgrounds? This one is like the stained walls in chain-smoker's house.
- Cross contamination from all the ai generated ghibli images, ai eats it's own shit and it is showing.
for some reason it reminds me of a quote from friends: "voice recognition is gonna be pretty much standard on any computer you buy. So you can be like 'wash my car', 'clean my room'. You know it's not gonna be able to do any of those things, but it'll understand what you're saying"
- I'm sure you could run Eliza in the Atari...
- My computer beat me at chess but I beat it at kickboxing.
- In that analogy, billions would be being spent by exotic car manufacturers saying they will replace all vehicules: airplanes, boats, scooters, bicycles, rockets....
  Also inexplicably the Lamborghini sometimes just throws itself into reverse and insists that it's moving forward.
No shit. Chess programs are specifically built and optimised to the nth degree for this specific use case and nothing else. They do not share the massive compute overhead and convoluted nondeterministic nature of an LLM.
This is like drag racing an F1 car and a Camry and being surprised at the result.
- This is like drag racing an F1 car and a Camry
  More like racing a Reliant Robin and an answering machine.
- I don't think the Atari Chess program is as optimized as you think.
  
  1.19 MHz, 1/8 kB RAM
  so no transposition tables, no endgame databases, nothing that requires pretty much any memory.
- Or have a real engine designer design a moderately powerful engine vs a computer throws together a blob of metal that looks kinda like an engine
In all fairness chessbots are REALLY REALLY good. Like incredibly good at chess. I am not shocked the guessing machine lost to one.
- You really don't understand how little processing a 2600 had, whoever wrote that chess algorithm is a fucking coding god
4O got wrecked. My ai fan friend said O3 is their reasoning model so it means nothing. I don't agree but can't find proof.
Has someone done this with O3?
- It’s a fundamental limitation of how LLMs work. They simply don’t understand how to follow a set of rules in the same way as a traditional computer/game is programmed.
  Imagine you have only long-term memory that you can’t add to. You might get a few sentences of short-term memory before you’ve forgetting the context of the beginning of the conversation.
  Then add on the fact that chess is very much a forward-thinking game and LLMs don’t stand a chance against other methods. It’s the classic case of “When all you have is a hammer, everything looks like a nail.” LLMs can be a great tool, but they can’t be your only tool.
  
  Or: If it's possible to create a simple algirithm, that will always be infinitely more accurate than ML.
  
  MY biggest disappointment with how AI is being implemented is the inability to incorporate context specific execution if small programs to emulate things like calculators and chess programs. Like why does it doe the hard mode approach to literally everything? When asked to do math why doesn't it execute something that emulates a calculator?
  
  It’s a fundamental limitation of how LLMs work.
  LLMs have been adding reasoning front ends to them like O3 and deep seek. That's why they can solve problems that simple LLM's failed at.
  I found one reference to O3 rated at chess level 800 but I'd really like to see Atari chess vs O3. My telling my friend how I think it would fail isn't convincing.
really funny video about chatgpt being horrible at chess https://youtu.be/l_wOsSda3Us

35 comments