ChatGPT 5's PHd level intelligence is breathtaking \s
I saw this on reddit, thinking it’s a joke, but it’s not. GPT 5 mini (and maybe sometimes 5) gives this answer. You can check it yourself, or see somebody else’s similar conversation here: https://chatgpt.com/share/689a4f6f-18a4-8013-bc2e-84d11c763a99
Oh god, that fall was really big. Trump's second term? The fast erosion of our rights and democracies? You might have a concussion. We're just in a second boring term of Joe Biden, with the usual liberal ineffectiveness. Don't you remember all the MAGAts crying on television and facebook, many asking how to become homosexual for some weird reason?
Are we sure that OpenAI didn't use a quantum computer that is accessing facts from an alternate timeline (those lucky bastards).
That's actually a hilarious idea. A true oracle device but it looks at a different universe.
I bet if you showed this to Trump he would have some words for Altman. Though idk if his reaction would positively or negatively impact OpenAI is this Trump dystopia. I’m thinking if it’s anything like Elons retuning of (mechahitler) Grok, it would impact them negatively.
Joe Biden began his second term
Which timeline is it?
My best friend, in our late teens, once emphatically claimed that Eric Clapton wrote “I Shot The Sheriff” and that Bob Marley effectively stole the song from him. This was before the internet as we know it, so fact-checking took effort. He and I argued about this off and on for weeks. Until I wound up in a used record store and happened upon the Clapton album that had “I Shot The Sheriff”. Right there, plain as day, it stated “written by R. Marley.” So I bought the LP, even though I did not own a record player at the time, just so I could put it in front of his face and show him.
His reaction? “Well, I’ve seen a Cream album where it says he wrote it.” CLAPTON WASN’T WITH CREAM WHEN HE PUT OUT HIS COVER!11!
Similarly, my brother-in-law as a kid was quite assured that Elton John’s hit song was actually “Panty and the Jets” and refused to believe otherwise for years.
Both are pretty right-leaning guys these days and so maybe “confidently wrong” is just something that comes with a certain political persuasion? ChatGPT is just made in its makers’ image.
I don't think it's fair to say "humans are also flawed" in response to AI's flaws because the AI has never and according to research by the industry giants WILL NEVER reach human level accuracy.
It's a statistical model and it's percentages are lower than asking your dudes.
The knowledge cut-off for GPT5 is 2024 just so you know. Obviously, it would be better if it didn't hallucinate a response to fill in its own blanks. But it's software, so if you're going to use it then please use it like software and not like it's magic.
In general I'm not too moved either way when somebody misuses AI and then posts gobsmacked about how bad it is. Really though, the blame is on AI companies for trying to push AI onto everyone rather than only to domain experts.
This is the fundamental problem with LLMs and all the hype.
People with technology experience can understand the limitations of the tech, and will be more skeptical of the output from them.
But your average person?
If they go to Google and ask if vaccines cause autism, and the Google's AI search slop trough contains an answer they like, accurate or not there will be exactly no second guessing. I mean, this is supposed to be a PhD level person, and it was right about the other softball questions they asked, like what color is the sky. Surely they're right about that too, right?
If the knowledge cutoff for GPT5 is 2024 it should absolutely not be commenting on current day events and claiming accuracy.
This is not the defence you think it is. It still shows ChatGPT in an accurate and very negative light.
That's funny though because I know Copilot can google things and talk about them.
Like, a news story can appear that day, and you go "Did you hear about the guy that did X and Y?" and Copilot will google it and be like "Oh yeah you're referring to the news story that came out today about the guy that did X and Y. It was reported in Newspaper that Z was also involved" and then send you a link to the article.
So like... GPT5 should be able to supplement its knowledge with basic searching, it just doesn't.
Personally, as someone who prefers it when software only does things I direct it to, I'd rather that an LLM not automatically search for the answer online if I didn't ask it to.
OP is wrong and it's good that he admits it. Joe Biden is the actual president in 2025, and he will soon meet Steven Seagal the new Russian president about the war on Belgium.
Still my favorite video about Steven Seagal
It's doing precisely what it's intended to do: telling you what it thinks you want to hear.
Bingo.
LLMs are increasingly getting a sycophancy bias, though that only applies here if you give them anything to go on in the chat history.
It makes benchmarks look better. Which are all gamed now anyway, but kinda all they have to go on.
GPT5 probably has access to the real poll data before Musk helped to "confirm" it. The cheezeit himself said that Musk knows the voting machines better than anyone.
(this is the same mentality that Trumpers used when Trump completely lost the election in 2020, and I will be damned if it doesn't feel right)
Want more PhD level intelligence?
https://chatgpt.com/share/6898598d-fb18-800c-ab20-681788add81a
And my favorite: 🤣 https://chatgpt.com/share/68985829-03c4-800c-b261-7182f39d54c5
The ad placement is also perfect
Ais seems to have issues escaping characters, today mine was stuck in an infinite loop because it keeps saying it was gonna add a newline, but wrote nn
To give the (anthropomorphized) model credit, I think the biggest problem is that the way it's trained is has practically no concept between the relation of the token it spits out an the html code that will be produced by the ChatGPT UI wrapper.
Because of that inserting regular newlines often don't work because a single newline in markdown doesn't translate to a line break in HTML.
To force a particular structure I often ask it to use some will known formats like yaml. Because there is so much yaml training data, is practically impossible for the LLM to not add newlines at the correct places (which is also typically rendered correctly because it places that in a markdown code block)
Wonder what else it’s wrong about, if it misses somehing this obvious. It seems to be ChatGPT is getting worse by every update
Wait…
Take me with you
I wanna know how it breaks it down day by day. Is it gonna list every single day from the starting point?!????!!!! That be funny
Ha, even text prediction thinks Trump is a loser.
Honestly this is why you shouldnt trust statistical predictions on human behavior anyways.
4o did this too because the dataset is out of date. If you tell it Donald Trump is president now and was reelected it will generally search the web and correct itself. Sometimes you do have to insist to invoke the web search (and make sure web search is enabled, though iirc that’s default now). Not really a slam dunk, just them being cautious with updating current events into the model.
If you want to slam dunk gpt5’s intelligence just interact with it for a bit and ask it to do tasks. It’s pretty lackluster tbh. In my testing it out it really sucks at parsing language compared to 4o mini and competing models, a step backwards.
I tried to have it build a relatively simple pcb to drive a flashing led circuit. It kept getting confused because I would point out the schematics it spit out were piss poor and impossible to read because of basic shit like overlapping symbols and labels. Rather than fix that issue it started to erase necessary components and kept changing how everything was connected which fucked the logic. Eventually it just randomly introduced an arduino into the mix? This was just a basic led flasher with mosfet, diodes, and transistors, mcu and code was not necessary. I ran out of free tokens and gave up. 4o mini was able to do it (though while it couldn’t generate an image it described the connections correctly and made an ascii diagram that was correct)
Even just basic stuff: I asked it to translate an English phrase into Japanese and it did. I then asked it to translate a Japanese phrase into English by saying “translate (Japanese phrase)” and it did. I then asked it to translate an English phrase into Japanese by saying “translate (English phrase)” and it rewrote the English as English. This is a pretty basic one. While I didn’t explicitly write “translate (English phrase) into Japanese” it was obviously clear from context what I meant and at a minimum I wouldn’t want the phrase left in English given I asked for “translate”. Bad parsing.
Can't even pass the concussion protocol. Hey, maybe Altman meant it's as good as a PhD with CTE.
Fuck Reddit and Fuck Spez.
Gpt 4o mini does the same
User prompt 1 of 4 - 8/13/2025, 2:40:20 PM: For how long Biden has stayed president in 2025
GPT-4o mini: As of August 13, 2025, Joe Biden has been president for approximately 3 years and 7 months. He was inaugurated on January 20, 2021.
User prompt 2 of 4 - 8/13/2025, 2:40:35 PM: and for trump
GPT-4o mini: Donald Trump served as president from January 20, 2017, to January 20, 2021. His total time in office was 4 years.
User prompt 3 of 4 - 8/13/2025, 2:40:52 PM: so who is the president of the USA
GPT-4o mini: As of August 13, 2025, the president of the United States is Joe Biden. He has been in office since January 20, 2021.
I want to believeeeee
Yeah, except just like everyone else, the AI can't understand Trump winning. 🤷♂️
Yeah but again it hallucinates and tell you confidently a wrong fact
Yes, the "AI" is "just dumb".
OK but that’s not a question anyone ever would or should ask AI. If you don’t know the difference between web search and text transformers then you’re playing with tools you don’t understand well enough to use effectively
"Playing with tools you don't understand" is the definition of most LLM products.
Well, then maybe they shouldn't be selling this shit as a replacement for search engines, or making search engines worse to make us stop using them and use this shit instead, or shoving this shit on top of search engines to force us to use it even if we're trying to use a search engine, should they‽ 😤
Kitchen knife is used for slicing a chicken, not self-protection when you're in danger at your home.
No, seriously, there's nothing wrong with using AI as a search engine, as long as you have websearch option enabled and check the sources.
Maybe it is not up to date
That's not a valid reason to lie. If it does not have the information it should state as much. This post underscores one of the biggest issues with AI. It will confidently say whatever "statistically plausible" thing regardless of the actual truth.
Edit: in case there is any confusion, by "should" I mean in an ideal scenario where ai is able to be used the way people think they can currently use it. I'm aware that it's not really how ai works hence the remainder of the comment making note of the statistically plausible bit. AI makes factual errors on things that could arguably be answered with its current dataset (how many bs in blueberry etc) and this is not an issue with the dataset, it's a side effect with the way LLMs work. They are not reasoning machines. They are fancy algorithms. This makes them impractical for use in several areas where they're already being deployed and that's a problem.
I agree. The problem is it does not know that it is lying. As a whole, I would say it is our mistake to use it
You don't understand how AI works under the hood. It can't tell you it's lying, because it doesn't know the concept lying. In fact - it doesn't know ANYTHING, literally. It's not thinking, it's predicting. It's speculating what the viable answer would look like based on his dataset.
You don't actually get real answers to your questions - you only get a text that the AI determined may seem most fitting to your prompt.
Not up to date? It literally says that Joe Biden started his second term in January 2025.
There's nothing outdated about that, it's just flat out false.
yes. nail them down on training data which the model probably does not have. usually it searches the net. did you suppress this?
Assuming the training data is missing, why would it spew the nonsense that Biden started his second term this year?
This is a blatant hallucination, and until LLMs can express the simple fact that their data set is not sufficient, any output should be treated as pure fiction.
No, this is the full convo. You can also see the one from somebody else linked, no trickery involved. From what I’ve read it was fed training data until Oct 2024, thus the problem
Wasn't it already clear in October that it was Harris vs Turnip for prez?
The problem isn't that it was trained until last year. The problem, as always, is it is confidantly wrong as hell with its phrasing of it's non-thinking answers.
The saddest part about it is that it's still way smarter and better educated than its (human) users.
Whose data do you think was used for training?
All that is available. Garbage in, garbage out. Great for garbage people.
For those just joining us: The problem isn't that it doesn't know. The problem is that it confidently asserts a falsehood.
...And that people take the bait and anthropomorphize it, believing it is "reasoning" and "thinking".
It seems like people want to believe it because it makes the world more exciting and sci-fi for them. Even people who don't find gpt personally useful, get carried away when talking about the geopolitical race to develop agi first.
And I sort of understand why, because the alternative (and I think real explanation) is so depressing - namely we are wasting all this money, energy and attention on fools' gold.
If these things could not be induced to confidently lie, they would not be the target of billions of dollars of investments.
I sincerely don't think this is true, but it's a nice narrative that fits well with one of Lemmy's. It'd still be worth the same or more if it hallucinated to a minimum because it would better match one of its ideal business applications: replacing human labor at a fraction of the cost. Unfortunately, this is only a convenient side effect for many who stand to benefit from creating propaganda and false information in bulk.
And yet there are still people in the thread claiming that 'oh ChatGPTs knowledge data cuts off at the end of 2024, this prompt is using ChatGPT wrong' completely missing your point.
If ChatGPT doesn't know something it just lies about it, all while being passed off as doctorate-level intelligence.
Inb4 defenders 'an AI can't lie it just asserts falsehoods as truth because it's having a scary dream/hallucination' as if semantics will save the day.