ChatGPT use declines as users complain about ‘dumber’ answers, and the reason might be AI’s biggest threat for the future

The free version of ChatGPT DEFINITELY is dumber than it was even a couple of months ago. Used to be able to get decent, useful code reviews out of it, now it barely knows how to write a nested loop anymore.

It’s storytelling capabilities fell off a cliff too, the drive towards safely sanitized unoffensive-at-all-times content it can output has rendered every story, choose-your-own-adventure or collaborative role playing game sterile, empty expressions of black and white stories with no nuance allowed where saintly goodness is the only choice possible

In my own experience, chatGPT has been massively nerfed for the use cases I used it for

Back in my day, we used to call ‘prompt engineering’ ‘asking a question’.

I asked it to review a function I wrote and it suggested that I rename create_archive() to create_archive() so that's it's less ambiguous than create_archive().

So yeah, it's definitely dumber.

I think its own popularity has been its downfall. Before, they were willing to waste money on servers and electricity to advertise ChatGPT, but the time has come to make it profitable and they have to tone it down to meet both increased demand by the users and the demands of their own sustainability.

It's getting worse based on the feedback unfortunately, the need for safety and lack of meaningful deliberation towards how AI companies should operate and what should and should not be done has led Sam and co to be indesicive towards doing anything. Alongside the "morality" of the thing being hyjacked has lead to other AI's performing better... lead by x employees of OpenAI, with actual bound morals and not inherently relying on user input to train future models, this will be the path forward, this will lead to safe and controlled integration.

I guess at the core of this, we are afraid of ourselves. We are afraid that the worste of humanity outpaces the better parts, that the inputs and training aren't altruistic but are more pointedly "bad" or "wrong", and thus leading to "harmful", whether through misinformation, lies, or fabrications.

I hope we find a way to do better. I'm still excited for the future of AI, I mean crap, I'm closer to having a family doctor that's a robot then I am to a real human doctor.

AI cannibalism simply isn't a thing yet. It definitely will be and good models will need to spend a lot of time and money sourcing good training data, but the models are not up to date enough to be contaminated yet.

I'm very confident the degradation has come from them trying to scale up. Generative AI is the most expensive thing on the cloud you can provide, and not only are they trying to make it faster, they're trying to roll it out for way more consumption. Major optimizations will require an algorithmic breakthrough so in the meanwhile all they can really do is find which corners they can cut that are less bad.

When reality catches up to marketing

Nonsense. Less people are using it because there are viable alternatives and the broader novelty has worn off.

I use it every day in my job and the quality of answers only drops off when prompts are poorly crafted.

By and large, the average user doesn’t understand the fundamentals of prompt engineering.

The suggestion that “answers are increasingly dumber” is embarrassing.

So what are the fundamentals in prompt engineering?

Article talks about the potential of AI cannibalism were it is now learning from data that it (or other AI) has generated.

Does ChatGPT use modern data I was under the impression that it's most modern dataset was a few years old

ChatGPT usage is a very poor metric. Anything interesting is happening via API. Even the chat completion endpoint still isn't "ChatGPT" on its own. None of these complaints about it being "dumber" apply to the API outputs. OpenAI don't care about nerfing chatGPT because it's not their real product.

As long as it continues to do my resumes for me that’s all I need lol.

I feel like it is still too early to talk about "AI cannibalization" or "feedback loops" as that would mean that a big proportion of the training data is AI-generated content itself, against all the rest that could be scraped off the internet or the public domain, I don't think this is happening yet.

What people might experience instead, and perceive as dumbness, is that given that the datasets used to train AIs cannot really change that much in a short time (unless we wait for another hundred years so humans can produce actual human original content to train the AI again), and as the mathematical models used to build answers based on the datasets are pretty much the same, a person talking with ChatGPT will over time perceive more and more that the answers are built using a "pattern" or a "structure", aka the model derived from feeding the dataset into the AI training itself.

Just my pennies on this, let's also consider that is in human nature to be excited for something new that sounds cool, and then to get bored when you got accustomed to it and pushed it to its boundaries.

Why is it relevant what Peter Yang - Roblox product lead and enthusiastic child labor exploiter - tweets about it? Let me guess he's a prompt engineer?

Permanently Deleted

I had my first WTF moment with AI today. I use the paid Chat-GPT+ to help me with my c# in Unity. It has been a struggle to use, even with the smaller basic scripts you can paste into its character limited prompt, as they often have compile errors. That said if you keep feeding it the errors, guide it where it is making mistakes in design, logic etc. it can often produce a working script about 60-70% of the time. It takes a fair amount of time quite often to get to that working script but the code that finally works is good.

Today I was asking it to edit a large c# script with 1 small change that meant lots of repetitive edits and references. Perfect for AI, however ChatGPT+ really struggled on this one which was a surprise. We went round and round with edits and ultimately more and more errors appeared in the console. It often ends up in these never ending coding edit loops to fix the next set errors from the last corrected script. We are taking 3 hours of this with ChatGPT+ finally saying that it needs to be able to see more of my project which of course it cannot due to many of its input limitations including number of characters so that is often when I give up. That is the 30-40% that do not work out. Real bummer as I invest so much time for no results.

It was at the movement so gave up today that a YouTube notification popped up about how Claude.ai is even better than ChatGPT so I gave it the initial prompt that I gave ChatGPT above and it got the code right the first time. WOW!!!

Only issue was it would stop spitting out code every 300 or so lines (unsure what the character limit is). To get around this I just asked if it could give me the code from line 301 onwards until I had the full script.

Unsure if this one situation confirms coding with Claude.ai is better than ChatGPT+, but it certainly has my attention and I will be using it more this week as maybe that $20/month for ChatGPT+ no longer makes sense. Claude is free with no plans for a premium service it said. Unsure if this is true as I have not spent anytime investing it yet, but I will be.

Surely the rampant server issues are a big part of that.

OpenAI have been shitting the bed over the last 2 weeks with constant technical issues during the workday for the web front end.

It definitely got more stupid. I stopped paying for plus because the current GPT4 isn't much better than the old GPT3.5.

If you check downdetector.com, it's obvious why they did this. Their infrastructure just couldn't keep up with the full size models.

I think I'll get myself a proper GPU so I can run my own LLMs without worrying that they could stop working for my use case.

error loading comment

Maybe I’m not as adept at prompt engineering as the next guy, but at what point are we going to start /thinking/ in prompts just to get a half decent answer?

I’ve definitely seen GPT-4 become faster and the output has been sanitized a bit. I still find it incredibly effective in helping with code reviews where GPT-3 was never helpful in producing useable code snippets. At some point it stopped trying to write large swaths of code and started being a little more prescriptive and you still need to actually implement snippets it provides. But as a tool, it’s still fantastic. It’s like a sage senior developer you can rubber duck anytime you want.

I probably fall in the minority of people who thinks releasing a castrated version of GPT is the ethical approach. People outside the technology bubble don’t have a comprehension of how these models work and the capacity for harm. Disinformation, fake news and engagement algorithms are already social ills that manipulate us emotionally and most people are too technologically illiterate to see how pervasive these problems are already.

I was talking about it a month ago - others made fun of me.. 😂

I think you've nailed it though. We are very well versed toward documenting the details or such atrocities; we don't pay the same tribute to the good done by humanity. And this is certainly evidence that just "letting loose" and AI without clear and static "morals" is a bad idea.

The people who complain about how they no longer can get answers on how to eliminate juice in the style of Hitler are people who are - to be honest - completely missing the point of this revolution.

ChatGPT is the biggest developer productivity booster I have ever seen and I spend so much more time writing valuable code. Less time spent debugging, less time spent reviewing, etc. means more time for development of things that matter.

Each tech company who just saw massive growth over the past 10-15 years have just received a new toy which will multiply their developer's outputs. There will be a clear difference between companies who manage to do this will and those who won't.

It's irrelevant if I can get ChatGPT to write a poem about poop or not. That's not the goal of this tool.