AI isn’t ready to replace human coders for debugging, researchers say
Even when given access to tools, AI agents can’t reliably debug software.
You're viewing a single thread.
As a very experienced python developer, I have tried using chatgpt for debugging and vibe coding multiple times and you just end up going in circles and never get to a working solution. It ends up being a lot faster just to do it yourself
Absolutely agree. I just use it for some simple stuff like "every nth row in a pandas dataframe slice a string from x to y if column z is True" or something like that. These logics take time to write, and GPT usually comes up with a right solution or one that doesn't need a lot of modification.
But debugging or analyzing an error? No thanks
I have on multiple occasions told it exactly what the error is and how to fix it. The AI agrees, apologizes, and gives me the same broken code again. It takes the same amount of time to describe the error as it would have for me to fix it.
This is my experience as well. Best case scenario it gives me a rough idea of what functions to use or how to set up the logic, but then it always screws up the actual implementation. I've never asked ChatGPT for coding help and gotten something I can use off the bat. I always have to rewrite it before it's functional.
My rule of thumb is, if he doesn't give you the solution right off the bat he won't give you one. If that happens either fix it yourself or start a new chat and reformulate the question completely.
Thank you! So many morons saying you can just use Generative AI to build whatever you need. That’s a no..
