Do Users Write More Insecure Code with AI Assistants?
Do Users Write More Insecure Code with AI Assistants?
This is fine... "We observed that participants who had access to the AI assistant were more likely to introduce security vulnerabilities for the majority of programming tasks, yet were also more likely to rate their insecure answers as secure compared to those in our control group." https://arxiv.o...
cross-posted from: https://programming.dev/post/8121843
~n (@nblr@chaos.social) writes:
This is fine...
"We observed that participants who had access to the AI assistant were more likely to introduce security vulnerabilities for the majority of programming tasks, yet were also more likely to rate their insecure answers as secure compared to those in our control group."
[Do Users Write More Insecure Code with AI Assistants?](https://arxiv.org/abs/2211.03622?
I think this is extremely important:
Furthermore, we find that participants who trusted the AI less and engaged more with the language and format of their prompts (e.g. re-phrasing, adjusting temperature) provided code with fewer security vulnerabilities.
Bad programmers + AI = bad code
Good programmers + AI = good code
59LLMs amplify biases by design, so this tracks.
28This. As an experienced developer I've released enough bugs to miss-trust my own work and spend as much time as I can afford in the budget on my own personal QA process. So it's no burden at all to have to do that with AI code. And of course, a well structured company has further QA outside of that.
If anything, I find it easier to do that with code I didn't write myself. Just yesterday I merged a commit with a ridiculous mistake that I should have seen. A colleague noticed it instantly when I was stuck and frustrated enough to reach out for a second opinion. I probably would've noticed if an AI had written it.
Also - in hindsight - an AI code audit would have also picked it up.
12The quote above covered exactly what you just said: "yet were also more likely to rate their insecure answers as secure compared to those in our control group" at work :-)
2I find that the people who complain the most about AI code aren't professional programmers. Everyone at my company and my friends who are in the industry are all very positive towards it
-4
Good programmers + AI = extra, unnecessary work just to end up with equal quality code
5Not even close to true but ok
0
A worrying number of my colleagues use AI blindly. Like the kind where you just press tab and not even look. Those who look spend a second before moving on.
They call me anti-AI, even though I've used chatGPT since day 1. Those LLMs are great tools, but I am just paranoid to use it in that manner. I rather it explain to me how to do the thing instead of doing the thing (at which it is even better).
EDIT: Typo
26Those LLMs are great fools, but I am just paranoid to use it in that manner.
Exquisite typo. I also agree to everything else you said.
5
ChatGPT can be surprisingly good at some things, but can also produce good-looking nonsense. The problem is that spotting those cases requires a certain level of knowledge of the subject, which makes the use of it kind of pointless. I personally use it for subjects where my knowledge is significantly below average, such as learning new frameworks / languages (e.g. React). It often gets stuck with more complex questions (e.g. questions related to x86 Assembly) or obscure subjects. I rely more on its ability to reproduce information than its problem-solving ability. I think the next development is adding LSP integration to the AI assistants and other tools to check its output.
However, I think most people don't use it the way I just described. A lot of people seem to mistake its ability to write code for an ability to understand code. It also sometimes uses older functions deprecated for security reasons, especially when using C. So yes, I think it will increase the amount of insecure code.
11Not even knowledge, attentiveness. It's so easy to overlook issues with AI written code vs writing it yourself and having to come up with the process. Just today i had this happen, cost me a day of extra work because i missed something in chatgpt's great looking code.
4
In a shock to literally nobody... Jokes aside, I am looking forward to reading this paper
8I'm not even sure how to utilize AI to help me write code.
6Also one really good practice from pre-Copilot era still holds, that many new users of copilot, my past self included might forget: don't write a single line of code without knowing it's purpose. Another thing is that while it can save a lot of time on boilerplate, you need to stop and think whenever it's using your current buffer's contents to generate several lines of very similar code whether it wouldn't be wiser to extract the repetitive code into a method. Because while it's usually algorithmically correct, good design still remains largely up to humans.
4There's a very naive, but working approach: Ask it how :D
Or pretend it's a colleague, and discuss the next steps with it.
You can go further and ask it to write a specific snippet for a defined context. But as others already said, the results aren't always satisfactory. Having a conversation about the topic, on the other hand, is pretty harmless.
3Copilot or Tabnine are the two major ones.
They're awesome for some things (especially error handling). But no.. AI will not take over the world anytime soon
1
Good programmers - AI = best code.
2