Microsoft's Pricey AI Assistant Copilot Leaves Early Adopters Feeling Cheated
Microsoft's Pricey AI Assistant Copilot Leaves Early Adopters Feeling Cheated
The enthusiasm of early adopters waned when they encountered instances of Copilot "hallucinating" inaccurate responses.
Copilot isn't actually bad for developers, it's just that you need to be careful with it and recognize its limitations.
Writing a bunch of REST endpoints for an API and need to implement all the typical http verbs, and you already have all the matching methods for reading, updating, and deleting values in a complex SQL database for each endpoint to call? Copilot can turn a ten minute chore into a ten second one. Very handy.
Writing those complex SQL methods in the first place? Yeah... Copilot will probably make a ton of mistakes and its work will need to be triple-checked. You'll save time just doing it yourself if you know how. (And if you don't, you have no business calling yourself a developer.)
Copilot is best for easy boilerplate and repetitive code. Problems arise as soon as you ask it to get "creative."
61This is about the other copilot.
45Do you mean a revision or a different "copilot". If the latter then this confusion is brought upon by Microsoft, the company that names a successor to a gaming console "Xbox One".
8Yeah, I figured that out eventually, but also figure the same probably applies to the other Copilot. Same underlying technology.
Wish Microsoft would use different names for different implementations.
5
One time I decided for shits and giggles to just keep pushing tab and see where it went. It didn't take long for it to enter a useless recursive loop, hallucinating a new iteration of the same thing on each line.
It definitely isn't gonna magically think up new algorithms for you. I don't know what everybody is scared of. It ain't even gonna replace my kid programming on Scratch.
14I mean didn't we all do this when phones started autocompleting sentences like a decade ago? (Or however long it was, time perception is fickle)
3It will if employeers only want ai code
1
Copilot isn't actually bad for developers, it's just that you need to be careful with it and recognize its limitations.
Is it me or is this a weird statement for what's supposed to be an exact science?
Imagine working in construction and using a level and you're told "it's not that it's a bad level, you just gotta be careful with it".
How much margin for error should we allow for getting our code right? Is it now acceptable if we only get 80% right?
9It's more like you get some kind of weird construction multitool that promises to be a level, a drill, a hammer, and a dozen other things, and it turns out to be a really good, innovative, and helpful level... and a really bad everything else.
7I use copilot a bit for my work - and I treat it like copy-paste from StackOverflow - sure that codeat look right, but you've gotta double check it and test it a few times before you commit and push.
5As a software developer I promise you that software development is very much not an exact science.
Programs are complex and there are so many different ways of achieving the same thing that all code has problems and gets a bit messy in places. You can test, but it's not easy to ensure that everything works the way it should.
The best code you're going to get will probably be in the space industry, but even that will have bugs. The best you can do is make the code robust even when bugs make things go wrong.
In many cases copilot will do just as well as a junior developer. It's very good at repetitive tasks and filling gaps in your existing code.
4
Always ask it to write tests for the code it generates. Of course, then you have to validate that the code works AND that the tests work.
4Or just write the damn thing yourself and save a bunch of headaches and wondering if you got the tests right it if there’s some screwy corner case lurking because of its implementation.
0
It’s funny that Microsoft fucked up the branding so bad that half of these comments are about GitHub Copilot — a specialized, useful tool for experienced developers to speed up rote tasks — and the other half about their whole “Shoehorn AI into everything.” strategy. And Bing Chat is also now Copilot, apparently? Is Cortana now Windows Copilot? Is there an Xbox Copilot that plays Starfield for you?
46Remember that internal Microsoft video that leaked where a marketing person was begging them to use Apple’s iPod box designs instead of their usual ones: https://youtu.be/EUXnJraKM3k?si=3aAZ9xulggtHDVp6
It was apparently 17 years ago but it’s still true that Microsoft can’t brand things for shit.
12Here is an alternative Piped link(s):
https://piped.video/EUXnJraKM3k?si=3aAZ9xulggtHDVp6
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I'm open-source; check me out at GitHub.
2
There is a setting for Xbox Controllers to use two of them like it's the same one called Copilot.
5Co Xbox Windows pilot
3Co pilot pilot for Microsoft Flight Sim 2025
4
Well copilot is the category of those assistants. Most other companies call them copilots too, even in B2B space
2
Similarly, other users told the outlet that the AI hallucinated wrong answers or miscalculated spreadsheets. AI experts, including The Wharton School professor, Ethan Mollick accused Copilot of making bizarre suggestions for weekend meetings.
It seems these users never used GPT
45It's because it's not as good
30... In contrast, Microsoft's corporate VP, Jared Spataro, told us that users are finding immediate value in Copilot
haha no really?
30Guy who's job it is to sell a product, lies to sell a product.
31
What part of "early adopter" did you not understand before you wasted your money?
24So people are surprised that GPT-4 is performing as well as GPT-4 always has?
24As far as I am concerned Copilot is a giant theft of open source code and breaches the license. I expect in the future a lot of repositories will be used to poison these AI agents just as is happening with images. The agents will get better but the quality of what they produce will also be poisoned and get worse precisely due to the theft.
10Poisoning code should be ludicrously easy: They crawl pretty much everything and a random AST walk looks suspiciously like real code while it's the equivalent of showing an image generation model noise. Or maybe better: Mondrians that are indistinguishable from Vermeers. (I hope I didn't offend anyone by calling Mondrian abstract nonsense but it is abstract nonsense).
I don't think copilot will hold out for long anyway, the novelty is wearing off and even inexperienced programmers are beginning to see that it helps you write code faster that shouldn't have been written in the first place. Code is like 90% maintenance and excessive boilerplate doesn't make it easier.
OTOH please don't let that "Let's scam artists by selling them snake oil that if it wasn't trivial to circumvent would break naturally within a week" guy fool you. On the actually interesting side of poisoning attacks, people have made cars hallucinate radar blips I bet a couple of companies are getting quite tough questions from regulators right now.
7
Yeah it's a classic case of Microsoft marketing. So far I found the office integration to be the least useful and most over hyped in marketing. However what it is good at is actually helpful. Join a meeting late it already has an update for what's happened on the meeting so far and it's really good for summarizing a meeting especially a key topics and a summary of action items. Tedious tasks like taking data copied from a PDF file and reformatting it correctly in CSV. And my favorite is making custom graphics based on a specific colour palette, though most images are really good for entertainment, demos and samples, but not production quality for final products. Weird results include creepy human images just don't look right in a disturbing zombie-like way.
10It's ChatGPT Open AI, but as Microsoft LoVES to do things, bloated as those old time web browsers infested of tool bars
7It is multiple things, in chat it is just GPT4-turbo, with support to request data through Microsoft Graph API. So basically it can do API searches to data you have access.
In Excel, it can perform things like formulas, with context of your excel sheet. This is actually quite useful.
But still bit disappointed, the system prompt of copilot is in my opinion not as good as open AI's.
0
I have not had a chance to test the office integrations (and the $30 price tag may keep it that way,) but I’ve been testing the web chat version, and once toggled to “precise” mode I find it close to chatgpt. Just a little more prone to lapse into acting like a bing chat app and very limited in conversation length…
I think the trick is to treat it like a junior assistant or maybe an intern who might make mistakes and not as a seasoned, experienced employee who always puts out perfect work.
7$30? Idk how it is in the consumer space, but in the Enterprise world the initial opening for copilot required 300 seats of E3 or better, and then a purchase of 300 seats of copilot 365 at $30 each. They were supposed to drop the 300 copilot 365 seat req q1 but I'm not sure if they did.
3030012=108,000 per year for copilot. E3 is 36 a month, 30012=129,600.
Big money.
8
Yes, but when will it make porn?
6They already have porn chatbots.
4The bosses are airborne while I can’t work online, so I generate my porn on the company’s time.
3
Trash company once again dumps trash on the market and attempts to sell it to morons.
5This is certainly not about finding fools who not only are beta testers but also provide training data... And not only for free, but they pay also for this.. Win win win for M$
4My favorite are the gigantic companies falling all over themselves to buy this for their workforces before it's even clear what value a bot that can't reliably do arithmetic is going to add to Excel
3According to the article, dow says they've seen significant productivity gains and are rolling it out to half the company.
1
[redacted]
1