This was pretty much inevitable - there is so much insecure code out there from practically every era of computing. There’s only one answer, and that is giving developers and security experts the resources (that includes LLMs) and time they need to ensure that all this legacy code is being fixed or replaced with better structured code that is less attackable.
It will be a wild ride for a while, and then things will slow down again when the well of not-yet-discovered exploitable bugs starts to run dry.
I’m quite staunchly anti-LLM, but code review is where I think they are most suited. Mind you, only as PART. But their ability to hold large bodies of well-structured code in context is what gives them the edge to find possible bugs and exploits a human has overlooked.
They never should’ve been used for code creation, and in reality they’re being use on both sides of the equation going completely unmonitored or, at best, there’s a burnt out human sitting in between rubber-stamping commits because the entire pitch for this shit is to increase productivity (output) and move at a rate designed to outpace them.
AI Is Breaking the Entire Structure of Software That Powers the World
On both sides even - both by better automating brute force hacking strategies AND by generating weakness-riddled code for network-facing software.
What a great time to be alive!
The only ‘AI’ at play is the variant where LLMs infect the Peter Principle: Amplified Incompetence.
Don’t believe the hype.
The truth is nobody ever has time to test things thoroughly. Proprietary software has to ship early due to unrealistic deadlines. Bad specs, client doesn’t know what the fuck they want. Open source dev doing weekend hobby coding suddenly getting pressured by billion dollar corps for features while trying to balance mental health, family, pay bills etc.
So we take shortcuts. Ship code. Hope for the best. Better compiler warnings, linters and strict coding standards can help but only so much.
LLMs let security researchers search for classes of bug that have been harder to find with other methods. Lots of low hanging fruit for them. Most of the vulnerablities are low impact and over hyped. Once they find them and they are fixed they aren’t going to find the same exact one again. The security stuff will get better. Unfortunately not so much careers.
Nobody has time to test things thoroughly because the fastest code that looks good succeeds because we looked at “the ui is the product” not as a problem to fix but as a customer to be appeased and now we’re paying for it.
Software engineering needs to become a licensed profession like any other with professional engineering standards for efficiency and safety. It’s long over due.

It is already broken. Just the cracks come to light.
Waiting for the reveal of the first serious bugs in AMDs and Intel’s management engines. Those tiny OS running on their own core next next to the user’s OS. Or the firmware of the network cards. This will be a fun day for users of older PCs without the chance to receive patches.
IT has been cut to bare bones for a long time, now. Some VP made their cost cutting bonus 10 years ago. Their successor did the same. And their successor. THIS is the result. This is our culture. Make the rich richer now no matter who suffers later.
I work in IT security and I’ve buried many colleagues and friends that died from over-stress. Usually their hearts give out. My best friend died last December. Not that rich C suite executive, though. Now there’s no one left to work late and fix all the problems. They’ve deputized whole divisions to the patching and vulnerability teams, just to stem the bleeding so the companies don’t end up in the news. But as soon as we fixed one, another comes up. I’m not speaking in abstract terms - that was literally my Wednesday. We can’t get ahead of it. A concentrated cyber attack by AI that doesn’t get tied, bored, or blocked will eventually succeed. Our legacy code running Java dependencies on Windows 95 isn’t getting replaced because it’s not in the budget.
I just hope they don’t go for infrastructure.
I’m sorry for your loss, comrade.
They will go for infrastructure. There are bad actors and ai capacities enough to attack every system on this planet.
Every system.
AI isn’t breaking software by itself. It’s revealing just how much of the world runs on code held together by hope, legacy dependencies, and one exhausted maintainer. The exciting new feature is that attackers can now find those cracks for $200 while the security team is still waiting for budget approval.
That was always going to happen. If one supplier offers bulletproof software but demands a NASA budget and timelines and the other one offers to do it for pocket change and promises, the latter one is chosen.
Yes it is, it is making programmers leave, and it is preventing new programmers from learning the basics of a project.
My wife just told me she that Debian is losing some of it’s most experienced programmers, because code submissions are scanned by stupid LLM that as silly questions about the submissions that MUST be responded to.
Disrupting maintenance absolutely breaks software, and AI is doing that all by itself.
There aren’t even any $200 saved, as Debian is maintained by contributors.And security by obscurity. Keeping source code secret to hide your failures.
Yep, in the age of AI code decompiling, secret source code isn’t going to help you much.
Eh if they can get the binaries off your server somehow, they don’t really need to decompile shit, they can just test their own instance of it
Decompiling is still usefull. It’s easier to find problems in the source code than blindly probing.
A lot of places are using off-the-shelf software that attackers could just buy/download a copy of from the vendor.
this is JAVA owned by ORACLE jesus fucking christ. AI is helping FIND BUGS because well paid well resourced humans make mistakes too.
the luddite lemmings strike again
Refusing to acknowledge that AI helps every sociopathic script kiddie do real damage to the real world doesn’t help either.
This is why you need strict quality control whenever you have AI making things.
Cybersecurity experts need to thoroughly examine all the code and how the software works…or doesn’t work and help fix it
Then you need ethical hackers to test the security and you need the cybersecurity experts to fix the problems the ethical hackers find.
it’s boring, grueling work but if you don’t do it everything falls apart
This is why you need strict quality control whenever you have AI making things.
The article is about AI finding exploits.
sure, but you can’t just patch shit Willy nilly without also breaking things.
doing things correctly takes time and money, neither of which we have time for when we have the poor, defenseless, baby bird shareholder to take of!!
I mean i agree that proper quality control and review is needed.
Its just that the conclusion Zeus takes from this article is not really corroborated by the article.
It’s funny how the narrative until this point has been that AI is too crappy at coding to be useful.
In my experience, AI is really good at finding problems because of the large context window and speed. But it is really crappy at writing good code, because it can’t distinguished good code from bad. The way AI writes code is just to generate stuff and fix problems until all problems are gone. That is a terrible way to develop.
generate stuff and fix problems until all problems are gone. That is a terrible way to develop.
True, but let’s be honest, that’s also how a LOT of human programmers do it.
oh nooooo









