Accenture sells AI transformation to thousands of clients while quietly telling its own employees they get a fraction of what some were already spending. The internal memo reveals a tension at the heart of every major consulting firm racing to profit from AI.
Same. AI has fallen down repeatedly trying to analyse both of my repos. And they were written by a bad human originally. The problems are all the same, except AI will get you spaghetti faster.
Messy brownfield code requires a lot more meta-analysis from previous sessions, just to get all of the spaghetti code into a manageable set of rules and documentation.
Don’t just send the agent blindly into the code. Every session, the agent is just going in fresh without any context on how the code works. Break the codebase down to a 10,000 foot view as a summary, and then get more documentation and agent rules for each of its major sections. Good tests help a great deal, too, and if there aren’t tests, start building them. Start creating skills just for the codebase, or any common tasks you’re asked to do against it.
Also remember that you don’t have to write any of this stuff. Have it write the docs, rules, skills, tests, and just review it to make sure it looks right.
That is a pitfall that is difficult to mitigate. When using these models your own code base is the primary input, and it will match the style. Garbage in, garbage out. It may be worthwhile to refactor prior using a model. Even having intelligible method and variable names can make a difference. Maybe do a sweep of simple renaming, and adding supplemental explanatory comments, then you might get better results?
Yours is a good point. I know that the guy who solo-created Redis uses AI effectively to write tests - he gives it a lot of handwritten tests and a lot of context, and then asks it to match the style and pattern, which it does well.
I’ve got something like half a million to a million lines, a crazy level of cyclomatic complexity, and a team that doesn’t see a problem with it. That’s the tough part - socialising the changes. We tend to arbitrarily rename vars/methods and not keep things in the same place, so there is a lot of drift.
I think AI might be able to unpack some of that, but there is a sort of irreducible accidental complexity that would require a human to make decisions about what some things mean. In some cases, you could get a gaggle of devs in the same room and they would not be able to arrive at a decision about what the code is doing or even what it should be doing.