The Economic Benefit of Refactoring

I find it funny how the best practices for programmers, ignored in most IT companies, get reinvented as the best practices for AIs.

Boring: The documentation should be in code, not in external Word documents uploaded to the company SharePoint server.

Exciting: The documentation for the AI should be in code, not in external Word documents uploaded to the company SharePoint server.

Boring: You should give your developers the big picture of the project, not just micromanage them using Jira tasks.

Exciting: You should give your AI the big picture of the project in CLAUDE.md, not just micromanage it using prompts.

Boring: Refactoring makes your developers more productive in long term.

Exciting: Refactoring makes your AI more productive in long term.

This is such a nice piece, this is how people should write about AI. Specific, grounded to how the tools are actually being used, and quantitative. There is so much bad AI commentary that is incredibly vague, divorced from any actual use cases, or written by people who don't actually use the tools. It is good to see a critique that is "here is a thing AI is bad at and measurements to show it" rather than gesturing "here is why I think AI is problematic for society."

It is rather different but another piece of research I liked for the same reason was this report that interviewed Boko Haram members about how they used AI to assist terrorism. You get these interminable online debates that are so unproductive and reporting that is specific is such a breath of fresh air.

https://casp.ac/reports/ai-enabled-terrorism

I really enjoy refactoring. Like, doing it by hand (5 miles uphill in the snow both ways) rather than AI. I don't even understand why I enjoy it, because if done right, there's no visible change. When people ask me why I'm so pumped up after having worked on the codebase all day long, I can't give them any sort of answer that makes sense (coming from a small business with a smaller dev team). It's just... "I'm future-proofing our website and we won't see any direct results today but things will be so much easier going forward..."

There's something about the puzzle. Looking at my old, deranged coding workarounds that tried to solve problems that have already been solved a thousand times before with established paradigms; and then moving them toward said best practices; and doing it in a way that no NEW technical debt is created. It's just satisfying.

I think one of the best learning experiences for me has been the fact that I created a bunch of sloppy shit by hand, auth and all, which forced me to learn things the hard way. All along the way, people were shouting from the rooftops: "use established libraries, dummy!", which is the same advice I would give to somebody today. But by doing things the hard way, I learned so much more about the inner workings. And, I've also given myself a decade's worth of refactoring work, which I really enjoy!

I think this is one of those things where a human in the loop is indispensable

An agentic refactoring pass does make sense cause one LLM reviewing work can spot things the ‘generator’ LLM missed while focused on the initial task output

But can the reviewer agent ever actually have in mind what this project actually is? And how the code all comes together to do the work involved? In other words what parts of the code are redundant or can be made more elegant

Asking coding agents to refactor your codebase is maybe like asking trauma surgeons to increase your exercise capacity. The agents are gonna need a really holistic POV to do this properly

I guess part of my point is that just splitting big files into multiple files is only refactoring in a superficial sense without having a theory of what code belongs together and what can be extracted into utility functions etc. Is splitting files actually like decomposing factors or is it like splitting a larger number into smaller numbers that still eventually get added together

A good example of what I mean is that agents often don’t ~actually~ understand the whole system anyway. They might implement a system to store and calculate something that is already being fetched via API. Humans often have a dual perspective — a holistic sense of the project and (when applying our mind to a task) a precise scalpel: ‘oh if we just look at this this JSON it has a key with this data already’

    "enhance"

The benefits go beyond reducing token consumption. Compact contexts also foster better reasoning, enable intelligence across more layers if you can load them in a single context, and writing software to enable this, results in more correct software, software that _generalizes_, that has higher probability of being correct not just for tested cases, but for the interpolations and extrapolations of these cases.

Refactoring towards good abstractions is more powerful than people realize. There's information theoretic bayesian math to back this up.

It's a bit of a divine coincidence that software that is more economically and energy efficient to process and run tends to also be more correct.

It's all about reducing the entropy of your code. https://benoitessiambre.com/entropy.html

It's very interesting to have some data on this.

It matches my experience which is that LLMs greatly benefit from well factored code, but are not particularly adept at creating such code.

Much like most human developers I suppose!

the punchline: "Refactoring reduces token consumption"

I appreciate the effort to quantify the benefit rather than pontificate. It's worth mentioning Martin Fowler wrote a whole book on refactoring [1], in which he states, "to refactor, the essential precondition is [...] solid tests", which I think is the real benefit here, AI or not. Good tests protect against regressions, whether human or robot. They also help encode the spec, which humans and robots can read.

[1] https://www.oreilly.com/library/view/refactoring-improving-t...

Great piece, but it misses the elephant in the room: the lion's share of economic benefit from refactoring will come from the fact that it makes it easier for humans to understand. That means 3am pages get resolved faster, fewer bugs will end up in prod, and your team can ship faster than your competition - gaining a leg up in the market. Most importantly, folks will be more comfortable accepting responsibility and ownership of a system when they understand it... this means that if/when something goes wrong, people will more quickly jump in and fix it, and when things could be better, folks will jump in and improve it.

Agent code can only be read and understood by agents. We’ve reached the point where people just can’t comprehend these giant code blobs.

Feature, bug or emergent property?

I don’t think the distinction matters as much as the reality.

We’re being locked into using the AI tooling bc the code was generated with AI tooling.

These giant files of doom were being generated by humans anyway and were very hard to work with. With LLMs it’s at last manageable or feasible to edit, refactor etc.

I honestly think LLMs are going to save us from ourselves as the codebases became too large and “messy” for humans to comprehend. (Mono repos of doom)

On a personal level these giant files are abhorrent but that’s just personal taste and I don’t think any of the Martin Fowler refactor/cleanup stuff is going to matter at all anymore. Kinda sad on some level.

Interesting that the amount of code remained essentially unchanged. In my experience it is not unusual that refactoring messy code cuts the number of lines in half.

Tangent, but I have a theory that refactoring is one of the best symptoms of a healthy dev team.

It's partly that refactors themselves have benefits, but I think more that the benefits to refactoring aren't visible to something like product-owners, feature tickets, etc.

If teams are refactoring to ensure the health of the overall software, it's a tell-tale sign that developers are happy making recommendations for good software, and that those recommendations are being taken seriously.

I think Martin Folwer might have actually coined the term "software rot" - either way, as an issue it happens most severely when a team either aren't motivated or empowered to build their vision of high quality software. When a team can follow their judgement of excellence, that's usually a great sign!

(and yes, obviously this can go to far, there are probably some teams who rewrote all their stuff in Ruby then Node then Rust and now something else to be "agent native", but in the coorporate world, I see a lot less of this than teams who just don't feel like they have permission to improve things)

It’s interesting that cyclomatic complexity or cognitive complexity scales with token usage. A codebase that enforces upper bounds on one of these two also (potentially) helps AI agents stay efficient.

> Add a new ItemWatchStore public async trait to the Firestore layer, following existing patterns exactly. The trait must have three methods:

async fn watch_item(&self, item_id: &str, user_id: &str) -> Result<()> async fn unwatch_item(&self, item_id: &str, user_id: &str) -> Result<()> async fn watched_items_for_user(&self, user_id: &str) -> Result<Vec<String>>

This shows the limitations of vibe coding. It takes someone with a long history of software development to prompt for something like this.

Even though the model is writing 100% of the code, still needed someone with a lot of programming knowledge to write the prompt.

This is interesting, but I think it would be even more interesting to see a comparison of the token cost of adding a new feature, in the original codebase vs the refactored one. You'd think/hope that the cost of adding the new feature would be lower when starting from the refactored codebase, indicating that refactoring makes economic sense in the long run by lowering the cost of adding new features.

Can't agree more. Feels like LLMs are not able to understand your requirements at a high level and always add complexity by default. Really need a handbook to guide them to do those cleanups. Obviously they know the knowledge in their weights but they just don't know how to apply it.

Interesting take-away:

> Claude is unable to look at code, look at refactorings in general and work out which are suitable to apply: a human needs to actively guide it.

Claude is happy to produce a very large Rust file. But you need human guidance to make it smaller.

The author said that Claude was not good at refactoring. But the author was using Sonnet 5. I supect Opus 5 would have done a lot better at refactoring. It would have come up with a multstep approach, i suspect.

Opus/Fable 5 (or really, any thinking model) do a better job of refactoring than Sonnet would. I think the takeaways are still valid, just not as valid when using a more expensive model at a higher effort level.

I can very much relate to the experiences described in the article. In my projects, I have multiple Claude skills and rules aiming at making my files (the code, but also the Claude files themselves) more token-effective. I have seen huge differences in token usage before and after invoking those skills, which I regularly do as part of having Claude audit my projects; although I haven't measured them. Those projects are (like the one in the article) fully developed by Claude Code, so I found such audits and mindful token usage very necessary.

I think the name of the game is going to be "how do we maximize output while keeping token counts low". For example, finding ways to replace AI workflows, even if minor ones, with scripts/code.

Anthropic told me that I have till August 19th to be get my act together becuase they are going to reduce my token count by 50%. lol. I have been abusing my Pro Max allowance and need to start being less wasteful.

Articles like this, can help us come up with ideas on how to do it.

I have a blanket, unignorable 'files must be <= 1000 lines' lint for my Rust projects for exactly this reason. Exploring large files is _costly_ for agents, so it's much better to lean into the filesystem hierarchy to a greater extent than you would normally with a team of human developers.

>The goal of refactoring an agentic code base is to spend tokens now in refactoring to make token consumption for future work lower.

Couldn't have said it better myself. Deciding to refactor should be like a discounted cash flow analysis but for tokens!

Was hoping for a process diagram on how refactoring removes bottlenecks during software developement and allows you to ship faster etc :)

Still a nice writeup and love how these meta analysises (presumeably) done via AI can now easily capture metrics that inform your workflow.

Great point. Using good software architecture and clean code will deliver direct and measurable economic value to projects and reduce AI costs right from the start, whether during refactoring or when planning a logical reorganization.

> and used tiktoken to approximate tokens, by dividing character count by four.

This is just the type of thing that stands out.

You used a tokenizer library to determine the number of tokens in your text by dividing its length by 4.

That makes no sense!

Make it work, make it right, make it fast, in that order

Make it right is the refactoring

> Claude is unable to look at code, look at refactorings in general and work out which are suitable to apply

I am just sitting here waiting for Grady Booch to write "Architecture!"

I wonder, how close can we get to a reasonable simulator for testing software engineering practices before inflicting them onto the real world?

> Claude.ai was better than Claude Code

Why is this something users have to distinguish? Why can't these generative AIs choose good names for things?

Putting this in agent.md didnt help?

"always give ultimate decent code in term of maintanibility, cleaness, and perf".

Being a bit cheeky here -- the amount of comments on this post is a telltale sign of how common / desirable refactoring is for the majority of developers (xD). It isn't our fault though, maybe if those pesky managers read this article then they would understand..

You can tell this is BS because there's no way that app is actually functioning at all. Fully vibe-coded, all that? No, I don't believe it.

I've been experimenting with not only refactoring but rewriting history. I'm always moving around commits, inserting, splitting things up. The amount of times I've instructed to "tell the story right the first time"...

More quantitative and qualitative analysis to come

I have been enforcing this myself through AGENTS files and other explicit instructions. What’s crazy is that none of the existing coding harnesses include such instructions by default. Two lines in Claude Code and Codex and the entire world would be writing better, more efficient code. Makes me believe in the conspiracy that these companies want more verbose code and longer files because it directly results in higher token usage.

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]