The other day, a colleague asked our internal coding agent to “refactor the fee calculation module like we discussed last month.” The agent cheerfully rewrote the whole thing — except we’d never discussed it. It had hallucinated a conversation from thin air, stitched together fragments from three unrelated PR reviews, and produced code that compiled but violated a regulatory constraint we’d explicitly documented six months ago.

That incident crystallized something I’ve been wrestling with: memory in agent systems is not a solved problem. It’s the new capacity planning.

Recently, one of our internal chatbots gave a confidently wrong answer about a collateral margin rule, and the incident thread quietly died with the words “looks like a hallucination, closing as won’t fix”. I have started hearing that phrase the way I used to hear “it’s flaky”. Technically true, completely useless, and usually a polite way of saying nobody looked.

Six years ago I wrote about correlation IDs because we refused to accept “it’s flaky” as a root cause for a request crossing five .NET services. We widened log windows, followed IDs, found the rolled-back transaction hiding in the noise. The same fight has come back, except this time the misbehaving component bills by the token and improvises its own control flow. The old excuse was that you simply could not see inside these systems. That excuse is expiring, and this post is about what replaced it.

I have written about running LLMs on this laptop twice before. In late 2023 I got Mistral 7B humming away in llama.cpp and called it a glimpse of the future. Early last year the DeepSeek-R1 distills made reasoning models light enough for my hardware. Both times the punchline was the same though: local models were promising but compromised. You traded down to a smaller dense model, and you felt it in every single response.

This spring something changed, and it is not that my aging IdeaPad got faster. It did not. The models got sparser instead of smaller, and that turns out to be exactly the trade my hardware needed.

One Saturday morning I opened Task Manager before I opened my mail. There it was: a model server holding nine-odd gigabytes of RAM, alive since Friday night, with a couple of things on the machine quietly pointing HTTP requests at it. A summarisation job had run overnight against my local model, on schedule, nobody watching. Somewhere in the past year or so, without any ceremony, my laptop stopped being a machine that occasionally plays with AI and became a small server that other software depends on.

I did not plan this. When I first ran models locally back in late 2023, it was a toy: Mistral 7B in a terminal, mostly to see whether it could hold a conversation at all. Fun toy. But a toy.

During one recent sprint I ran a small experiment. Instead of assigning a routine bug ticket to a teammate, I assigned it to our coding agent. Twenty minutes later, while I was still in standup, a draft pull request opened itself: commits pushed, checks running, a tidy checklist of what it intended to do. Nobody typed anything. The backlog had effectively started opening its own pull requests.

I have written before about reading AI-generated diffs and about repo instructions for coding agents, but those were about the inner loop, me at the keyboard with an agent beside me. Background agents are a different animal. They change not how I code, but how work flows through the whole team. Somewhere along the way my issue tracker quietly turned into a job queue.

Not long ago, I watched a teammate’s agent loop burn through $2,300 in API calls overnight. The task? Classifying support tickets into three buckets. The model? GPT-4o. Every single ticket — even the “reset my password” ones — got the full frontier treatment.

That’s the trap. When you first get API access to a reasoning model, it feels like a superpower. You start reaching for it everywhere. Why wouldn’t you? It’s smarter, it reasons, it gets the hard stuff right. But somewhere between the demo and the production bill, the economics invert. You’re paying premium prices for commodity work.