The other day, a colleague asked our internal coding agent to “refactor the fee calculation module like we discussed last month.” The agent cheerfully rewrote the whole thing — except we’d never discussed it. It had hallucinated a conversation from thin air, stitched together fragments from three unrelated PR reviews, and produced code that compiled but violated a regulatory constraint we’d explicitly documented six months ago.

That incident crystallized something I’ve been wrestling with: memory in agent systems is not a solved problem. It’s the new capacity planning.

Recently, one of our internal chatbots gave a confidently wrong answer about a collateral margin rule, and the incident thread quietly died with the words “looks like a hallucination, closing as won’t fix”. I have started hearing that phrase the way I used to hear “it’s flaky”. Technically true, completely useless, and usually a polite way of saying nobody looked.

Six years ago I wrote about correlation IDs because we refused to accept “it’s flaky” as a root cause for a request crossing five .NET services. We widened log windows, followed IDs, found the rolled-back transaction hiding in the noise. The same fight has come back, except this time the misbehaving component bills by the token and improvises its own control flow. The old excuse was that you simply could not see inside these systems. That excuse is expiring, and this post is about what replaced it.

The other week a flaky integration test in one of our Spring Boot services started failing on the build, and I did what I do these days: I pasted the stack trace into the coding agent and went to make coffee. By the time I was back, the fix was in — a test ordering dependency, one shared fixture leaking state between two test classes. Green build. Merged by lunch.

And here is the embarrassing part: if you asked me today exactly which fixture leaked what, I would have to look it up. The agent found the bug. I just carried the coffee.

The staging environment died on a Saturday morning, and for twenty minutes everyone was sure it was a bad deploy. It was not a bad deploy. Friday evening someone had pushed a perfectly good build, and Saturday morning every call to one Spring Boot service started failing TLS handshakes. The certificate on the ingress had expired at midnight, quietly, while we were all asleep.

I have seen the long-lived database password version of the same movie too. A credential that has not changed in two years, pasted into three config files and one wiki page, and rotating it feels like defusing something because nobody knows what will break. Both outages taught me the same lesson: rotation is not a ticket you handle when it fires. It is a drill you practice until it is boring.

A few weeks ago, someone two levels above me asked a simple question: how much faster is the team with AI coding tools? I opened my mouth, and what came out was a feeling dressed up as a number. Everyone feels faster. I feel faster. The cursor barely rests before the next suggestion lands, and on a good day with a Spring Boot service I have written before, the boilerplate practically types itself.

Then I looked at our actual delivery numbers for the quarter, and the feeling did not survive contact with them. Throughput was flat. Review times were up. That gap between what my fingers tell me and what the merge log says has been living in my head ever since.

I have written about running LLMs on this laptop twice before. In late 2023 I got Mistral 7B humming away in llama.cpp and called it a glimpse of the future. Early last year the DeepSeek-R1 distills made reasoning models light enough for my hardware. Both times the punchline was the same though: local models were promising but compromised. You traded down to a smaller dense model, and you felt it in every single response.

This spring something changed, and it is not that my aging IdeaPad got faster. It did not. The models got sparser instead of smaller, and that turns out to be exactly the trade my hardware needed.