A few weeks ago, someone two levels above me asked a simple question: how much faster is the team with AI coding tools? I opened my mouth, and what came out was a feeling dressed up as a number. Everyone feels faster. I feel faster. The cursor barely rests before the next suggestion lands, and on a good day with a Spring Boot service I have written before, the boilerplate practically types itself.

Then I looked at our actual delivery numbers for the quarter, and the feeling did not survive contact with them. Throughput was flat. Review times were up. That gap between what my fingers tell me and what the merge log says has been living in my head ever since.

I have written about running LLMs on this laptop twice before. In late 2023 I got Mistral 7B humming away in llama.cpp and called it a glimpse of the future. Early last year the DeepSeek-R1 distills made reasoning models light enough for my hardware. Both times the punchline was the same though: local models were promising but compromised. You traded down to a smaller dense model, and you felt it in every single response.

This spring something changed, and it is not that my aging IdeaPad got faster. It did not. The models got sparser instead of smaller, and that turns out to be exactly the trade my hardware needed.

One Saturday morning I opened Task Manager before I opened my mail. There it was: a model server holding nine-odd gigabytes of RAM, alive since Friday night, with a couple of things on the machine quietly pointing HTTP requests at it. A summarisation job had run overnight against my local model, on schedule, nobody watching. Somewhere in the past year or so, without any ceremony, my laptop stopped being a machine that occasionally plays with AI and became a small server that other software depends on.

I did not plan this. When I first ran models locally back in late 2023, it was a toy: Mistral 7B in a terminal, mostly to see whether it could hold a conversation at all. Fun toy. But a toy.

Somewhere in our account there is an S3 bucket that nobody owns and everybody uses. Our Spring Boot services dump daily statements, audit exports, and application logs into it, versioning switched on years ago by someone careful, lifecycle rules configured by nobody. Last quarter I opened Storage Lens out of curiosity and just stared. One prefix held three years of files nobody had opened since the week they were written, all sitting in Standard storage, all billed every single month.

Storing data feels free because the per-GB number is tiny. Keeping it there forever, and then moving it around carelessly, is where the real money goes.

A few weeks ago I demoed a small internal assistant to my team — a chat box over our runbooks and incident notes, Spring Boot service in front, retrieval over docs we already own, pinned model version behind it. People liked it. Then one senior asked the question that ended the celebration early: “Is it registered?” He did not mean the demo URL. He meant the model inventory.

So I spent the next two weeks doing something I had never done for any CRUD service: filing my LLM app with the model-risk board. Forms, diagrams, eval sheets, a meeting where serious people asked unserious-sounding questions like “what stops it from making things up about a production database?” I walked in annoyed at the paperwork. I walked out thinking every team building LLM apps in a bank should do this on purpose.

Our Spring Boot services build on GitHub Actions ten, fifteen times a day, and for a month the pipeline had a habit I hated: a different integration test failed every other run, somebody hit re-run failed jobs, it went green, everybody moved on. The suite was green on paper and nobody believed a word it said.

Last month it bit us properly. An OrderStatusIT test failed on my PR, passed on rerun, failed again on a teammate’s completely unrelated PR, and blocked three merges in one afternoon. The failure had nothing to do with any of our changes. That evening I stopped asking “whose change broke this” and started asking the better question: why does one unreliable test get to hold the whole team hostage?