OpenAI released GPT-6 Astra yesterday (September 3), and president Greg Brockman called it a “generational leap” — reports from the briefing have him closing with “Welcome to the AGI era.” I have been doing this long enough to discount launch-day adjectives, so let me skip the AGI debate and tell you what actually caught my attention as someone who builds AI systems inside a bank: the rollout is gated, the cyber capabilities are split into a restricted track, and OpenAI’s own chief scientist spent part of the launch call explaining why this model is harder to monitor than the last one.

That combination matters more to practitioners than the headlines.

Last quarter, a prompt tweak landed on a Friday. By Monday, 8% of our collateral-margin advisory bot’s responses were hallucinating policy clauses that had not existed since the 2021 Basel III update. The change looked harmless in review — just “make the tone more helpful” — but the model took that as permission to improvise. Nobody caught it until a relationship manager flagged a client email. The rollback took twenty minutes. The post-mortem took three days.

That incident is why I no longer ship agent changes without an eval gate. Not “we should really add evals.” Not “evals are on the roadmap.” The gate is the only way I sleep at night.

The other day, a colleague asked our internal coding agent to “refactor the fee calculation module like we discussed last month.” The agent cheerfully rewrote the whole thing — except we’d never discussed it. It had hallucinated a conversation from thin air, stitched together fragments from three unrelated PR reviews, and produced code that compiled but violated a regulatory constraint we’d explicitly documented six months ago.

That incident crystallized something I’ve been wrestling with: memory in agent systems is not a solved problem. It’s the new capacity planning.

Recently, one of our internal chatbots gave a confidently wrong answer about a collateral margin rule, and the incident thread quietly died with the words “looks like a hallucination, closing as won’t fix”. I have started hearing that phrase the way I used to hear “it’s flaky”. Technically true, completely useless, and usually a polite way of saying nobody looked.

Six years ago I wrote about correlation IDs because we refused to accept “it’s flaky” as a root cause for a request crossing five .NET services. We widened log windows, followed IDs, found the rolled-back transaction hiding in the noise. The same fight has come back, except this time the misbehaving component bills by the token and improvises its own control flow. The old excuse was that you simply could not see inside these systems. That excuse is expiring, and this post is about what replaced it.

I have written about running LLMs on this laptop twice before. In late 2023 I got Mistral 7B humming away in llama.cpp and called it a glimpse of the future. Early last year the DeepSeek-R1 distills made reasoning models light enough for my hardware. Both times the punchline was the same though: local models were promising but compromised. You traded down to a smaller dense model, and you felt it in every single response.

This spring something changed, and it is not that my aging IdeaPad got faster. It did not. The models got sparser instead of smaller, and that turns out to be exactly the trade my hardware needed.

One Saturday morning I opened Task Manager before I opened my mail. There it was: a model server holding nine-odd gigabytes of RAM, alive since Friday night, with a couple of things on the machine quietly pointing HTTP requests at it. A summarisation job had run overnight against my local model, on schedule, nobody watching. Somewhere in the past year or so, without any ceremony, my laptop stopped being a machine that occasionally plays with AI and became a small server that other software depends on.

I did not plan this. When I first ran models locally back in late 2023, it was a toy: Mistral 7B in a terminal, mostly to see whether it could hold a conversation at all. Fun toy. But a toy.