We spent two sprints blaming GPT-4 for wrong answers in an internal Q&A POC. The model was fine. The chunks were garbage, the top-k was polite noise, and half the “sources” never contained the sentence we needed.

If you are wiring retrieval-augmented generation over real docs in early 2024, the generator is rarely the first place I look anymore. Retrieval quality is.

OpenAI’s DevDay was three weeks ago. GPT-4 Turbo with a 128k context window, cheaper tokens, custom GPTs, an Assistants API with built-in retrieval. The cloud path just got louder and, for once, cheaper — Turbo input is about $0.01 per 1k tokens and output about $0.03, roughly a third and half of what GPT-4 was charging.

I still spent last weekend trying to get a 7B model to answer questions without leaving my machine. Not because I think my laptop beats a frontier API. Because in a regulated shop, “the prompt never left the building” is sometimes the only sentence that gets a POC past the first security review.

In Part 1, I wrote about the hard parts of moving from individual contributor to management — losing the IC identity, swimming in ambiguity, juggling stakeholders, learning to delegate, and becoming a coach. I promised tips. A few months into living this role for real, here is what actually helped me, not the polished advice from a slide deck.

None of this is universal. These are the habits that stopped me from either drowning in other people’s work or quietly sliding back into being the best engineer on the team again.

According to kubernetes.io:

Kubernetes is the defacto container orchestration platform today. It is a powerful tool that can help you manage your applications more efficiently and reliably. Kubernetes uses a declarative approach to configuration, which means that you define the desired state of your application and Kubernetes will work to make sure that the actual state matches the desired state. This makes it easy to deploy changes to your applications and to scale your applications up or down as needed.

However, managing kubernetes can be a nightmare. It has a steep learning curve and has a lot of features, which can take some time to master. While there is the official kubernetes dashboard, in my opinion Rancher by Suse, is the preferred tool for most administrators. The real power of Rancher is in its ability to manage multiple kubernetes clusters, while providing a single pane of glass for all administrative tasks.

A few weeks into Spring Boot land and I already have a new favourite kind of production pain: one slow downstream dependency quietly taking the whole request path with it. Not a crash. Not a clean 500. Just threads waiting, timeouts stacking, and suddenly your service looks unhealthy because someone else’s database is having a bad afternoon.

In the .NET world I reached for Polly when this showed up. In Java the answer that keeps coming up in code reviews is Resilience4j — the library that basically replaced Netflix Hystrix after Hystrix went into maintenance mode back in 2018. If you are wiring Spring Boot 3 services that talk to other services (and who isn’t), this is the pattern I am finally treating as non-optional.

One of my strengths at work is debugging/troubleshooting technical issues. I have been told that I have a knack for finding and fixing problems that others find hard to solve. Many people come to me for assistance when they encounter difficulties and I often am able to give them effective solutions or workarounds that allow them to continue their work. This is an ability that has come with a lot of practice and experience in dealing with various technical challenges. There are a few core ideas that underpin these abilities, some of which I try to enumerate below