OpenAI shipped GPT-4o in mid-May. Anthropic took tool use generally available across the Claude 3 family at the end of May. Every internal slide deck I have seen since then has the word agent on it twice.

Most of those decks are still describing a chatbot with extra steps. The useful shape is simpler and less glamorous: a model that emits a structured tool call, your code that runs it, a result that goes back into the conversation, and a hard stop when the loop has gone on long enough.

In the rapidly evolving world of technology, few innovations have been as transformative as the internet and generative AI. From the early days of the web, when accessing information meant waiting for static pages to load, to the present era of conversational AI, the journey has been marked by groundbreaking developments.

The advent of web applications ushered in a new era of how we interact with software and services. Rather than installing programs locally on individual machines, web apps enabled accessing applications through a web browser over the internet. This fundamental shift democratized access to software and catalyzed new ecosystems of apps delivered as services.

As software developers, we are driven by a passion for creating innovative and impactful solutions. We dedicate ourselves to mastering new technologies, frameworks and methodologies, pouring over lines of code, testing and iterating until we are satisfied with the result. However, despite our best efforts, many of us struggle with a nagging sense of self-doubt. We feel like we are just pretending to be experts, that we are just winging it and that someone will eventually discover our lack of expertise.

Impostor syndrome, also known as impostor phenomenon, is a psychological phenomenon where individuals doubt their abilities and feel like they are just pretending to be competent. Despite their achievements, they feel like they are just lucky or that someone will eventually discover their lack of expertise. This feeling of inadequacy can be paralyzing, causing individuals to doubt their skills, fear failure and even avoid taking risks.

Claude 3 shipped in early March with a 200K context window and a loud story about stuffing whole codebases and policy binders into one prompt. Two weeks ago Anthropic published research on many-shot jailbreaking — a long-context attack that gets more effective the more fake dialogue you pack in front of the real question.

If you are wiring LLMs into internal tools in a bank-shaped environment right now, that combination should change how you design the app, not just how you word the system prompt.

We spent two sprints blaming GPT-4 for wrong answers in an internal Q&A POC. The model was fine. The chunks were garbage, the top-k was polite noise, and half the “sources” never contained the sentence we needed.

If you are wiring retrieval-augmented generation over real docs in early 2024, the generator is rarely the first place I look anymore. Retrieval quality is.

OpenAI’s DevDay was three weeks ago. GPT-4 Turbo with a 128k context window, cheaper tokens, custom GPTs, an Assistants API with built-in retrieval. The cloud path just got louder and, for once, cheaper — Turbo input is about $0.01 per 1k tokens and output about $0.03, roughly a third and half of what GPT-4 was charging.

I still spent last weekend trying to get a 7B model to answer questions without leaving my machine. Not because I think my laptop beats a frontier API. Because in a regulated shop, “the prompt never left the building” is sometimes the only sentence that gets a POC past the first security review.