~7 min read Most software engineers are using AI Agents wrong, but it's not their fault. Not entirely. I say this because they try implementing net new features with it and then get confused when it doesn't do well. It's been their mental model of what's happening is wrong. What is an agent actually? AI agents are a software package to coordinate letting an LLM to make tool calls. So anything that's true of the underlying capability of an agent is purely derived from the LLM. The part we often describe as intelligent. So lets first step back and establish something. LLMs are not intelligent, not at least the way you and I would use that word. You can find plenty of arguments about this online, but I don't feel the need to defend this too rigorously here. LLMs are primarily token producing machines based on what's most likely the correct next one. The most correct next one is based on what its training data. And thus, it's producing an average of the probable outcome. I must repeat that last part, because it's the core piece that software engineers nod their heads about like they get it, but they're skipping over it: LLMs produce an average of the probable outcome. Most software engineers get this theoretically. It's in practice where this falls apart. The place of low leverage where everyone flocks to If you are doing something novel, it by definition is unprobable to be in the training data. If you're doing something boiler plate, it's a probable outcome. New feature work is generally a mixture of both novel and boiler plate work. The boiler plate goes well. Things like an API endpoint. That stuff you used to use a scaffolding tool for. It'll one shot that. The novel stuff goes unevenly. Sometimes it gets it. Other times it doesn't. You can improve how likely the right outcome is with a spec. But most people's specs are, ahem, underspecified. And features are unfortunately where software engineers are being encouraged to maximize their AI agent time. But that area is fundamentally capped by attention. Your software engineering attention isn't infinitely divisible. In fact I'd argue it's not divisible at all and when coding—even with agents—you should be monotasking. I think that's why a lot of people started to say, "don't read the code." That way lies madness. Piles and upon piles of unverified outcomes obscured away in code duplication and incorrect abstractions. Legibility of the underlying system is not something that can be left to chance. The place of high leverage where most overlook Throw them at the boring and easily verified crap. And throw it hard. The thing you used to have a runbook for fixing. Put an unattended agent in a box somewhere and let it just make a PR for that thing whenever it gets a ticket. Your return on your attention goes way higher than features. You can show up in the morning to like 10 PRs just needing a quick glance before you approve and merge. For maximum leverage, though, figure out how to deterministically verify that the outcomes are correct and your return on attention will go through the roof. PS I promise that this newsletter will not become an AI newsletter. I will continue posting about AI, but my topics will tend to rove around. Just in case you were worried. This message is 100% organic human effort, so it will have typos. No, I didn't send it through an LLM to typo check it. It was made with just my fingers and my eyes. |