It answered the question it heard

At the start of September I asked Claude Code for a deep security review of this site and everything behind it. The request was one paragraph: a public website, a private console behind a login, audit data that might need watching, and the usual defend, detect and correct. What came back read as thorough. It went through the code itself, tightened what was obvious, flagged what wasn't, and suggested a regular look through the login records for anything odd. I worked through it and felt reasonably settled about the state of things.

Three and a half weeks later, a piece of unrelated work turned up something it hadn't. An earlier audit had checked which secrets each container held, but nobody had asked what each container could reach, and the public site had been sharing the private console's key to the data service. So I asked for another review, this time of the server as what it had quietly become: one machine holding every secret, every backup and the house's security feeds, which also happened to serve a website. That one found the container facing the internet could open connections to far more of the home network than it had any reason to, a whole layer the first review had never looked at.

Nothing in the first review was wrong. It had reviewed a website, because a website was what my paragraph described, and it had done it in the same confident, complete-sounding way it would have used for anything else.

It's happened enough times since that I've stopped putting it down to luck. Server problems turn up when I ask for them as part of checking the server, and not otherwise. In early November I split a review in two, one for the code and one for the server, and each found things the other couldn't have.

Later that week I gave the same kind of combined security and tech-debt review to two different systems. The content platform I've been separating out of this project was judged as a public commercial product, with the relevant standards named, and came back with 57 findings, reaching as far as repository protection and signed releases. The home system was judged as what it actually is, a family's home system with one public website at the front. That found nothing high and 19 things worth doing. Same tools, same week, and two quite different lists, because each review was told where it stood.

I've more or less learned to pick the mode myself now: code, infrastructure, hosting, sometimes all three in turn.

An experienced security reviewer would be much less likely to accept that frame without questioning it. Ask one to look at a system and they'll start with what it holds, who can reach it and what it's connected to, and only then decide where to look. The model could plainly apply that material once a later review was pointed at it. What it doesn't do on its own is decide which parts of what it knows belong to this question. It takes the frame from the words of the request and from whatever it's looking at, and a request made inside a code repository looks a lot like a request about code.

The game I'm building, Sword Beyond Worlds, had the same thing happen for months, on a bigger scale. It started in early July as a Unity project, moved to MonoGame the next day, and from then on every foundation was built for what the game needed that week. The starting island is flat, and tile heights were whole numbers. Water had one depth per tile. Interface panels sat in rectangles placed by hand. Each choice was sensible, and each was the kind of thing I'd sign off on an app project without a second look: build what's needed now, don't over-engineer, change it when a real requirement turns up.

In When the cells overlap I wrote about where that led with cliffs and water, fix after fix going round in circles. Nineteen of the commits from that stretch are reverts. Near the end of September I finally put into words what had been nagging at me. Whenever a foundation came up, the session reached for the smallest version that worked today, and argued for it with reasons I'd have used myself. A game's foundations have to carry features that don't exist yet, like caves, overhangs and water that actually flows, and retrofitting them was where the months had gone. So the instruction changed: design the core systems for the game I'm aiming at, and keep the individual features small.

Things moved quickly after that. The same agent that had been patching cliffs laid out a rebuild in five phases that day, starting with tooling to check its own work, and within a few days the terrain was a density field and the last of the grid code had gone. I'd thought about starting again, or moving to Godot, and decided against both, partly because a third-party engine might put a ceiling on how unusual the game could get. Neither the engine nor the model changed. The difference was a few sentences about what kind of project this was.

I've briefed enough people to recognise this. Someone new gets the steps. Someone senior gets the situation: who it's for, what matters, what went wrong last time, and you leave the how to them. The model ends up an odd mix of both: once it's framed, it can apply something close to the senior person's knowledge, and it has none of the situation, because it hasn't lived through any of it. What it needs from me is rarely more detail about the task and nearly always more about where the task sits.

Writing that down once is the part that scales. Most of these tools read a standing brief at the start of every session, for a person, a project or a whole organisation, and that's where I've ended up putting the situation. The game's agent keeps a standing note that foundations are designed for the target game. This project's brief opens by saying what each product is and who it's for, and the latest reviews start by saying what they're calibrated to. The recent combined reviews are the first ones that haven't later turned out to miss a whole layer, though they're only days old, so I'd hold that loosely.