Why I shut down Deep Sheep, my AI coach for Age of Empires IV
Esports is a big industry, and the people who grew up on real-time strategy games are getting older. They have jobs and children now, and much less time to play than they used to. What they do have is more disposable income to invest in the hobby, which you can see in how much more people spend on cosmetics every year. As a gamer myself I’m not a big fan of microtransactions, but it is what it is.
The bet behind Deep Sheep was that some of those players would pay to shortcut the learning. Being competitive in an RTS is a lot of fun, but getting there has always meant hundreds of hours of watching your own replays and working out what went wrong. If a coach could tell you the one thing to fix after each game, you could improve on a couple of evenings a week.
I built it between May and October 2026 for Age of Empires IV. You pasted in a link to a game from aoe4world, and a minute later you had a report on why you lost.
How it worked
The rule I set early was that the model doesn’t do the analysis. Everything that could be worked out with code and SQL was worked out that way, and Claude’s job was to explain the results to a player. It’s how I’d build any LLM feature for a business.
For each game, the server fetched the match summary and pulled out the facts for both players: when they reached each age, how many villagers they had at each point, which units they built and when, and which upgrades they bought. A points-based classifier sorted each player’s game into one of six strategies, such as Fast Castle or Feudal Aggression, and the player’s timings were compared with benchmarks for that strategy.
The benchmarks came from data. A crawler worked its way out from a handful of players through their opponents’ match histories, and by the end the database held about 225,000 ranked 1v1 games. That was enough to say things like “Chinese players who win against English with this strategy reach Castle Age around 10:15”, and to look up win rates for a strategy against a particular civilisation.
On top of that sat a hand-written library of about 50 named strategies across 18 civilisations, each with the signals that give it away and how to counter it. If the opponent’s build matched one, the coach was told by name. Patch notes went into a pgvector store, so the report could mention a recent balance change when it was relevant.
All of that went to a Claude agent as structured context, with a prompt that listed what the data could show and what it couldn’t. Micro, positioning and intent aren’t in a match summary, so the coach had to say “check the replay” rather than guess at them. Every claim in a report had to come from a number it had been given.
Here’s the start of a real report from one of my own games, as it looked in the app.
It goes on to give three specific things to fix, with the timestamps. A Sonnet run cost about two cents, mostly output tokens.
Later I added a profile review, which read your last ten games and found the mistake you kept making.
The rest was the usual work of a small product: Supabase for accounts and data, an Express API on Railway, a React front end on Netlify, Stripe for payments, free reviews capped per day, credits that refunded themselves if a run failed, and a feedback button on every report.
Old-school guides, made from the data
Plenty of players who grew up on RTS games still like a printed guide, the kind you keep next to the keyboard, so I sold one-page cheat sheets for each civilisation at €5 each. Nine were for sale by the end. I think there’s a real product opportunity in keeping guides like this physical, as something people can print. With so much of life on screens now, I always keep in mind how users could take parts of a digital product offline.
The usual community guide is one good player’s opinion, and it goes stale with every patch. These started from SQL over the 225,000 games. Each one has timing benchmarks for every age, win rates for each strategy and landmark, and notes on every matchup.
An agent did most of the legwork. It read the transcripts of coaching videos from well-known players, pulled out their build orders and matchup advice, and checked each claim against the data. It reported back in three lists: advice the numbers backed up, advice they contradicted (with the size of the gap), and advice the data couldn’t speak to. After that, a script rendered each sheet as a high-resolution image, a print-friendly light version and a redacted preview for the store.
Where it fell over, and knowing how your users feel about AI
You need to know how your users feel about what you’re putting in front of them. Call of Duty added a Nicki Minaj skin in 2023 and a lot of players said the game was turning into Fortnite. Then players spotted a zombie Santa with six fingers on a Black Ops 6 loading screen, and Activision’s note on Steam about using generative AI became news.
By July there were 24 sign-ups, and nearly two thirds of them ran a review, but most ran one and didn’t come back. I sent an email announcing the profile review with free credits attached, and got no replies at all. Too few people who tried it ever paid.
When I asked gaming communities for feedback, a lot of the pushback was about having to log in at all. Every review cost real money in API calls, so I argued a login was needed to keep costs in check and to put some limits on usage.
It needed people who wanted to be properly competitive at a game and would also pay for a coach, and that’s a small group. Most competitive players learn from free content and from each other, and the few who pay for coaching want a person. On top of that, a lot of the games industry and its players are hostile to AI features right now. It was always going to be an uphill battle, and not worth more of my time.
I shut it down on 12 October. Users got a month’s notice, every service holding personal data was deleted, and the game dataset is kept, since it doesn’t hold anything personal.
I didn’t do much market research before building it, on purpose. What people say they’d pay for and what they actually pay for are often different things, and a working product was the quickest honest way to find out. I got my answer. If anything, I’d decide earlier what result means stop, since the signs were there by the summer.
As AI gets better, building the real thing is becoming an easier way to test a product idea, as long as you wind it down properly when the answer is no, and get rid of your users’ data when you do.
The approach is the part I’d use again. Do the analysis in code, give the model a tight brief, and make it say when it doesn’t know.
Gamers mostly don’t want AI in or near their games. In a Quantic Foundry survey of 1,799 gamers at the end of 2025, 85% were negative about generative AI in games and 63% picked the most negative answer. Older players were the least negative, though even among those over 45 only 22% were positive. Quantic Foundry, December 2025. On Black Ops 6 and Steam’s AI disclosure, PC Gamer and Creative Bloq. On the reaction to the Nicki Minaj and Snoop Dogg operators, EssentiallySports.