What building games with AI taught us about AI engineering.

Two games built with AI coding models

Levy Street has used AI coding models to build two very different games in public. They are useful because games punish sloppy engineering immediately, and because their code is open to inspection.

Pokémon Claude Red started from a one-sentence prompt, after which, in the README’s words, “Everything in this repository grew from that sentence, followed by a few days of playing it and asking for fixes.” It is a fan project: “not affiliated with or endorsed by Nintendo, The Pokémon Company, GAME FREAK, Creatures Inc., Anthropic or OpenAI.” We do not show its imagery here because the characters belong to their owners.

Lesson 1: the first version is fast; structure decides whether it lasts

The first commit of World of ClaudeCraft, co-authored by Claude Fable 5, already contained a 16,589-line game with three faces sharing one simulation: a browser client, a multiplayer server and a training environment for AI agents. That speed is real.

What let the game survive 60 contributors and close to 20,000 commits was a handful of decisions made early: one simulation core on a fixed 20 Hz tick, all randomness through one seeded generator, no browser or rendering code inside the simulation, and every combat roll resolved on the server. AI made the build fast. Architecture kept it maintainable.

For business systems, the equivalent is deciding the boundaries before generating code: which component owns the data, which may call external services, and which must stay pure and testable.

Lesson 2: write rules where agents will read them, then enforce them in tests

World of ClaudeCraft ships instruction files for three families of coding agents (CLAUDE.md, AGENTS.md and GEMINI.md), and “Most directories carry their own CLAUDE.md with local conventions.” Those files tell an agent the local rules before it writes a line.

Instructions are not enough on their own, because agents miss them. So the critical rules are also code: an architecture test “scans every sim file for a forbidden import, a DOM global, or a stray clock or Math.random call”. An agent can overlook a sentence; it cannot merge code that fails the build.

This is the pattern we recommend to every client adopting Claude Code or similar tools: put conventions in the repository, and turn the ones that matter into tests. Our lesson on evaluating AI-generated code shows how to start.

Lesson 3: give the agent source data, not memory

Pokémon Claude Red separates facts from presentation. Map layouts, collision, warps, trainers, wild encounters, species, moves and the music’s note data are converted by scripts from the community pret/pokered disassembly. “No graphics or original text are taken from it.” Everything visible is generated: the game draws into a 320×180 pixel framebuffer, each map cell carries a semantic label such as “tall grass” or “roof” that a procedural painter renders, and all 151 Pokémon are described in a small shape language that is rasterised at runtime.

Two engineering lessons follow. First, when correctness matters, convert authoritative data with deterministic scripts rather than asking a model to recall it. Second, generating output from a compact specification makes it reviewable: the launch video’s description says the 23 weakest Pokémon were redrawn after review, which is only practical when each one is a short description rather than a hand-drawn image. In business terms, this is the case for structured outputs and validation.

Lesson 4: let automated players exercise every path

Neither game relies on people clicking around to find bugs. Pokémon Claude Red runs the whole game headless inside Node and builds test robots on top:

Dialogue fuzzer

Visits all 224 maps, talks to every character and reads every sign, more than 1,000 conversations per run.

Battle fuzzer

Random battles with random species, levels and moves, reporting exceptions and stuck battles.

Link-battle fuzzer

Two copies of the game fight through their choices alone; every turn must match in both.

Step fuzzer

Fires every step script on every walkable cell under random story states and flags anyone walking through a wall.

Warp checker

Walks the warp graph and confirms every exit leads back where it came from.

World of ClaudeCraft goes further: its core runs as a Gymnasium environment, and the README reports that an automated five-bot raid clears the Hollow Crypt in about five minutes. The business equivalent is synthetic customers, replayed transactions and agents that walk every branch of a workflow; see testing AI with synthetic data.

Lesson 5: determinism makes failures reproducible

Both games treat randomness as an input. World of ClaudeCraft routes all randomness through one seeded generator, and its training environment is “Deterministic by construction. No wall clock, no Math.random.” In Pokémon Claude Red’s online battles, “Both games roll from the same random seed”, and each turn the host sends a checksum so a drifting copy can be corrected.

If you can replay an event exactly, you can debug it, test a fix against it and explain it afterwards. For AI systems in business, log the inputs, versions and seeds behind each decision, as described in our guide to AI audit logs.

Lesson 6: generated documentation stays true; hand-written copy drifts

World of ClaudeCraft’s wiki is “generated straight from live game content”, so it cannot disagree with the game. Hand-written descriptions elsewhere did drift: across the press release, the Steam page, itch.io and the README, the number of supported languages appears as 14, 19, 13 and 22. The README and the live site say 22, which is correct. With AI agents shipping changes daily, any document maintained by hand falls behind; generate documentation from the system where you can, and date it where you cannot.

Where AI-built software still goes wrong

Honest outside review helped. Early coverage by MMOHuts called World of ClaudeCraft “Small, rough around the edges, but functional”, and WinFuture reported experts describing the generated code as unstructured. The response was to keep shipping in public, release by release.

The limits we see repeatedly:

• Speed outruns judgement. Design, balance, art direction, moderation and release decisions stayed with people. AI did not decide what the game should be.
• Provenance needs recording. World of ClaudeCraft’s credits file states which art was AI-generated, commissioned or taken from CC0 libraries. Without that, nobody can answer rights questions later.
• Rights are easy to ignore. An AI model will happily rebuild something you do not own. Pokémon Claude Red is non-commercial and carries a non-affiliation notice; a commercial product would need permission.
• Channels lag. The iOS build of World of ClaudeCraft trailed the web version. Every distribution channel needs its own release check.

• Boundaries and ownership are decided before code is generated
• Critical conventions exist as automated tests, not only as instructions
• Source data comes from authoritative systems through deterministic converters
• Automated users exercise every path before release
• Randomness is seeded and decisions are logged so incidents can be replayed
• People own product, rights and release decisions, and AI contributions are recorded

We apply the same approach in custom AI development for clients, and in our research on Code Policy Models, where an AI agent edits a program and automated evaluation decides what stays.

Written by the Levy Street team in Wellington. Last updated 19 October 2026.