Home / Guides & Tutorials / The AI Coding Workflow That Ships: From …

CHAPTER 7 OF 7 · 10 min · 3 COPYABLE ASSETS

How to make coding agents improve after every bug (System Evolution)

Phase 3: treat every bug as a defect in your AI layer. Run a retroactive session, upgrade rules and commands, and version them like code so the team compounds.

TL;DW

When a PIV loop produces a bug, fixing the code is only half the job. Fix the system that allowed it. Run a retroactive session: you let this problem in, so inspect your rules, commands, and skills and propose improvements so it cannot recur. The four things to evolve are your commands, your on-demand context, your global rules, and your plan and PRD templates. Check them into source control and review changes through pull requests. One improved command can save a team dozens of hours, permanently.

Watch: FULL Guide to Becoming a Principled Agentic Engineer (Build Anything with AI)

▶ Watch this chapter on YouTube from 57:12

Lesson · 9 parts

However hard you align during planning, implementations will sometimes miss: coding agents are non-deterministic by nature. What sets this system apart is what happens next. A bug is not a one-off fix before the next ticket. It is evidence about your setup.

Instead of patching and moving on, open a session aimed at the system itself. The agent inspects the rules, commands, and skills, the workflow that produced the bug, and proposes concrete improvements so the class of issue cannot recur.

Typical outcomes of a retroactive session, from real usage: a new line in global rules ('every new component matches the styling of existing components in the same directory'), a new validation step in the implement command ('run the browser skill against changed pages before handing back'), or a sharper section in the plan template ('name the consumer of every new utility'). Ten minutes of session time, and the fix applies to every future ticket.

Review these edits like code. A too-broad rule ('be careful with styling') does nothing; a specific one changes agent behavior permanently.

The retroactive prompt

You allowed this bug into the codebase. Dive into your AI layer, the global rules, commands, and skills, and the workflow I ran you through. Identify what we should improve so this class of issue can't happen again. Propose specific edits.

Say a new frontend component ships ugly, inconsistent with the rest of the app. The code is the symptom. The cause might be missing style conventions in your global rules, or a validation workflow that never checks new components against existing ones. Fix the cause and the bug becomes impossible, not just fixed.

Weak rule vs strong rule (global rules)

Weak:  "Keep components consistent with the app's style."

Strong: "Every new component must reuse the design tokens in
src/styles/tokens.css. Before finishing, compare your component
against two existing ones in the same directory and match their
prop and naming patterns."

Rules, commands, and skills get checked into source control just like the codebase. The whole team reuses them, and changes go through pull requests with real review. That is what makes evolution compound: one improved validation command can save engineers dozens of hours across every future ticket.

The four things to evolve over time

Commands: your repeatable procedures
On-demand context: docs (even Confluence pages) optimized for AI reading
Global rules: the conventions the agent must always follow
Plan and PRD templates: fix structural gaps as you discover them

The system is two nested cycles. The inner loop: everything works, and you chug through tickets with the PIV process. The outer loop: something significant went wrong, so you step out, evolve the system, then re-enter. You skip the outer loop when things run clean, but take it often. It is also how the general starting-point commands become your commands, specialized to your codebase.

The QR feature completes on its own branch, validation run, files summarized. One detail is worth stealing: Cole's implement command now compares the finished code against the plan to catch deviation, an improvement born from his own system evolution. The agent even updated the JIRA ticket and posted a detailed comment. Too detailed? That preference is itself an evolution cue. Specify the comment format you want in the command.

Cole creates a poll and sees no QR code. First suspect: a stale app instance, so he has the agent restart it and verify with the browser skill. The real answer sits in the agent's report. The utility was built with no consumer yet, because wiring it into the UI belongs to the next ticket. 'That's on me.' The agent did exactly what the scoped ticket said; the human misread the scope.

For the live demo, Cole has the agent quickly wire a consumer page, then watches the agent-browser skill open the site, navigate, and screenshot to validate visually. That is the same end-to-end testing you can wire into any plan's validation strategy. The QR demo page renders, and the full presentation view arrives with the follow-up ticket, as scoped.

Planning to PIV loop to system evolution: idea to production, with pull requests, ticket updates, and validation handled along the way. It takes real time, and that is fine. Even careful plan iteration and code review beats doing the documents, tickets, and code yourself by hours. 'The days are gone of being a Stack Overflow warrior.' Take the foundation and mold it to your team.

Study notes

Part 1 of 9 · 57:12 to 57:57

Agents are non-deterministic. Plan for imperfection

Concept Phase three begins from an engineering fact rather than a hope: coding agents are non-deterministic, so identical prompts can produce different code on different days. Zero-defect delegation is not on the menu, and pretending otherwise is how teams get burned. Systematic improvement is on the menu, and it compounds.

Why it matters The design response is to stop treating bugs as isolated events to patch and forget. In this system a bug is data about your setup: which rule was missing, which command allowed it, which template had the hole. Data is only wasted if you refuse to use it.

Do this

Accept the premise: log every agent mistake this week instead of just fixing it.

Part 2 of 9 · 57:57 to 58:38

Run the retroactive session

Concept The retroactive session inverts ordinary debugging: the target is the system, not the code. You tell the agent it allowed the problem, and it audits its own operating context, the rules, commands, and workflow it ran under, for what made the failure possible.

Why it matters Class-level fixes are the multiplier that makes this phase the most powerful of the three: a patched bug fixes one ticket, while a patched rule deletes the entire category from every future ticket, for every teammate who shares the AI layer through source control.

Do this

On your next real bug, paste the retroactive prompt from the lesson panel before you patch the code.

Part 3 of 9 · 58:38 to 59:41

Example: an off-style component is a rules bug

Example Trace the worked example end to end: an off-style component ships. The symptom is ugly code. The cause is a missing style convention in global rules, or a validation workflow that never compares new components against existing ones. Two candidate fixes, both upstream of the code itself.

Concept Real retroactive outcomes look small and compound hard: one sharper global rule, one added validation step in the implement command, one stricter section in the plan template. Ten minutes of session time each, permanent effect on everything downstream. The asset below shows the difference specificity makes.

Do this

Trace one past bug to its cause in your rules or workflow.
Write the missing rule, and make it specific.

Part 4 of 9 · 59:41 to 1:00:46

Version your AI layer like code

Concept The AI layer is code now in every sense that matters: checked into source control, changed by pull request, reviewed like anything else that ships. One person's improvement becomes the team's default at merge time, and regressions in your process become revertable.

Concept The four evolution targets, as a standing checklist: commands, on-demand context (documents written for AI reading, Confluence included), global rules, and the plan and PRD templates. When something goes wrong, one of those four allowed it. Finding which one is the retroactive session's whole job.

Do this

Check your rules, commands, and skills into source control today.
Route future changes through pull requests.

Part 5 of 9 · 1:00:46 to 1:02:11

Two loops: inner (ship) and outer (reflect)

Concept Cadence guidance straight from practice: enter the outer loop for significant issues, and loop straight back to the next ticket when work runs clean. Evolution is a habit with a defined trigger, not a ceremony performed after every minor wobble, and the trigger is yours to set.

Why it matters The two-loop picture explains where mastery comes from: the inner loop ships, the outer loop learns. Run the outer loop often enough and Cole's generic starting commands become your commands, specialized to your codebase's actual patterns, failure modes, and vocabulary.

Try it Set your outer-loop trigger now, in writing, before the next bug argues you out of it. For example: any bug that reaches review twice, or any agent output you rejected outright. A concrete trigger is what turns we-should-improve-the-system into a practice that survives busy weeks.

Do this

Define your trigger for the outer loop. For example: any bug that reaches review twice.

Part 6 of 9 · 1:02:11 to 1:03:14

Back in the demo: the implementation lands

Why it matters The landed implementation shows evolution in the wild: Cole's implement command now compares finished code against the plan to catch drift, an upgrade born from one of his own retroactive sessions. It exists because a past failure taught the system something, which is the phase working exactly as designed.

Try it The verbose JIRA comment is a live evolution cue happening on camera: the format annoys, so the fix is specifying the desired format inside the implement command. Notice the pattern and apply it ruthlessly: every irritation with agent output is an edit to a procedure somewhere.

Do this

Steal Cole's upgrade: add a plan-versus-code comparison step to your implement command.

Part 7 of 9 · 1:03:14 to 1:04:59

Live debugging: validate before you celebrate

Concept The debugging sequence is a compact masterclass in hypothesis ordering: cheapest first. No QR code visible? Suspect the stale app instance before suspecting the code. Restart, then re-verify with the browser skill, and only then read deeper. Most mysteries die at step one.

Concept Second lesson from the same minute: read the agent's report before forming theories. The answer was already written down, the utility exists but nothing consumes it yet. Agents document their work unusually well; the habit of actually reading that documentation is free accuracy.

Pitfall 'That's on me' is the professional core of the clip. The agent implemented the scoped ticket exactly; the UI wiring belonged to the next ticket. Scope-check before blame, every time, or you will end up fixing correct code, which is the most confusing bug you can create.

Do this

Before blaming the agent, reread the ticket scope.
Restart stale processes before debugging deeper.

Part 8 of 9 · 1:04:59 to 1:06:05

Agent-browser validation closes the loop

Concept The quick consumer page exists for the demo, but the validation pattern around it is the durable lesson: the agent-browser skill opens the site, navigates the flow, and screenshots the result. Visual proof, produced by the agent, attached to the work it proves.

Try it Adopt browser-in-the-loop as your default for anything with a user interface: add it to the plan's validation section and require a screenshot before handback. Screenshots end works-on-my-machine arguments before they start, and producing them costs the agent seconds rather than costing you a meeting.

Do this

Locate the agent-browser skill in the companion repo.
Add it to one plan's validation list this week.

Part 9 of 9 · 1:06:05 to 1:07:00

The whole system, recapped

Concept The recap compresses the whole course into one breath: planning turns intent into artifacts, the PIV loop turns artifacts into validated code, and system evolution turns failures into permanent upgrades. Three phases, all artifact-driven, all tool-agnostic, and all molded to your team rather than imposed on it.

Why it matters The honest cost accounting stands at the end as it did at the start: this takes real time, and it is still hours cheaper than hand-writing the documents, tickets, and code. You also exit with something the fast path never grants: a system that is measurably better than it was last sprint.

Do this

Write your one-page version of this workflow.
Pin it where your team plans work.

Key terms

System Evolution: Treating every bug as a defect in the AI layer and upgrading rules, commands, or templates so the class of issue cannot recur.
Retroactive session: A session where the agent audits its own rules and workflow for what allowed a failure.
Inner and outer loop: Inner: ship tickets through Plan-Implement-Validate. Outer: pause to improve the system, then re-enter.

Key takeaways

Coding agents are non-deterministic. The question is not whether mistakes happen but whether each one improves the system.
Every bug traces back to context you gave the agent, and context is editable.
Version your AI layer like code: source control, pull requests, code review for commands.
Two loops. The inner PIV loop ships work; the outer evolution loop makes every future inner loop better.

Watch out

Skip the outer loop for small wobbles. Cole steps into system evolution for significant issues and otherwise cycles straight into the next PIV loop.
A feature can be done and still invisible. Cole's QR utility had no consumer because the UI belonged to the next ticket. Read the ticket's actual scope before declaring the agent wrong. His words: that's on me.

Check your understanding

A bug made it through the PIV loop. What does system evolution say to do beyond fixing the code?

Run a retroactive session against your AI layer. Find what in the rules, commands, or templates allowed the bug, improve it so the whole class of issue cannot recur, then check that improvement into source control for the team.

Why did the QR code not appear, and what is the lesson?

The agent had correctly built only the piping; the UI consumer belonged to the next ticket. The lesson: validate against the ticket's actual scope before concluding the agent failed.

01 57:12 Concept Agents are non-deterministic. Plan for imperfection ▾

02 57:57 Step Run the retroactive session ▾

03 58:38 Concept Example: an off-style component is a rules bug ▾

04 59:41 Concept Version your AI layer like code ▾

05 1:00:46 Concept Two loops: inner (ship) and outer (reflect) ▾

06 1:02:11 Step Back in the demo: the implementation lands ▾

07 1:03:14 Watch out Live debugging: validate before you celebrate ▾

08 1:04:59 Step Agent-browser validation closes the loop ▾

09 1:06:05 Concept The whole system, recapped ▾