The failure that finally forced the rewrite is dull enough to be convincing. A ticket merged on the 14th. Its documentation phase completed correctly — the handoff was folded into the domain specs and deleted, source-ticket lineage intact. Its four phase children stayed open anyway: one sitting at In Review, one at In Progress, until I closed them by hand a day after the work merged and nine days after the phase they represented had actually finished.

The cause was a sentence in the skill’s own text:

Moving a child ticket (or the root) to “Done” once its PR is actually merged is a manual action the human does.

Four tickets per feature, each requiring a person to remember, none of which any automation read. Stale by construction. That’s the same shape as the ceremony nobody performs — except here the un-performed step wasn’t optional politeness, it was the state the next run would route from.

The process couldn’t run the ticket

Both versions of the harness were unavailable at once, and the bind is worth naming because it recurs.

The old one couldn’t run it, because the work was editing it. Following /corpus-work would have created the four phase child tickets the ticket explicitly forbids and written a handoff doc in the exact template the ticket replaces — the process would have violated the deliverable. There was nothing for the phases to bite on either: no domain spec for the harness skills, no test layer for prose instructions, and an acceptance criterion demanding all eight reference files be rewritten in one atomic change. Every phase boundary the pipeline exists to enforce was degenerate.

The new one couldn’t run it for the duller reason that it didn’t exist yet. It was the deliverable.

So the ticket ran with no harness at all: read everything, put a plan on a review surface, one branch, one PR, and me standing in for every gate that would otherwise have fired on its own. That isn’t the escape hatch — the escape hatch is a sanctioned cheap path for small work, and it still runs through the machinery. This is the condition a harness hits precisely when it’s being changed, and it’s the one moment none of what you’ve built is available to help.

One record, and only one thing allowed to write it

The fix is a YAML frontmatter block on docs/handoffs/<ROOT-ID>.md, and it is the only machine-readable record of where implementation stands: schema version, Linear id, domain, surface, feature branch, phase, round, phase status, active branch, active PR, test baseline commit, final PR.

Routing reads that block and nothing else. Not the Dev Log. Not a branch-name convention. Not a Linear comment, a prose heading, or the Linear status. Linear keeps Backlog/Todo → In Progress → In Review → Done, and none of those map to a phase number — the phases live entirely in the repository now, and there are seven of them instead of four, because the old single “implementation” phase split into unit, integration and E2E.

That leaves three authorities with clean edges:

And one rule the whole design rests on: the checkpoint records intent and the last known position; GitHub decides whether that position is still true. A recorded PR is never assumed merged.

Wrap the mechanism in a skill

The plan went out as a review artifact with three open decisions on it. Two came back as I’d recommended. The one that mattered arrived as a freeform annotation:

Can we ensure that the harness interacts with the handoff with a dedicated skill, and that skill in turn wraps over that mechanical tool?

That turned what had been section 4 of a plan — an “evidence rules” table the agent was supposed to read and apply — into an executable command with a skill in front of it. Three layers, each doing one job:

• The data. Typed frontmatter, one file per feature. No second state file beside it.
• The tool. pnpm handoff — arithmetic over that data plus a GitHub check. show returns a resolution.action from a closed set: plan, start, resume, report-and-stop, advance, blocked, stop-ask. It also returns the exact reference file to open and the exact set command to run next.
• The skill. corpus-handoff, the only agent-facing interface. corpus-work never touches the file, and never touches the tool’s flags.

The skill is the interesting layer, and it’s the one I wouldn’t have built unprompted. Its content is almost entirely about obligation — the bridge between a typed answer and what a model is allowed to conclude from it. The action table is that bridge in miniature: a mechanical enum on the left, a sentence about judgment on the right.

Plus the line that keeps the layer from decaying into advice: show is not optional and its result is not advisory. Skipping it and reading the frontmatter yourself reintroduces exactly the assumed-it-merged bug the whole thing replaced.

This is the same ladder as the enforcement layers, read from the other end. There, the question was which layer can apply a rule without anyone choosing to. Here it’s which layer can answer a question without anyone choosing to — and the skill exists precisely because the answer still has to be handed to something that reasons.

The challenge that found the defect

Then the design got pushed on: is the split really “what’s next” mechanical, “commit to it” judgment?

Where it held: show is pure computation with no side effects and can run any number of times; set is the only mutation; the tool never calls set itself. A recommendation can sit in front of you indefinitely without the checkpoint moving. The code already enforced that — I just hadn’t stated it.

Where my implementation failed its own claim: next_phase was not mechanical. Phases 2–4 each implement one test layer and are conditional, and which layers a feature runs was decided in phase 0 — as prose, in the Plan section, which the tool cannot read. So for a feature with no integration layer, show would have confidently returned phase: 3 and the skill would have had to overrule it.

Worth being precise about what kind of defect that is, because my first description of it was too strong. Nothing was failing. The model has always got this state machine right — reading the plan, noticing a layer wasn’t in scope, routing past it. What I found wasn’t a bug caught in the act. It was that the correctness was resting on the model reading prose carefully, every run, indefinitely — and I was in the middle of building a tool whose entire premise is that nothing load-bearing should rest on that.

Which is the more useful version of the point anyway. A mechanical answer derived from incomplete data isn’t a neutral non-answer — it’s a confident wrong one that a human record then has to contradict, which is exactly the two-records-disagree shape I was removing. The door was open. Nobody had walked through it yet, and “not yet” is not a property you can schedule around.

Worse, it doesn’t get to pick its moment. A latent gap like that fires in the middle of some larger piece of work — an interruption to something already half-built, which is the most expensive order in which to discover anything. The obvious objection is that closing a door nobody has walked through is premature optimisation. But that objection was always a claim about price, not about correctness — and AI has brought that price down massively.

The guard here was one schema field in the handoff’s YAML frontmatter — layers, the list of test layers this feature actually runs, defaulted at init, revisable mid-flight when a layer turns out to be unnecessary — plus the branch in the resolution function that routes past the phases those layers don’t cover, a validator rule rejecting any checkpoint parked in a phase its feature never runs, and the tests around all three. Small, but exactly the kind of small that got deferred anyway when it had to compete for the same hands. When building the guard gets cheap enough, “we’ll deal with it if it happens” stops being prudence about effort and becomes a bet that the interruption will land somewhere convenient. It’s the same shift that made a 100% coverage gate affordable: the standard didn’t get better, the tax on holding it collapsed.

The fix was to make the layer decision data. layers is now a checkpoint field. Routing skips absent layers and says so — skipping 4 (e2e) — not in this feature's layers — rather than looking like a miscount, and the validator rejects a checkpoint parked in a phase its feature never runs.

The rule that came out of it went into the skill verbatim:

Anything the tool needs in order to answer correctly belongs in the schema; anything requiring judgment stays with you.

With the second half enumerated, so it can’t quietly migrate: whether to start the phase at all, whether a failure is a wrong test or a wrong contract, whether a layer turned out to be unnecessary, and every human gate.

What the tool refuses

The schema isn’t held up by good intentions. pnpm handoff validate runs on every commit through the pre-commit hook, so a malformed checkpoint fails at the commit that introduces it rather than several phases later, when it would be the last thing standing between the branch and the final PR.

The same rules run inside set. The tool validates the transition before performing it, so a rejected change leaves the file exactly as it was rather than half-applied:

error: that change would leave the checkpoint invalid — nothing was written
issues[1]:
  `phase: 2` with no `test_baseline_commit` — implementation is measured
  against the merged phase-1 specification, so the baseline is pinned
  before phase 2 starts

  `active_branch` is still set while `phase_status: complete` — clear
  evidence when a phase ends, or a later run resumes a branch that
  already merged

Both errors name the rule rather than the field, which is what makes them usable by the thing reading them. And the skill closes the obvious escape route: hand-editing costs you the validator and gains nothing — if a transition seems impossible through the tool, that’s a signal the transition is incoherent, not that the tool is in the way.

Write the tool for the agent; the human reads it anyway

The steer I had to keep repeating while the tool was being built: its audience is a peer, not a person. Left alone, an agent writing a CLI writes it for a human — progress lines in stdout, help text where content should be, a nicely aligned table, an error phrased as an apology. Those are the conventions in every CLI it has ever read, so they’re the default, and saying “this is agent-facing” once doesn’t hold it. It has to be said again at the next design decision.

What changes when the reader is an agent is specific enough to have a standard, and the tool follows it: AXI, the Agent eXperience Interface. Output is TOON rather than JSON — the standard puts that at roughly 40% fewer tokens for the same content. Errors go to stdout, in the same shape as success, because an agent doesn’t read stderr and an error it can’t see is an error it retries blindly. Exit codes mean something: 0 including no-ops, 1 error, 2 usage. Nothing prompts interactively. And every response carries its own next step:

resolution:
  action: advance
  phase: 1
  reference: .agents/skills/corpus-work/references/phase-1-tests.md
  reason: phase 0 is complete — route to phase 1
help[1]:
  Run `pnpm handoff set COR-999 --phase 1 --status not_started`

That help line is the entire ergonomic argument in one row. The expensive token cost is rarely a longer response — it’s the follow-up call. So the command the agent is about to have to assemble gets handed over already assembled.

Then the asymmetry that makes this a free choice rather than a trade: I can read that too. It’s English tokens in a shape a person parses fine on sight, arguably better than the aligned table I’d have asked for, since nothing is hidden behind a column width. The reverse does not hold. An agent handed human-shaped output reads Fetching data… as data, can’t distinguish a truncated list from a complete one, and never sees the error on stderr at all.

So the two audiences aren’t symmetric, and the tie goes to the agent every time. Optimising for the human costs the agent real capability; optimising for the agent costs the human some polish. That isn’t a close call — it’s just a call the default pulls against, which is why it has to be made out loud and then made again.

Prose was already working. That’s the argument, not the objection

The thing I want to be careful about claiming: the harness was already behaving well. Tickets in Linear with real acceptance criteria, phase instructions written as prose, a gate that costs you a review round rather than a merge — that combination has been producing work I’d defend for weeks. This isn’t a rescue.

It’s the same designs, moved down a layer. Every rule in that frontmatter schema was already written somewhere in the skill’s prose: don’t start phase 2 without a merged phase-1 baseline, don’t leave a branch recorded once a phase completes, don’t cut a second branch for a phase that already has one, don’t assume a recorded PR merged. Prose stated them. A model followed them, mostly, and I couldn’t tell the difference between “followed” and “happened not to be tested” from the outside. Now the transition that would violate one of them exits non-zero and writes nothing.

Two properties change when a rule moves from a paragraph to a validator. It stops being negotiable — there’s no reading of phase: 2 with a null baseline that gets through, and no deadline under which it becomes reasonable. And it stops being distributed: eight reference files each restating a fragment of the state machine becomes one schema, one resolution function, and one skill that says what each answer obliges. The failure I opened this post with was a coordination failure between three records. Centralizing was never about tidiness — a single record is the only kind that can’t disagree with itself.

There’s a third property, and it’s the one that decides whether any of this was worth building. To trust a guarantee written in prose, I’d have to evaluate whether the model follows it — across phrasings, across context lengths, across the runs where it’s mid-task and reasoning about something else. That’s an open-ended behavioural eval. It’s expensive to build, expensive to run, and it goes stale the moment either the prose or the model changes.

Putting the guarantee in a validator collapses that surface to a single question: does the agent actually call the skill and the tool? Everything downstream of the invocation is arithmetic, and arithmetic is covered by 36 unit tests with no model in them at all. The remaining eval is binary, cheap, and observable in a transcript rather than inferred from the quality of the output.

That’s also the honest statement of what’s still unguarded, and it lands exactly where the enforcement ladder already put skills: triggering on model-judged intent, the probabilistic layer. I haven’t removed the probability. I’ve concentrated all of it into one event I can watch, which is a strictly better place for it than distributed across every rule in eight reference files.

What I’d resist calling it is a railroad, at least not yet. The tool can refuse an incoherent transition; it can’t make anyone run show in the first place, and it has no view on whether the work in the branch is any good. It narrows the space of states the agent can leave behind. Everything the model does inside a phase is still governed by prose, review, and the same semantic residue a human has to read. The gate got harder; the judgment didn’t move.

Where I’d temper this

Nothing here has been exercised. The change can’t dogfood itself — running the pipeline on it would have created the tickets it forbids — so the substitute was walking the finished instructions against ten forward-test scenarios, six of them driven through the real tool and four confirmed only as instructions. That’s a walkthrough, not a run. The first real exercise is the next ticket, and the rubric argument applies unchanged: a mechanism nobody has seen fire is not known to work.

The one real gap in this whole session wasn’t found by the validator, the 36 tests, or the four green gates — none of which would have gone red, since nothing was actually failing. It was found by someone reading a plan and asking whether the boundary I’d drawn was the one I’d actually built. A gate can catch a rule being broken. It cannot tell you a rule was never mechanised in the first place.

The tool tells you what’s next. It still can’t tell you that what’s next was computed from a field you forgot to give it.

There are changes in my repos with no ticket, no scoping artifact, and no phase PR. Not because the harness broke — it works fine. Because I didn’t invoke it.

The part worth writing about isn’t that I skipped it. It’s that the skipping is binary. I don’t run an abbreviated version — a thin ticket, three of the eight sections, a scoping phase whose artifact I drop. I run none of it, and then I run all of it again on the next thing that clears the bar. The usual diagnosis for that pattern is discipline, and I think the usual diagnosis is wrong.

I also thought I knew which work was falling through, and I was wrong about that too. I counted, about two thirds of the way through writing this, and the count is in the middle of the post rather than at the end because it changed what the rest of it says.

A process with one mode is a step function

Ceremony cost is roughly fixed. Task value varies over three or four orders of magnitude. Put a fixed cost against a variable payoff and you get a crossover point, and below the crossover the rational move is not to pay less — there is no less — it’s to pay nothing.

That alone explains a gap. It doesn’t explain the shape of the gap, and the shape is the interesting part.

A pure cost calculation predicts a gradient, and my harness is built for one. It has a named escape hatch: a bug or a DX-only improvement with no new user-observable surface skips the four phases and goes straight to a single PR, after an explicit propose-and-confirm. There was a two-tier inspection dial next to it, until I deleted it — which turns out to be the best single piece of evidence in this post, so I’ll come back to it.

The hatch is real and it gets used. And most of what I commit still goes around it entirely. That’s the more useful version of the observation, because it rules out the obvious fix: the tier existed and I bypassed it anyway. Three things explain that:

The discount is on the wrong axis. The escape hatch cuts four phases to one, and phases are mostly the agent’s work. What it doesn’t cut is the cost of entering: a ticket has to exist, the scope has to be written, the propose-and-confirm is an exchange I have to have, and there’s still a handoff doc and a PR at the end. That’s a large discount on the cheap half and almost none on the expensive half. The crossover is set by entry cost, and entry cost barely moved.

Nothing marks which parts are load-bearing. Every section of a ticket template looks deliberate, because it is — each one got added by someone solving a real problem. Which of them earn their place on this ticket is a different question, the answer changes from ticket to ticket, and nobody could have marked it in advance. So the trimming has to happen at write time, by the person least inclined to do it. “Just fill in the useful parts” isn’t an option you can execute; it’s an option you’d have to design.

Partial compliance feels like a lie. Total avoidance feels like a decision. This is the asymmetry that makes the edge sharp. A ticket with half its sections empty is a visible failure to do the thing properly, sitting in a tracker with my name on it. No ticket at all is just a small change that happened. One of those is embarrassing and the other is invisible, and they’re both the same underlying choice. Nothing about the system makes it easier to be honest at low effort than to be absent at zero.

So the sanctioned cheap path is cheaper than the full process and still dearer than nothing, the operator opts out whole, and the process reports full compliance on everything that reached it.

Cheap generation moved the crossover the wrong way

Here’s the part that’s specific to now, and it’s why my own harness got harder to justify without changing a line.

Ceremony is a human-executed cost. Deciding what the thing is, writing the scope, reviewing what came back. It sits entirely in the bucket that didn’t get cheaper — the reading-and-deciding bucket, the one that got worse because there’s more output competing for the same attention. Implementation sits in the bucket that collapsed.

Hold ceremony cost fixed at C and let build time B fall. Ceremony’s share of the task is C / (C + B), which does this:

That table is arithmetic, not a measurement. I haven’t timed my own tickets, and I’d rather say so than dress an identity up as data.

The direction doesn’t need the data, though. Nothing in the last two years reduced the cost of deciding what a thing is and reviewing what came back; a great deal reduced the cost of building it. So every harness that was proportionate at the top row has been sliding down the table ever since.

Every row is the same harness. My process didn’t get heavier; the work got lighter, which is arithmetically the same thing. And there’s no event for it. No commit moves a design across a row, so nothing ever prompts the re-read.

And the collapse isn’t uniform across kinds of work. A feature still has a build phase long enough to keep ceremony’s share respectable. Editing a Markdown rule doesn’t — the build time for “add a rule to the rubric” is close to zero, so ceremony is essentially the entire cost of doing it properly. Anything whose deliverable is a paragraph sits at the bottom row of that table permanently.

I wrote that as a prediction and then went and checked it, which is the next section, and it’s the point where the post stopped being the one I set out to write.

A process can only measure the work that agreed to be measured

This is the line I’d keep from the whole post.

I review my tickets and my scoping artifacts and I conclude the harness is working. It is — on its sample. The sample is “tasks that cleared the crossover,” which is the definition of the tasks the ceremony was cheap relative to. Every metric drawn from a process’s own artifacts is computed over the subset that opted in.

That makes the record worse than no record, for the same reason a rendered view becomes dangerous once it’s committed: people trust it. A tracker with nothing in it is obviously incomplete. A tracker with forty well-scoped tickets reads as the history of the project, and there’s no field in it for the eighty changes that didn’t come through.

You can’t detect this by asking, including asking yourself — the whole point is that the skipped work didn’t feel like skipping. You have to count from the side that can’t be avoided, which is the repository. Two queries, and I was confident the first one would be the informative one:

1. Merged PRs with no ticket reference. The classic hole.
2. Mainline commits that never came from a PR at all.

So I ran them

The window is 2026-07-11 to 2026-08-15 — the day after the harness skills first landed, to the day I’m writing this. 39 first-parent commits on main.

The first query came back empty. 37 merged PRs, 36 of them naming a COR- ticket. The one exception is a turbo version pin. The metric I’d have led with detects nothing here, and that’s worth recording rather than quietly dropping, because a null result is ambiguous until you know why it’s null. It’s null because PR discipline isn’t where this process leaks.

The second query is where everything was.

Then the split that actually matters, which is what those commits changed:

Product code has a spotless record. Every feature went through the four phases or the hatch, every PR named its ticket, and not one line of application code has reached main unreviewed since the harness existed. The single product file in the direct column is a test file that rode along inside a harness commit. The gate works exactly as advertised.

Everything that went around it is the harness. Ten of those commits edit .claude/skills/ — the files that define what the phases are. Three edit docs/rubrics/, including the commit that first added the testing rubrics I’d just spent three posts on.

The process governs the product completely, and does not govern itself.

I had this backwards for the entire first draft. I assumed the leak was small work, and that the fix was a cheaper tier. Small work isn’t the category. Rules are the category.

The receipt

The sharpest one isn’t an addition. On 2026-08-09, direct to main:

chore(skills): drop inspection mode from corpus-work and corpus-ticket

Standard/Exhaustive added a why-trail requirement at every hard gate, all of which fire before the PR is reviewable — ceremony that cost context without changing what got reviewed.

That commit deletes a governance tier out of both skill files. It’s why the inspection dial I described in the harness posts isn’t in the list of tiers above: it’s gone. I still think the reasoning is correct — it’s the argument this whole post is making, in a commit body. But a change that removes a review requirement landed with no ticket, no PR, and no reviewer, and the justification I just quoted approvingly is one I wrote and nobody else has read until now.

Which is precisely the failure the hard-gate post named and that I was most confident I’d avoid: a wrong rule amortizes a wrong standard. That post’s own closing move was that rules need the same gate the tests got. Five weeks of commits say they never got it — and not because I changed my mind. Because the only gate on offer was priced for a feature.

The fix that makes it worse

The instinct on seeing that ratio is to close the hole. A commit hook that rejects a message with no ticket ID. Branch protection requiring a linked issue. Required fields that won’t let you save.

That converts avoidance into fake compliance, and fake compliance is strictly worse.

Pre-2023 it was at least weakly self-limiting: filling in eight sections of boilerplate to land a typo fix cost enough that most people would either do it properly or give up loudly. The cost was a bad proxy for care, but it was a proxy. That proxy is now gone. A model will produce a fluent, well-structured, entirely plausible eight-section ticket for a one-line change in about four seconds, and it will do it every time without ever getting annoyed enough to tell you the format is wrong.

Same shape as the beautiful ignore-comment reason, one level up. So:

Anything you mandate that a model can produce, you have mandated nothing. You’ve mandated the string, and the string was never the point.

And you’ve paid for it by destroying the one signal you had. Bypass leaves a hole you can see. A mandate leaves a filled field you’ll believe — present, readable, well-formatted, and doing nothing, which is the inert-is-worse-than-broken failure with a compliance rate attached. The metric goes to 100% and the information content goes to zero, and those two things happen in the same commit.

Who executes the step, and which budget does it draw on

The useful audit question isn’t “is this ceremony valuable.” Everything in a template is valuable to someone. It’s:

Harnesses grow downward through that table. Adding a field to a human-facing template is the single easiest change to make and the only one that requires no implementation, so it’s where accumulation lands by default. Each addition is individually defensible and individually small. The sum of them is the crossover point, and no single commit ever moved it.

The corollary is more optimistic than it sounds: a lot of what’s in my human-facing ceremony is there because, when I wrote it, a human was the only thing that could execute it. That’s a stale assumption in most cases now. Moving a step from row three to row two isn’t a reduction in rigour, it’s a change of payer — and it’s the only kind of ceremony reduction that doesn’t cost you the ceremony’s function.

Make the tier the artifact

The wrong response to “the step is too high” is to lower it uniformly. That just relocates the crossover and loses the rigour on the work that warranted it.

More steps is the right shape, and I want to be careful here, because I already had a second step and it didn’t work. So the shape isn’t the hard part. Tiers only work under one constraint, and it’s the one my own escape hatch misses:

The cheapest sanctioned path has to be cheaper than bypassing it. If bypass is git commit -m, the small tier is one line in the commit message — not a short form, not a “lightweight” template with four fields. Anything heavier than the bypass is a tier that will never be used, and you’ve added documentation instead of a path.

Measured that way, my escape hatch isn’t a small tier at all. It’s the full entry cost with the phases removed, which prices it against the four-phase path rather than against git commit — a discount computed relative to the wrong alternative. That’s the mistake I’d expect to be common, because from inside a process the interesting variation is all at the top end. The tier a process actually needs is the one immediately above doing nothing, and it’s the one nobody designs.

Then borrow the move that makes the coverage gate honest — three exits, and the diff won’t merge until you’ve taken one. The tier choice itself becomes the recorded decision. Not “did you follow the process,” which is unanswerable and, once mandated, dishonest — but “which mode did you pick,” which is one greppable word that someone can disagree with six weeks later. A wrong tier is reviewable. An absent process isn’t there to review.

The honest catch: tier boundaries drift downward. Everything becomes small, and you’ve built bypass with a permission slip. The countermeasure is that the distribution is the metric, not the compliance rate. My own numbers make the case: 36 of 37 PRs cited a ticket, which is a compliance rate of 97% and tells you nothing, while the distribution says two thirds of my commits never entered the system at all.

And the count says something about which tier to build, which I would have got wrong from the armchair. I’d have designed a tier for small changes, sorted by size. The work falling through isn’t sorted by size — it’s sorted by kind, and the kind is rule changes. A rubric line and a feature are not the same object and shouldn’t share a pipeline: one needs acceptance tests and a running-feature review, the other needs someone to argue with the rule before it starts applying itself to everything downstream. What a rule change actually wants is the thing that made the rules trustworthy in the first place — planting the violation and watching it fire — which is cheap, mechanical, and has nothing to do with four phases.

The ceremony that survived

Sorting my own harness by what I still actually do, the split isn’t by value. Some of what I skip is more valuable than some of what I keep. It’s by who initiates.

The things that survived — dependency-cruiser, the architecture test, types, lint, the coverage threshold, the phase PR that won’t let phase 2 start — all fire on their own, at an event, whether or not I remembered them. The things that eroded are the ones where the first step is me deciding to begin. That’s the same three-tier table from the hard-gate post, read from the failure side: prose written for humans “needs discipline,” and discipline turns out to be a budget I’ve spent by lunchtime.

So the rule I’d put next to the other ones:

Ceremony you have to remember to perform will be performed exactly as long as the work is expensive enough to justify it, and not one task longer. The only durable ceremony is the kind that happens to you.

Which reframes what configuring a quality gate is even for. I’d been treating “find a mechanism that applies this without anyone choosing to” as an anti-forgetting measure. It isn’t. It’s an anti-economics measure. Forgetting is random; opting out is systematic, it concentrates on one category, and it leaves a record that looks complete.

There’s a structural reason the harness ended up as the uncovered category, and it isn’t only price. Every gate I have is bound to an event in the product’s lifecycle — a commit, a push, a PR, a merge. Changing a rule isn’t an event in that lifecycle. It’s an event in the lifecycle of the thing that watches the lifecycle, and I never built a clock for that one. The layer that governs is the layer with no events in it, which is why it ends up ungoverned by default rather than by decision.

Limits

I’m one person, and ceremony has a function for teams that it doesn’t have for me — consent, coordination, and giving people who weren’t in the room a way to object. Bypassing in a team also has a different cost: someone else finds out from the diff. Read the tiering advice above with that in mind; the crossover point moves when the record’s audience isn’t just you.

There are also contexts where the ceremony is the deliverable — regulated work, anything with an audit obligation — and “nobody reads it” isn’t a defect in the process, it’s the process functioning. None of this applies there.

The count is one repo, one person, five weeks, and 39 commits. It’s enough to falsify my guess about which work leaks — that part is unambiguous — and nowhere near enough to establish that rule changes are the leaking category anywhere but here.

And the obvious one: this is a post in which I describe my own avoidance and then explain why it was rational, which is precisely what someone being lazy would write. The check that would settle it is whether the un-ceremonied work caused problems the process would have caught. The one piece of evidence I have points the wrong way for me — the three consecutive commits that each reversed the rule in the commit before were exactly this category, landing exactly this way. Three reversals is a small sample and I found them by reading, not by measuring, so I’d rather call it a warning than a result. What I can’t do is claim nothing went wrong, because I never had anything watching.

So the fix I’d defend is narrower than the essay and it’s specific: whatever governs your work needs a gate of its own, priced for a paragraph rather than a feature, fired by an event that exists in the rules’ lifecycle rather than the product’s. Everything above is the argument for why nobody builds that one — a process you can’t afford isn’t a lighter process, it’s a process you don’t have plus the belief that you do, and the belief is the expensive half. Mine cost me five weeks of unreviewed rules while I wrote three posts about how well the gates were working.

It worked. It found blind spots in our tests passively, week after week, in a way no review round ever did — not because we were disciplined, but because the tool didn’t need us to be. And it was tedious enough that I would not have recommended it to anyone.

The tedium had a specific shape, and naming it matters, because it’s the part that changed. The gate never left you wondering what to test — it printed the file and the line. The cost was that writing the fifteenth test for a default: branch nobody will ever hit costs exactly as much as writing an interesting one. Sustained tedium is how a standard dies: not repealed, just quietly lowered to 85 during a bad sprint.

Test code is now the cheapest thing in my pipeline. So the standard is worth another look, and I think it’s better now than it was then — for a reason that has nothing to do with generating tests faster.

The gate the model can’t argue with

I keep coming back to the same want: assertions that hold without the AI’s cooperation. Coverage instrumentation is nearly the ideal shape of one. It’s a counter, not an opinion. It doesn’t read the diff, has no taste, cannot be persuaded, and has no view on whether this particular exception is reasonable given the deadline. It reports which lines executed. The line ran or it didn’t.

Compare that to the layers I actually rely on for semantics: a reviewer agent’s findings, a rubric applied by a model, a doc that gets followed. All useful; all model-judged. Coverage is arithmetic. That makes it a poor quality bar and an excellent blind-spot detector — a distinction I’ll come back to, because conflating the two is how 100% coverage earned its bad reputation.

Why 100, and not 90

The number isn’t ambition. 100 is the only threshold with no argument left inside it.

Any number below it has to be defended, and re-defended every time someone wants to land something. Nobody has ever won 85-versus-90 on merit; the winner is whoever is more tired. At 100 there’s no debate to have, and — more to the point — nothing for an agent to negotiate with either.

Below 100, global and per-file thresholds are two different tools with two different failure modes, because a well-tested core carries an untested module. At 100 that distinction collapses: 100% global is 100% per file. Vitest even has a shorthand for the case where it stops mattering.

coverage: {
  thresholds: { perFile: { 100: true } }
}

And only at 100 is the check monotone in the direction you need. Any newly uncovered line fails, so the report is a diff. At 92, you can add a pile of uncovered code, stay at 92, and learn nothing.

Which is also why I’d skip the ratchet. Tools will offer to raise the threshold for you as coverage improves — thresholds.autoUpdate in vitest, the same idea by other names elsewhere. A ratchet beats a static 80. It’s also a gate whose position is set by whatever last landed, which means the bar is an artifact of your history rather than a decision. 100 is the only value a ratchet converges to that nobody had to pick.

Three exits, and you choose one in the diff

The gate’s real effect isn’t that it forces tests. It’s that an uncovered line has exactly three exits, and the diff won’t merge until you’ve taken one.

The middle row is the one I underrated at the time. A hard gate prices defensive code, and a surprising amount of defensive code turns out not to be worth its price. Discovering that an if (!x) throw cannot be reached from any caller is a design finding, and it arrives as a coverage failure rather than as someone’s opinion in review.

The third row is the whole design. Without it, a 100% gate is a lie factory — people write tests that execute lines for the sake of the number, and the gate reports green over a suite that asserts nothing. With it, the number stays honest and the exceptions become a short, greppable list you can actually read.

The escape hatch is where all the judgment went

The hatch is a comment: /* istanbul ignore next */, and its siblings for if, else, and file. Istanbul tolerates trailing text, so the convention is to attach the reason to the hint itself:

/* istanbul ignore next -- process.exit(); the test runner would die with it */

Istanbul does not care whether you wrote one. So the policy that makes this design work is not a coverage setting at all: the coverage tool enforces the number, and something else has to enforce that the reason exists. That split is the same shape as the config block that did nothing: a bare ignore and a justified ignore look equally deliberate in a diff, and only a second mechanism tells them apart.

The first rule I’d carry over unchanged: a bare ignore fails the build — it’s an unexplained hole in the number, with no record of who decided it was fine.

The second one I had as istanbul ignore file is banned outright, and that’s too strong. Debug harnesses, CI fixtures, and modules whose only job is to be driven by an integration test are all real, and demanding unit coverage of them manufactures exactly the assertion-free tests this gate is supposed to catch. A file that is covered somewhere else is not the same object as a file nobody tested.

What I’d defend instead is narrower, and it’s about where the decision lives: a whole-file exemption is the only exception whose scope grows by itself. Every other ignore covers a fixed branch. This one covers code nobody has written yet — including the module that quietly grows inside the fixture six months from now. So the rule isn’t “never”; it’s that the exemption has to be visible from outside the file it exempts, and it has to say where the coverage actually comes from. Three moves, in the order I’d try them:

1. Merge the coverage. If an integration suite genuinely exercises the file, the honest fix is to make the number say so rather than to carve out an exception — blob reports plus vitest run --merge-reports, or nyc merge on the istanbul CLI. No exemption at all, and the day the integration test stops touching that file, the gate tells you. That’s the one option where the claim “it’s covered by the integration test” stays true rather than becoming folklore.
2. Exempt in config, not in the file. coverage.exclude for a fixture, or a per-glob threshold when you want a lower bar rather than none: 'src/fixtures/**': { lines: 0 }. One list, in one place, in the diff — the same enumerate-don’t-pattern-match move that keeps structural rules from dying. Keep the glob tight enough that a new module can’t drift into it by accident, because a broad exclude is the failure mode wearing a policy’s clothes.
3. ignore file with a reason naming the covering test, when neither of the above is worth the work. The check below treats it like any other hint — a reason is required, nothing more — which is the permissive reading. Rejecting it outright is defensible too. The part that isn’t optional is that something records why.

The distinction that survives all three: an exemption is fine, an invisible exemption isn’t. A comment at the top of a file is the least visible place to put a policy, which is the whole objection — not the exemption itself.

What actually enforces it

A grep, in CI. That’s the whole mechanism:

# fail on any coverage hint that doesn't carry a reason
if grep -rPno --include='*.ts' --include='*.tsx' --include='*.js' --include='*.jsx' \
     '(?:istanbul|c8)\s+ignore\s+(?:next|if|else|file)\b(?!\s+--\s+\w)' src; then
  exit 1
fi

Two things about it are policy rather than plumbing. The reason has to sit on the same line as the hint — partly because a check that looks for a hint and for dashes as two separate facts can be satisfied by unrelated code on the same line, and partly because a reason short enough to fit on the line is a reason that stays scannable. And it’s restricted to source extensions, or else the document describing the policy fails the policy, which is how a check gets deleted.

Wrap it in if ...; then exit 1; fi, never grep | grep -v. An inverted pipeline exits 1 when nothing matches, which is the success case. Get the polarity backwards and you ship a check that’s either permanently red or permanently green, and permanently green looks exactly like working.

That’s run, not sketched — a fixture of eleven awkward comments, including hints sharing a line with code, dashes with nothing after them, and reasons split across lines. It fails the seven that should fail.

One thing to get right rather than infer: the pattern has to recognise everything istanbul recognises. Istanbul anchors the directive at the start of the comment and does nothing with whatever follows, which is exactly why -- reason is a convention and not a feature. Anything narrower than istanbul leaves a comment that suppresses coverage without tripping the check; anything broader is merely noisy. Reading the two patterns won’t tell you which you’ve built — plant a bare ignore and watch CI go red, then plant a justified one and confirm coverage really does stop counting that line.

ESLint can carry the same rule if you want it at edit time rather than at push time, but not as a config line: comments aren’t AST nodes, so no core rule reaches them and you end up writing the rule yourself. I’d still start with the grep. One line, one place, no parser, no config resolution, and it sees every file type rather than only the ones your lint config parses. The editor loop is the only thing the other version buys.

Worth naming the layer above both, since neither can do it: checking that a reason exists is deterministic, but judging whether the reason is any good is a model-judged CI step. Different tier, and a different post.

Then there’s the part that’s genuinely new since the old project: a model will write you a beautiful reason. Fluent, plausible, well-formatted, and produced in the same second as the ignore hint it justifies. The gate is deterministic; the hatch is prose, and prose on demand is precisely the thing on the other side of the table.

So the sentence I’d put on the wall: a deterministic gate with a prose escape hatch relocates all of the judgment into the escape hatch. Which is fine — as long as you know that’s where it went, and you read it. The consolation is that it’s a bounded amount of reading, in one grep, and the count going up is itself the signal. Vague coverage anxiety becomes a specific list of lines someone argued their way out of.

What cheap code generation actually bought

Not what I expected. The tedium was in writing the boring tests, and yes, that’s now nearly free — the same inversion as noticing being the job while fixing is free. The gate stops trading iteration speed for blind-spot detection, which was the trade that made it unrecommendable.

But the cost didn’t vanish; it moved, and it moved toward the scarcer resource. A 100% gate generates a lot of test code, and every line of it lands in a diff I’m supposed to review. Volume is the raw material of a rubber stamp. Iteration speed I had to spare; review attention I don’t.

That’s the honest version of “AI compensates for the tedium.” It converts a throughput cost into an attention cost. Worth it — but only if something other than my attention is checking whether those tests mean anything. Which is the ceiling.

Coverage measures execution, not assertion

Istanbul counts which lines ran. It has nothing to say about whether anything was checked.

it("formats the record", () => {
  format(record); // 100% of format(), zero claims about it
});

That test takes a function to full coverage and asserts nothing at all. It’s the exact failure that rubric question exists for — would this test fail if the behaviour it names were wrong? — and coverage is structurally incapable of asking it. That question also has a name and a tool, which I’ll come back to in a moment.

Which sharpens the claim I started with. The gate an agent can’t weasel out of is one-dimensional: it must make the line execute, and no amount of arguing changes the counter. It can weasel completely on whether executing proved anything, and a green coverage threshold is a direct incentive to do exactly that. 100% coverage without an assertion standard is a machine for manufacturing tests that run code.

Paired, though, they cover each other’s blind spot precisely: coverage is deterministic about where nobody looked, the rubric is judgmental about whether looking meant anything. Neither substitutes for the other, and I’d be nervous shipping the first without the second now in a way I wasn’t years ago, because back then nothing was generating tests at volume.

The name is mutation testing, and it’s the honest next rung: Stryker and friends perturb the code and check that the suite notices, which is would this test fail if the behaviour were wrong asked mechanically rather than by a reviewer. So the rubric is the hand-rolled approximation of a check that already exists as a tool — which is a slightly deflating thing to discover, and also the best argument for eventually running the tool.

I haven’t. It costs a multiple of your suite runtime on every run, and cheap code generation does nothing about that: it’s compute, not typing. Which makes it nightly rather than per-PR, and a thing I want to try properly rather than assert about here. I name it because the temptation, having found one deterministic gate, is to assume it covers more than it does.

In a tests-first cycle, the gate grades the plan

There’s a second reading of the same number, and it only exists if the ordering is enforced rather than intended. My harness runs one phase per pull request: Phase 1 writes tests, red by design; Phase 2 makes them green. So picture Phase 2 landing with every test passing and coverage at 96%. Nothing is broken. Every assertion anyone wrote is satisfied. And yet there are lines in the implementation that no test written the day before anticipated.

That isn’t a missing test. It’s a measurement of the distance between the plan and the build, and it has exactly two explanations:

• The implementation built something nobody asked for. An extra branch, a defensive path, a case the ticket never mentioned. The plan was fine; the code is over-specified, and the fix is a deletion.
• The plan asked for it and the tests-phase rubric didn’t produce a test for it. The code is right; the rubric has a hole, and that hole recurs on every ticket until someone writes it down.

Both are findings about upstream artifacts, which is what makes this worth more than the raw percentage. Every other check I run grades the diff. This one grades the ticket — and it grades it automatically, on every cycle, without anyone deciding to ask whether the planning was thorough. That question is normally unanswerable except retroactively, after the bug.

Which is why the direction of the repair matters more than the number does. The tempting fix — write a test for the uncovered line — is the one move that’s almost always wrong here, because a test authored after the implementation in order to satisfy a threshold is derived from the code rather than from the requirement. That’s the fabricated premise in its purest form: it will pass forever, and what it asserts is that the code does what the code does. In a tests-first cycle the legitimate paths back to 100% are deleting the code or amending the plan and re-running the tests phase. Topping up coverage at the end quietly converts an implementation detail into a requirement, and the number goes green while the signal is destroyed.

So the three exits from earlier acquire an order in this setting. Delete first. Amend the plan second. The written reason last, and only when the first two genuinely don’t apply.

And it closes the loop on what makes these gates worth paying for at all: the deliverable of an expensive gate is a rule that makes the next one cheap. A coverage failure diagnosed as “the rubric never asks for an error-path test on external calls” produces a rubric line, and the next ticket’s tests phase writes that test before any implementation exists. The number stays at 100 because the plan got better, not because someone chased it.

The honest limit is the same one that makes red-by-design work: this reading requires that the tests came first and that you can prove it. Where the ordering is a convention rather than a gate, “the test anticipated the code” and “the test was written to cover the code” are indistinguishable after the fact, and a coverage gap stops being evidence about the plan.

Once you’re in that position, fabricating the premise is close to unavoidable — the only test available to you is one derived from code that already exists. Which is why the durable response to an uncovered line is not the test. It’s the rubric: write down the rule that would have made the tests phase anticipate that code, and the next cycle starts from a stronger spec regardless of how honest this one’s ordering was. A test written after the fact fixes one line and teaches nothing. A rubric line changes what gets written before the implementation exists, which is the only place the fix can actually land.

The backlog of things we knew were right

The coverage gate is one instance of a larger category, and the category is the interesting part: practices that were abandoned on labour cost, not on merit. Not the ideas we tried and disliked — the ones we agreed were correct, scheduled, and then never got to, because there were always more tasks than people. Every team I’ve been on has that list, and nobody writes it down, because a backlog item whose only blocker is “this is a lot of typing nobody will fund” doesn’t feel like a decision. It feels like weather.

Cheap code generation is a re-litigation of that entire list. Which is more interesting than it being a faster way to do the things we already do, and it deserves a filter rather than enthusiasm, because only part of the list actually comes back.

The filter is the same one that just showed up twice above: ask what the practice was actually paying for.

• Cost was writing code — the fifteenth default: branch test, the fixture that takes an afternoon, the error path nobody wanted to simulate, the property-based generators, the doc for the second consumer of an interface. These are now nearly free, and every one of them is a candidate to reopen.
• Cost was reading code — a second reviewer on every PR, a design doc per ticket, manual QA passes. These got worse, not better, because there’s now more output competing for the same attention. Volume is the constraint, and generation adds to it.
• Cost was compute or wall-clock — mutation testing, exhaustive fuzzing, full-matrix CI. Unchanged. Nothing about cheap generation makes your test suite finish faster.

Three buckets, and the whole win lives in the first one. So the question worth asking of an old abandoned standard isn’t “could an agent do this now” — it’s “was the thing that killed it typing?” If yes, it’s probably back on the table at close to zero cost. If it died because someone had to read the output, cheap generation has moved it further out of reach, not closer.

100% coverage happens to sit almost perfectly in the first bucket, which is why it’s the one I reached for first. That’s also the reason it isn’t a general endorsement: the same reasoning that revives it rules out most of what’s next to it on the list.

The setup

• Thresholds at 100 on statements, branches, functions, and lines. At 100 the per-file question answers itself.
• /* istanbul ignore <hint> -- reason */ as the only exception, plus a separate check that rejects a hint with no reason.
• Whole-file exemptions in the coverage config where they’re visible, not as a comment inside the exempted file — and merged coverage instead, wherever an integration suite already covers it.
• Then plant an uncovered line, watch CI go red, and remove it — because a threshold nobody has seen fire is indistinguishable from a threshold set to zero, and the coverage config that silently excludes half the source tree is a very common object.

The gate’s output was never really a number. It’s a list of places nobody looked, regenerated on every push by something with no opinions and no stake in the deadline. That part was always worth having. What was missing was any way to act on the list cheaply, and that’s the part that arrived.

Which is why I’d rather people take the filter than the gate. The gate is one revived standard and it might not be yours. The filter is the reusable move: go find the practice your team agreed was correct and dropped anyway, and check whether what killed it was typing.

This one is about the question I hadn’t asked either time: how would I know the rules are any good?

It came up sideways. I was mid-review on an unrelated PR, found a mock I didn’t like, and went looking for whether the rubrics already covered it. They didn’t. So I drafted new rules, checked whether a ticket already owned that work, and found the one that owns the delivery mechanism — with a priority rationale I’d written myself a day earlier:

the payoff window is the next test-writing pass; a rubric that lands a pass late gets judged by the tests written without it.

I quoted that back as an argument for urgency. The pass in question was the one I was standing in. The rubric was about to miss it. Sunk cost, get moving.

The reply I got was one sentence: if we judge the rubric on known-bad items, that’s a good regression test.

That inverts the whole thing. The tests written without the rubric aren’t evidence the rubric arrived late. They’re labelled data. Lateness isn’t the cost — it’s the only reason there’s anything to score the rubric against.

Rule 9, aimed one level up

The rubric already contains the discipline that validates it. Rule 9, verbatim:

Prove the test can fail before trusting it. Plant the violation, confirm the failure names it, remove it, confirm green. A check nobody has seen fail is not known to work.

Its stated trigger is “always for a new lint or dependency-cruiser rule; whenever an assertion has only ever been observed green.” It points outward, at machinery. Turn it around and it reads: a rubric nobody has seen catch anything is not known to work.

That’s not a cute symmetry. It’s the same failure mode I’d just spent a session on in a different costume. A dead config block and a live one look identical in the diff; a rule that catches real defects and a rule that catches nothing look identical in the Markdown. Both are indistinguishable by reading, and both stay indistinguishable until something travels end to end.

The difference is that a config block needs a sentinel I have to invent. A rubric’s sentinels already exist — I’ve been generating them for weeks and throwing them away.

The corpus is mostly transcription — this time

Here’s what made me take this seriously rather than file it as a nice-to-have.

The test file with the mock I objected to carries a header documenting two rejected earlier revisions of itself: first a hand-rolled as unknown as Context fake, then an app whose routes were () => { throw x }. Both were reviewed, argued about, and replaced. And IntegrationTesting.md Rule 3 — “use a real route table, not an inline throwing app” — was plainly induced from the second one. Same story for the unit rubric’s Rule 1, whose bad example is a hand-constructed PrismaClientKnownRequestError with an assumed P2002 code, which is a real thing someone tried.

So the specimens are sitting in the tree, with recorded verdicts, next to the rules they produced. Building the corpus is largely harvesting.

That’s worth stating as a general property: a review process that writes down why it rejected something is already producing labelled data, and most of us discard it. The rejection lives in a PR comment thread, or a commit that got amended away, or — if you’re lucky and slightly obsessive — a comment block at the top of the file. A rubric extracted from those incidents is a compression of the corpus. Keeping the corpus means you can check the compression.

And then the counter-point that undercuts my own optimism, which I’d rather state than dodge: none of that debris is guaranteed to exist any more, and agents make it scarcer, not more plentiful. Nothing about that file header was inevitable. The default shape of an agentic correction is silent repair — I say fix the mock, it gets fixed inside the turn, and the rejected revision doesn’t reach a comment thread or even an amended commit, because it was never a commit. Human review is sloppy and leaves debris everywhere. An agent handed a correction leaves none unless something obliges it to.

So the velocity that produces specimens faster also destroys their labels faster, and the two effects don’t cancel — you end up with more defects fixed and less evidence about which rule would have caught them. The specimens I just called free exist only because a habit older than the agent happened to catch them on the way past.

Which flips the conclusion rather than weakening it. If the corpus is what makes a rubric checkable, recording the verdict cannot be a byproduct I hope to harvest later; it has to be something the review step is obliged to emit, for the same reason the rubric needed a delivery mechanism instead of my memory. Otherwise the eval set is whatever survived by accident — and “whatever survived by accident” is a description of a biased sample, not a test set. The specimens that get documented are the ones someone found interesting enough to write up, which is exactly the correlation you don’t want between your corpus and your existing rules.

Two failures that look identical from outside

The immediate payoff isn’t scoring. It’s a distinction I’d been collapsing.

When a defect gets past a rubric, exactly one of two things happened:

Coverage gap — no rule covers it. The rubric had nothing to say. My jest.mock("@corpus/database") is this one, and I checked carefully: nothing in either document addresses stubbing a first-party package, and Rule 1 forbids constructing a dependency’s error class while saying nothing about substituting the class object. Worse, the unit rubric’s infrastructure section says “Prisma’s own class/code branches are not hermetic — Rule 1,” which is correct and which is probably what made the mock feel sanctioned. The rubric told the author not to test those branches hermetically; nothing told them that stubbing the classes wholesale to reach the other branches drains every instanceof of meaning. The gap isn’t an absent tenth rule, it’s a sentence that opens a door and doesn’t close it.

Compliance gap — a rule existed and wasn’t applied. This is the category the delivery mechanism addresses, and the one I’d already measured: a repo-wide grep for rubrics returned zero hits outside the rubrics’ own directory. Nothing referenced them. Any rule in there could only fire if someone happened to name the path.

These want opposite fixes. A coverage gap needs content — a new rule, or an edit to the sentence that misled. A compliance gap needs delivery — a hook, an auto-loaded file, something that puts the existing rule in front of the writer. And the failure is symmetric: conflating them is how you ship a hook when you needed a rule, or a rule when you needed a hook. I have now done both. The hook I built delivers a rubric that would have said nothing about the specific defect that motivated me to build it; the rules I drafted would have sat in a document that nothing loaded.

You cannot tell the two apart by looking at the defect. You can only tell by asking, of a specimen with a known verdict: was there a rule, and did the mechanism deliver it? That’s a two-cell answer per specimen, and it’s the whole diagnostic.

It’s scorable, which surprised me

I expected “evaluate the rubric” to bottom out in taste. It doesn’t, and the reason is that the output format already exists.

The one mechanism that worked out of the box was the external review step finding the rubric unaided. What it produced, when I probed it on a stripped-down repo:

findings citing Rules 3 and 4 by name, naming the exact toBeGreaterThan(0) trigger in the diff, and quoting the rubric’s own headline test

That is eval output. For each specimen, the question is mechanical: did the reviewer cite the rule this specimen is labelled with? Not “did it seem to review thoughtfully.” Precision and recall over a labelled corpus.

What makes that work is a property of the rubric I’d filed under style. Every rule carries a mechanical trigger — “always for a new lint or dependency-cruiser rule; whenever an assertion has only ever been observed green” — and the document says so in its own header: check the rule against the code you are about to write, not against your intent. I’d read that as good technical writing. It’s actually the precondition for labelling anything. You cannot mark a specimen as violating write clear tests, so you cannot score that rule, so you can never discover it isn’t working. A rule without a trigger isn’t merely weak, it’s unfalsifiable — and a rubric made of them stays unmeasurable no matter how good the corpus is.

Which forces one addition the session didn’t get to. A corpus of only known-bad items measures recall — how many real defects the rubric catches. It says nothing about precision, and a rule that fires on everything scores a perfect recall. You need clean specimens too: tests that were reviewed and accepted, labelled “no finding.” Without them there’s no penalty for a rule that flags every mock in the repo, and vague rules are exactly the ones that drift that way. The accepted revisions are as much data as the rejected ones, and they’re in the same file headers.

Once it’s precision and recall over a labelled set, the obvious question is whether the existing LLM-eval harnesses just do this — a diff is text, a finding is text, and scoring predictions against labels is what those tools are for. I suspect the friction is that the unit under evaluation is a repository state rather than a prompt, which may want a thin adapter or may want something purpose-built. I haven’t tried it, so that’s a separate post rather than a claim here.

The trap, stated plainly

The specimens a rule was induced from are its training set.

A corpus built only from them tells you the rules encode their own originating cases, which you already knew, because you wrote them from those cases. It measures nothing about generalisation. And I’d flagged the overfitting risk in the first post — “mock returns must be distinguishable from their inputs” generalises, “prefix staged keys with uploaded-” does not — without noticing that the risk was measurable rather than merely acknowledgeable.

So the corpus needs held-out items: defects found after the rubric was written.

I have exactly one. The mock survived review, shipped into an open PR, and the rubric was sitting in the tree the entire time. Nobody was prevented from applying it — it just wasn’t. And the verdict on it came from a human read during review, which is what makes it a label rather than just a diff.

One held-out specimen, and it’s a coverage gap. Data point one, in the actionable category.

Which reorders the work

The practical consequence is that the corpus is a better first ticket than the new rules it was supposed to justify.

I had drafted two content rules — never stub a value whose identity is the contract, and identity that crosses a module boundary gets a pinning test in the tier CI actually runs. Both feel right. Neither is demonstrably necessary. With a corpus, the argument changes shape entirely: instead of asserting the new rule would have caught this, I can show that the current rubric provably misses a specimen and the new rule catches it. That’s the difference between a rule I’m confident about and a rule with a receipt.

There’s a second thing the scores would settle that I’d been deciding by instinct. The rules are not equally mechanizable. A mock that fixes a load failure is an import-graph bug has a tell precise enough to lint for — the suite failed to load, not to assert — whereas never stub a value whose identity is the contract cannot be machine-checked in the general case. The best that one reduces to is a question a reviewer has to actually ask: if this stub were the wrong object, would any assertion in this file fail? If the answer is no, the test agrees with itself. I’d been sorting rules onto the enforcement ladder by feel. A score sorts them by evidence: a rule the reviewer reliably cites is fine left as prose, and a rule it keeps missing is either badly written or needs to stop being prose.

It also, usefully, stays out of the way. The delivery ticket’s scope explicitly excludes rubric content and machine-enforcement — it changes only what loads the rubrics. Validating a rubric is neither of those. Four ticket-sized pieces that don’t overlap: does the standard exist, does anything apply it, does anything record the verdict when it’s applied, and does applying it catch anything. I had been treating the third as free.

So when should you introduce a rubric?

The answer I started the session with was “as early as possible, and the window is closing.” That’s not wrong, but it’s answering a different question than the one that matters.

You write the rules when you have incidents, because — as far as I can tell — you cannot author them up front; every line I have was earned by watching a specific assertion fail to distinguish a specific pair of behaviours. You deliver them when a mechanism exists that doesn’t depend on anyone remembering. But you can only know they work after a pass has run without them, because that pass is what produces the labelled specimens.

Which means the uncomfortable framing is the correct one: a rubric introduced before there’s work to judge it against isn’t early, it’s unmeasured. Lateness isn’t the failure mode. Lateness is the eval set arriving.

Honest about where this stands: none of it is built. The corpus is a design and a ticket I hadn’t filed when the session ended, with one held-out specimen in it, and the reviewer behaviour I want to score was observed three times out of three, which is a signal and not a guarantee. The interesting part isn’t the artifact. It’s that “is this rubric good?” turned out not to be a matter of taste, and that the thing making it answerable was the pass I’d been treating as a missed opportunity.

It isn’t, and the reason is a division of labour that took a whole session to see clearly. Structure is checked by dependency-cruiser and an architecture test. Types and lint are checked by their own tools. What’s left for a human is semantics: does this assertion distinguish the two behaviours it needs to, is this the copy a user should actually see, is a bare ZodError a 400 or a 500. That residue is small enough to read in a PR diff — and it’s the only part where my judgment was ever the bottleneck anyway.

But the residue is only small because a lot of it got moved. This is the story of moving it, which cost seven review rounds, and of why I think that’s the correct price to have paid exactly once.

The gate you can’t skip

My build harness runs one phase per invocation, each ending in a pull request I have to merge before the next phase can start — the gated style I described in Two Styles of Agentic SDLC Harness. Phase 1 writes tests. Phase 2 implements them. That ordering isn’t a convention I’m trying to live up to; it’s the shape of the tool.

Which quietly solves the thing TDD always lost on. TDD never failed on merit — it failed because nothing stops you from writing the tests afterward, so under deadline you don’t. Here there’s nothing to resist. The agent has no ego about test-first, and it cannot peek at an implementation that doesn’t exist yet.

Better, the Phase 1 PR is supposed to be red. Mine opened at 25 failing and 44 green, with a single cause: not implemented yet. A green tests-phase PR would have meant the new assertions asserted nothing new. Red-by-design is the receipt — you can’t fake the ordering.

The gate also gives me somewhere to stand. Over seven rounds I reversed five of my own earlier decisions and the agent retracted four of its claims. None of that survives a fluid harness, where there’s no natural moment to say “not yet” without feeling like you’re obstructing your own tooling.

The ladder

Everything below came out of reading assertions in a diff. No tool found any of it.

A mock that echoed its input. Cleanup was asserted with toHaveBeenCalledTimes(2). The storage mock echoed back the key it was handed — exactly like the real gateway does — so two genuinely different implementations produced identical calls: using the key that staging returned, versus re-deriving one locally from randomUUID(). The second leaves real orphaned files in storage forever. The count passed either way. Making the mock return uploaded-${key} and asserting that prefix separated them.

The same test again, harder. A prefix only proves the deleted keys were staged keys, not which ones. So: mock randomUUID to a counter, make every key predictable, assert literally.

expect(deletedKeys()).toEqual([stagedKeyFor(1), stagedKeyFor(3)]);

That immediately exposed a wrong assertion. In the staging-failure case a.txt and c.txt succeed while b.txt fails, so two keys need cleaning up — and the count-based assertion had expected one. It had sailed through because the suite was already red for an unrelated reason.

Sit with that one, because it’s the sharpest thing in the session. A red-by-design phase hides wrong assertions inside expected failure. The property that makes the gate honest is the same property that makes a bad assertion invisible to everything except a human reading it.

Worth noticing too that the expected keys are 1 and 3, not 1 and 2 — the failing file still consumes a uuid, because the key is generated before storage is ever touched. That asymmetry now lives in the test instead of being buried in a number.

Assertions no string could fail. A helper called expectSafeFailureBody checked five regexes for what an error message wasn’t — no model name, no storage key, no Prisma error code, no env var name — plus a non-empty check. Any string satisfied all six. It never said what a user should see. Replaced with the literal copy per status:

An exact match subsumes every leak regex — a body equal to one of those three strings cannot contain a Prisma code. Two things fell out of this that I’d keep as rules anywhere.

First: the literals are written into each test file rather than imported from the middleware, because importing them makes every assertion agree with whatever the code says, including copy someone changed by accident. A test that can’t disagree with the implementation isn’t testing it.

Second: strengthening these took the suite from 26 red to 24. The leak-pattern assertions and a separate response-shape test all collapsed into one toEqual on the whole body. Raising the bar deleted test code. That surprised me, and it’s a useful counter to the assumption that a stricter standard means more to maintain.

Fabricated premises. This one took me two attempts to articulate, and the sentence I landed on is the one I’d keep:

Even though the error does have a public constructor, building it manually bakes in a presumption that X failure from PG becomes Y code from Prisma. That’s what fabricated means.

A hand-constructed PrismaClientKnownRequestError doesn’t test that we map Prisma’s behaviour. It tests that we map my belief about Prisma’s behaviour, and it will keep passing after that belief goes stale. Same flaw in different clothing: as unknown as Context in the middleware tests — a fake that silently encodes which Context members the handler happens to touch today.

The fix for that one is instructive because the obvious destination was wrong. Moving those cases to the HTTP-level test file would have made them real, but that tier is excluded from pnpm test and needs a listening service, so it never runs in CI. Real-but-never-executed is the worse trade. Hono’s app.request() was the third option: real dispatch, real Context, real onError, real Response, in-process, no port, still hermetic.

The confirmation was accidental and pleasing. After the switch the failure message reads (reached from GET /api/health) — values that now come from Hono’s own dispatch rather than a literal I typed into a fake.

Same principle produced the CI fixture. Instead of an inline throw — a throw wearing a route’s clothes, where the error never travels the path a real one would — there’s a real route table calling the real env loader and a real Zod .parse(), never mounted anywhere. Nothing in a running service can reach it, which makes it strictly safer than a flag-guarded debug endpoint somebody can misconfigure.

And the ZodError case earned its own place, because .message is a JSON dump carrying field paths and received values — ["collections", 0, "name"], expected string, received number. A handler passing that through leaks the shape of internal data structures and returns a blob where a sentence belongs. Two more rules came out of building it: use a real failed parse rather than a constructed error, and make the fixture guard itself — it asserts that the ZodError’s own message still contains the field path, so if zod stops putting paths there the test fails loudly instead of passing while proving nothing.

The interesting assertion there is 500, not 400. Request validation (the client’s mistake) and parsing an upstream’s response (not the client’s mistake) produce the identical error class, so the boundary must not guess between them: a bare one falling through to a generic 500 is the correct conservative default, and a typed wrapper is what earns a 502.

Noticing is the job; fixing is free

Here is the part that changed how I work. I didn’t write a single one of those assertions. My inputs were one-sentence discomforts:

• “The mock should return uploaded-${key} so we can differentiate errors and logs in tests.”
• “I’d rather have hard expectations of keys, not simply a count of regex matches.”
• “We should assert hard user-friendly strings, as that is what the middleware is supposed to return.”

Each came back implemented in minutes. And note that not one of them named a bug. The first was about log legibility. The second was aesthetic. The third was about intent. All three turned out to be covering real defects.

That inverts the economics of diligence. The expensive part of test review used to be fixing what you found, which is exactly why you learn to swallow the vague objections — not worth the detour. When noticing is the entire cost, following unarticulated taste is free, so I raised the complaints I’d previously have let go. They paid, every time.

It’s also what steering by asking rather than asserting turns into once the fixing is free. A one-sentence discomfort is the smallest possible version of that move: it points at something without prescribing the answer, so what comes back is an implementation I still have to judge rather than an agreement I asked for.

The skill this rewards is not “navigate the codebase fast.” It’s “notice precisely, and say it in one sentence.” What I’ve replaced the editor with is prose.

The bill

Seven rounds. That’s not slow because gating is slow — it’s slow because every catch above came out of my head at review time, which means the standard for “a good test in this repo” was unwritten and got re-derived by hand at each gate. It doesn’t amortize. The next tests phase would start from zero and pay the same seven rounds.

The gate was doing double duty: the place the standard gets discovered rather than the place it gets applied.

But it wasn’t only producing an approved PR. It was producing contract. The three copy strings, the 500-versus-502 boundary, the requirement that staging generate each key before its first await — all of it went into the handoff document, so Phase 2 has to match the tests rather than the tests being bent to fit the implementation. Those are constraints the next phase inherits instead of rediscovering.

That’s the mechanism worth naming: careful test review doesn’t just filter the next phase, it specifies it. The gate’s real output is trust in work that hasn’t happened yet.

Which suggested the fix for the seven rounds. If the gate already produces contract at ticket scope, do the same move at repo scope — write down what makes a test good here, once.

Three tiers, not two

My first attempt at that conclusion was half right, and the half that was wrong is the interesting part.

The right half: structure belongs to tooling, and tooling is better at it than I am because it never gets tired at round six. dependency-cruiser and the repo’s architecture test earned their keep twice this week. When I asked for a fixture file named routes.ci.ts, the architecture test rejected it and told me the process, not just the verdict:

“The concern implies the file” — if this is a new concern, add it to the doc first; if it fits an existing one, rename it.

So the convention got documented before the allowlist widened. Without the check, the allowlist gets quietly widened and the convention is never written down at all. And it widened precisely rather than bluntly: a plain helpers.ts in that directory is still rejected. Worth saying, because the lazy response to a failing structural check is to loosen it until it passes, and that’s how these rules die.

The wrong half was concluding that everything tooling can’t express has to stay in my head, applied by hand, forever. dependency-cruiser cannot express a mock’s output must differ from its input. So I assumed semantic rules were mine to remember.

What showed me otherwise was one sentence I typed while the rubrics were being drafted: the intended audience isn’t a human, it’s the agent that writes the next tests.

That changes the genre completely. Each rule now leads with a mechanical trigger — “you are about to write toHaveLength”, “your mock returns its input unchanged” — followed by a bad/good code pair and one line of reason. The persuasion is gone. The narrative of how we found it is gone. An agent needs a decidable condition and just enough rationale to generalise, not convincing.

So there are three tiers, and the middle one is the one I didn’t know existed when the session started:

Two rubrics exist now, one per testing domain, each organised around a single question: would this test fail if the behaviour it names were wrong? The nine unit rules are nine distinct ways of answering “no” while still looking thorough. Rule 9 is literally “prove the test can fail,” so the verified-red standard is codified rather than remembered.

Rules need the same gate the tests got

The necessary counterweight: a wrong rule amortizes a wrong standard. Tooling outliving my attention is the entire benefit and the entire risk, and in the last stretch of this session three consecutive commits each reversed something from the commit before — all of them rules.

Three lessons, and they’re the same lessons one level up.

An unfalsified rule proves nothing. Every new dep-cruiser rule got verified by planting the violation, watching it fire, removing it, watching it pass. A rule never observed failing is indistinguishable from a rule that doesn’t work. That’s how I found the bug in my own: ci-fixtures-are-test-only rejected .ci.ts importing .ci.ts, so a fixture could only ever be a single file — directly contradicting the convention’s premise, since a route table held to our standards delegates to a service rather than inlining logic. Reading the pattern didn’t reveal that. Building the two-file case did.

Enumerate, don’t pattern-match. My first rule exempted *.ci.ts from canonical naming, which meant helpers.ci.ts passed where helpers.ts is rejected — any file could opt out of the convention by renaming itself. A special-case predicate in a lint rule is one rename away from being a universal escape hatch. The fix was to stop having a predicate: routes.ci became an ordinary entry in the canonical list, so a misnamed fixture fails like any other misnamed file. The correct rule ended up tighter than both attempts before it, and review pressure moving in the restrictive direction is not the direction unreviewed tooling drifts.

A null result is ambiguous. Twice a check “looked clean” only because the check itself was broken — a grep tripping on escaped quotes, and a link-verification script that silently matched nothing. Absence of output means either no violation or your verification didn’t execute, and you have to know which. The strongest version of this bit me hardest elsewhere in the same session: I argued a failure mode was impossible on the strength of a string in a binary, wrote that into acceptance criteria as self-verifying, and was exactly wrong. So the rule isn’t only prove the check can fail — it’s prove the thing you’re relying on to tell you it failed.

One shape worth preferring, since two of the session’s failures had it: inert is worse than broken. A thing that is present, readable, and does nothing looks configured in the diff, in review, and in the file forever after. Prefer the mechanism whose failure is loud; when you can’t, write the quiet failure down before it happens rather than after.

And a small loop-closing detail I like. The agent had documented the naming loophole as an acceptable trade-off, with reasoning — and writing it down is precisely what made it reviewable, and what got it killed a turn later. The payoff of recording a decision isn’t only future legibility. A written-down bad decision is far easier to spot than an unwritten one.

The context tax

One more property of agent-facing docs, because it has no human analogue and it changed how I wrote them.

The ordinary reason not to restate another document is DRY: two copies is how the unread copy goes stale. Two of my integration rules were restating the existing testing-setup doc, which is both derivative and precisely the failure the same file warned about three paragraphs earlier. They got demoted to an “inherited policy, not restated here” pointer.

The second reason is specific to this audience. The phase reference files are re-read in full on every single run, so anything restated in them is paid for on every invocation, forever, in exchange for a sentence the agent could have followed a path to. A human skims past a redundant paragraph once. An agent buys it again every time.

That rule also caught a misapplication of itself, which is the most reassuring thing a rule can do. I’d rejected a proposed test-writing skill partly on the grounds that it would be a third copy of the same pointer — except a pointer isn’t a copy. The objection was wrong on its own terms, and the rule I was citing is what made that visible.

What the gate is for

I started out thinking the value of a hard gate is that it stops bad code. It isn’t — bad code is what the next gate stops. The value is that the gate creates a moment where reviewing tests is the only thing on my desk, so I read the mock instead of the pass count.

Which gives me the rule I’d actually defend: a hard gate is expensive exactly once per class of mistake, and the deliverable of each expensive gate is a rule that makes the next one cheap. A gate that produces no rule is pure cost. The rubric accreting is what turns “was that slow phase waste?” from a hunch into evidence — six or seven lines earned so far, none of them authorable up front, every one of them from watching a specific assertion fail to distinguish a specific pair of behaviours.

Some honest limits. This was a tests phase, which is unusually readable by design; whether an implementation diff of the same size survives PR-UI-only review is untested, and I shouldn’t pretend otherwise. A rubric extracted from a handful of incidents will overfit in places — “mock returns must be distinguishable from their inputs” generalises, “prefix staged keys with uploaded-” does not. The no-editor feeling is downstream of the checks being right, and the moment a rule is wrong I’m reading prose about code I can’t see, which is exactly what those three reversed commits were. And this all runs on the gap I’ve written about before — no CI, no branch protection, and this week the automated reviewer hit its quota two PRs running, so my read was the entire review process. A gate is not a second pair of eyes, and I shouldn’t confuse the two.

There’s also a gap I didn’t see until the rubrics were written and committed: a repo-wide grep for them returned zero hits outside their own directory. Nothing in the system referenced them. I had a standard and no mechanism that would ever apply it — which turns out to be a completely separate problem, and a harder one: Your Quality Gate Isn’t Configured Until You’ve Watched It Fire.

The obvious fix is to pick a better branch. That’s the fix I spent an hour not making, and I’m glad I didn’t, because the branch was never the question.

Three homes, none of them right

Once you accept the artifact must be committed, there are only three places it can go: main directly, a dedicated docs/<id>-ticket-draft branch, or the ticket’s eventual feature branch. I rejected main on instinct. The AI built the docs-branch version. Then I rejected the premise underneath both remaining options: a skill whose entire stated job ends at “the ticket is filed” should not be creating branches at all. Cutting a feature branch is the first step of building a feature, not of scoping one.

Which surfaced the real tension. If the skill doesn’t commit, the artifact sits untracked from the moment the ticket is filed until someone actually starts the work — possibly weeks, possibly never, one git clean from gone. The skill’s own instructions defended committing for exactly that reason:

It’s a real record of what was scoped and approved, not scratch to end the session with — this skill has no Phase 4 of its own to fold it into later, so it just stays.

So: is an untracked artifact acceptable in exchange for the skill never touching git? Yes or no, and everything else follows.

The repo had already answered

The answer was in the commit log. A commit on main from the day before, chore: remove spent Lavish scoping artifacts, had deleted two of these files with this reasoning:

Per-ticket Lavish artifacts are scoping surfaces, not durable docs … Both of these have outlived their work and their content lives in docs/.

The repo had already ruled these things ephemeral. The skill’s “it just stays” sentence had been contradicting a decision that was already made, and the draft commit I’d been reviewing argued the opposite of it — citing one of the very artifacts that had been deleted as precedent for artifacts living on main. It wasn’t wrong about the branch. It was wrong on the merits.

That’s the part worth generalizing. Two instructions in the same repo, both plausible in isolation, silently contradicting each other. Nothing catches that except reading the log before writing the fix.

Naming the tiers

The rule I landed on isn’t about artifacts at all:

• Linear is the source of truth for planning.
• OpenSpec deltas are the archive of what actually got implemented.
• Lavish artifacts are human-friendly representations of those two, and own nothing.

From which one line does all the work:

A Lavish artifact persists exactly as long as the skill lifecycle that needs it as a review surface, and no longer.

Note what that buys over “don’t commit ticket artifacts.” My two skills behave differently — corpus-ticket never commits its artifact; corpus-work commits its Phase 0 plan artifact on the feature branch and deletes it in Phase 4. Stated as two rules, that reads as an inconsistency, and the next person to tidy up will fix the wrong one. Stated as one principle, both are derivable: the filing lifecycle ends at filing, so the artifact dies there. The build lifecycle spans five phases across multiple sessions, so committing is the only way the artifact survives within one lifecycle — that’s continuity, not durability, and Phase 4’s deletion is the lifecycle ending on schedule.

The principle also overrules something I’d have kept. The earlier cleanup commit had spared one artifact because its tickets were still in Backlog — “delete when spent” rather than “never durable.” Under the stated rule that exception doesn’t hold: the filing lifecycle ended regardless of what the tickets are doing. Worth writing the overrule down explicitly, so it doesn’t read later as an oversight.

The ignore pattern is load-bearing

One consequence I initially under-called. With the skill no longer committing, a leftover artifact shows up as untracked in git status — and I described that as “noise.” It isn’t noise. My build skill’s Step 0 says:

Run git status --porcelain; if it isn’t empty, stop and ask.

An untracked artifact therefore blocks the next run of that skill, for any ticket, until someone deletes it by hand. The .gitignore entry isn’t tidiness, it’s what keeps the harness startable. And the pattern has to be surgical: it must not swallow the Phase 0 plan artifact, which has to stay committable. A subdirectory for ephemeral drafts beats a negation pattern with a growing list of exceptions — the negation pattern is always one edit away from ignoring something load-bearing.

There’s a sharper version of the classification test, too. Not “is this a reference doc?” but “was this rendered as the review surface for a skill invocation?” My two skill explainers live in the same directory and look superficially identical to the ticket drafts. Under the weaker test they’re an exception to the rule. Under the sharper one they simply fall outside it, because nobody invoked a skill to produce them. Rules with exceptions rot. Tests that partition cleanly don’t.

What this is really about

Every one of these artifacts is a view. The failure mode — the branch collision, the duplicated content, the contradicting instructions — came from letting a view quietly become a record. The commit I ultimately threw away was doing exactly that: arguing that what mattered was that the artifact “exists somewhere durable,” when the thing it represented was already durable somewhere better.

The artifact was, by the skill’s own instructions, required to match the Linear description verbatim. So the committed file was by construction a second copy of something Linear already held. That’s not a principle, that’s just a fact — and it’s the cheapest possible argument for the rule. I’d spent an hour on the principled version first.

Writing them down was the cheap part. Then a repo-wide grep for rubrics returned zero hits outside docs/rubrics/ itself. No doc, no skill, no config, no code mentioned them. I had a standard that nothing in the system would ever apply.

So the configuration problem isn’t “which linter.” It’s: for every rule you want enforced, what mechanism applies it without anyone choosing to? And then, immediately after: how do you know that mechanism is actually running? Because I got that second question wrong four separate times in one session, and every wrong answer looked like a working one.

Rank the layers by what they don’t trust

The available places to put a rule sort into a ladder — ordered not by how much they can express, but by how little they ask of anybody:

1. Structural checks — dependency-cruiser rules, an architecture test in the suite. They run in pnpm test. Nothing to remember, nothing to trigger.
2. Hooks on the write itself — PreToolUse on Write|Edit. Fires whether or not anyone wanted it to.
3. Prompt files that are loaded in full every run — my harness’s phase references. A pointer sitting there is in the context window automatically; nobody has to fetch it.
4. The agent-facing index — AGENTS.md. Depends on the index being loaded, and then on the link being followed. More on this below, because I was wrong about the first half.
5. A skill — triggers on model-judged intent. Probabilistic, and the only layer that can load a whole document before the write.
6. A flag you pass at invocation time — technically the most reliable transport on the list, and last anyway. It attaches a standing responsibility to whoever is driving.

That last entry is what the whole ordering is really measuring. The question isn’t how reliably a layer delivers its payload; it’s how much operator responsibility it attaches to deliver it at all. A structural check attaches none — clone the repo, run the suite, it’s there. A flag attaches a permanent one, on every invocation, forever, to every person and every automation that ever pushes.

The temptation is to place each rule at whichever layer is cheapest to write, which is almost always near the bottom. Adding a line to AGENTS.md costs nothing and feels like progress. The useful move is the opposite: put each rule at the highest layer that can express it, and treat everything below as backstop rather than mechanism.

The hook’s constraint chose the design

I assumed a PreToolUse hook could hand the agent a document before it wrote the file. It can’t, quite, and finding out why reshaped what I built.

Three things about the API mattered:

• matcher is tool-name-only — "Write|Edit" — but each handler also takes an if field using permission-rule syntax, which matches arguments too. So if: "Write(**/*.test.ts)" gets real path filtering, and you can route *.api.test.ts to the integration rubric and everything else to the unit one.
• additionalContext is documented to land next to the tool result. That is, after the write. An allow-and-inject hook is a revise-after nudge, not a pre-write guide.
• Hook output is capped at 10,000 characters. My rubrics are 9,382 and 8,963 bytes. Neither inlines comfortably; both together are impossible.

The only shape that gets text in front of the agent before the file exists is permissionDecision: "deny" with the pointer in permissionDecisionReason. I specified it that way first, then reversed it — and the reversal is the part worth keeping.

A hook that denies the first test-file write in a session is a hook that can stall every test-file write in the repo when it’s wrong. Allow-only removes that failure mode entirely; the worst case becomes an unhelpful nudge. And “after the write” turns out to be exactly the right moment for what the rubric actually needs to say: now go verify what you just wrote against this checklist. The constraint I’d been treating as a limitation was describing the correct design.

Two knock-ons. Once nothing blocks, firing once per session is needlessly stingy — once per test-file path per session is right, because a genuinely new test file deserves its own nudge. And the review depth I’d budgeted for building the hook dropped a full tier, because the risk it was priced against had evaporated.

The hook injects the path plus a short checklist, not the file. Discoverability is what a hook is for; content lives where content lives.

The config block that did nothing

Here’s the part I’d want someone to take away.

The repo runs an external validation pipeline on every push — review, tests, lint, docs, PR, CI. Its config file already carried a 110-line document.instructions block, added months earlier, pointing the docs step at a README and telling it what to read first. Adding a review.instructions block with a pointer to the rubrics was obviously the same move at a different step.

I convinced myself it was safe to specify without testing, on three grounds: precedent in the same file, an explicit statement in the pipeline’s own prompt that repo-specific instructions “may narrow or clarify, never weaken” — so per-step injection is a designed feature — and, decisively, a string in the binary reading contains unknown field %q. A validator that rejects unknown keys means you cannot commit a block that quietly does nothing. I wrote that into the acceptance criteria as self-verifying.

All three grounds were true. The conclusion was wrong.

A sentinel probe settled it: throwaway repo, a unique marker inside review.instructions, and a diff adding exactly the file type the instruction described. The review agent’s own account afterward:

My review prompt contained only the generic review task text. It did not contain the marker, the rubric directive, or any other text from review.instructions. I only know the marker exists because I read the config off disk myself — not because the pipeline injected it.

Confirmed on two versions, with document.instructions running as a positive control in the same probe — that step did receive its marker, which is what makes the negative result mean something rather than just meaning the probe was broken. The review step simply doesn’t consume the key. It accepts it, and silently discards it.

So the failure mode I’d argued was impossible is the one that was actually there. A config surface that tolerates unknown keys is the worst thing a gate can be, because a dead block and a live block look identical in the diff, in review, and in the file forever after. And no amount of reading the binary, the docs, or the precedent distinguishes them. Only a marker travelling end to end does.

The generalisation I’d draw: a gate is not configured when the config is written. It’s configured when you’ve watched a sentinel come out the other side, with a control proving your probe can detect a hit at all.

It has to work out of the box, or it’s just a checklist

The obvious replacement worked on the first try. The pipeline takes an --intent string, and it reaches the reviewer verbatim — the probe quoted its marker back. Better, a bare pointer was enough: given only a path, the reviewer went and read the rubric, then filed an error-severity finding citing a rule by name, identifying the exact trigger in the diff and quoting the rubric’s own checklist. Nothing needed inlining.

It also had a property I hadn’t asked for and now want everywhere: in a repo with no docs/rubrics/ at all, it didn’t skip the check — it hunted, found a copy elsewhere on disk, used it, and filed a separate warning that the substitution was unverified. A gate that tells you its inputs were missing beats a gate that quietly grades you against nothing.

And I dropped the layer anyway, because --intent is a flag.

Not because I’d forget it — though I would. Because it converts a tooling problem into a human responsibility, and that’s the specific trade I’m unwilling to make. Every mechanism I add to this harness should work the moment the repo is cloned, with nobody told anything. A flag inverts that: the gate now depends on the operator knowing that a thing needs doing, remembering what it is, and getting it right on every push — including the pushes made by automation that was written before the rule existed, and by the next person, who was never in the conversation where we decided this mattered.

That’s not a weaker version of automation. It’s the thing automation was supposed to replace. Anything re-supplied per invocation isn’t a gate, it’s a habit — and habits failing is the entire reason I was doing this. The standard already existed; what didn’t exist was anything that applied it without me. Solving that with a flag I have to remember would have been the same bug one level up.

So: prefer the layer that lives in the repo, even when it’s weaker on paper. Repo-resident and 80% reliable beats invocation-bound and perfect, because the 20% is a known gap you can design a backstop for, while the operator burden is an unbounded one you can only apologise for later. And it’s worth being blunt that “we’ll document that you have to pass --intent” is not a mitigation — it’s the failure mode wearing a mitigation’s clothes. A doc telling someone to remember a flag has exactly the reliability of the doc nobody read, which is where this whole exercise started.

Check whether the layer is already free

Then the probe went one step further, and this is the cheapest lesson in the whole exercise: I stripped the config block and deleted AGENTS.md from the repo entirely, then pushed a diff with a weak assertion in it. The review agent still located the rubric on its own and filed error-severity findings citing two rules by name, the exact trigger, and the rubric’s headline test.

The backstop was free. It came from the rubric existing at that path, and nothing else. That layer went from an acceptance-criteria block to zero work — and it is, by construction, the out-of-the-box behaviour I’d just spent two dead ends trying to build: no flag, no config, no instruction, nothing for anyone to remember. Clone the repo and it happens.

The honest caveat is in the ticket too: three of three runs is not a guarantee. This is model discretion, not enforcement, and it’s scoped as a backstop with the hook remaining the mechanical layer. But the sequencing lesson stands — before building a mechanism to feed a rule into a gate, check whether the gate already finds it. I nearly shipped configuration for a behaviour I already had.

The index nobody loads

The layer I’d called weakest turned out to be weaker still, and for a reason that has nothing to do with agents ignoring instructions.

AGENTS.md in that repo is 20 KB of conventions with ten > **Reference:** pointers in it. It is not auto-loaded into a session. There’s no CLAUDE.md, so session context carries global rules and nothing from the repo’s own guide — the file is read only if something opens it. The probe agent could see this directly in its own injected context block: my global rules were there, the repo’s guide wasn’t.

So all ten existing reference lines are weaker than they look, and the testing pointer I was about to add would have inherited that. The fix is a committed CLAUDE.md symlink, which fixes every session at once and requires nothing of anyone starting one — with precedent, since the repo already commits eight symlinks at mode 120000 for exactly this fresh-clone reason. That property is why a symlink beats the obvious alternative of telling everyone to read AGENTS.md: one is a file in the repo, the other is a responsibility handed to a person.

One assumption got written down rather than left silent: on a filesystem without symlink support, a committed symlink materialises as a text file containing a path. The layer becomes inert rather than broken. That’s the worse of the two failure shapes, and it’s the same shape as the dead config block — which is why it’s named in the ticket instead of discovered later.

Four plausible mechanisms

That’s the tally: a deny-shaped hook that would have blocked every test-file write in the repo, a config block accepted and silently discarded, a flag that outsources the work to whoever pushes next, and an index that never loads. Every one was a reasonable design. Two failed silently. One worked perfectly and still had to go, because a gate that needs a person to remember it is just that person, with extra steps.

None of them were distinguishable from working by reading — not the docs, not the binary, not the precedent in the same config file I was editing. The only thing that separated the live layers from the dead ones was a marker travelling end to end, on a repo stripped down until nothing but the mechanism could explain the result.

Which is the whole thing, really. You don’t get to reason your way to a working gate, and you don’t get to delegate the last mile of it back to yourself. You get to watch one fire, on a clone nobody prepared, with nobody told anything.

The dangerous part is the same property viewed from the other side. The AI answers the question you asked. Ask a question shaped like a request for approval and you get approval. Ask for execution and you get execution, however under-specified the request was. Nothing about the harness pushes back on a badly framed ask — which means the work that used to go into doing the thing now goes into specifying it, and every failure mode below is a version of that same shift.

Steer by asking, not asserting

The first mitigation is to steer toward a direction by asking questions rather than asking the AI to evaluate your choice directly. “What are the tradeoffs of X vs. Y” invites a real comparison. “Is X good?” invites agreement. The framing matters more than it seems like it should.

But this comes with a tension: the goal isn’t to smuggle your preferred answer in through a longer path. If the questioning is just a more elaborate route to the same rubber stamp, you haven’t gained anything — you still want the AI to genuinely challenge your design, not confirm it with extra steps.

That cuts both ways. I once asserted flatly that a design doc was no longer relevant and should just be deleted. Instead of agreeing, the AI pushed back with a concrete reason it was still load-bearing elsewhere in the codebase — and it was right. Steering-by-question protects against my confirmation bias; the AI declining to rubber-stamp my confident claim is the same discipline running the other direction.

Dissent has to be manufactured on purpose

If agreement is the default, disagreement is something you have to go build. Three things reliably produce it, and none of them happen by accident.

Give a sub-agent one hostile job and then leave it alone. A failure mode specific to multi-agent setups: spinning off a sub-agent to research some ancillary concern, then having the parent intervene before it gets to explore independently. If you spun it off specifically so it could investigate without your assumptions in the loop, stepping in early defeats the entire point. I once gave a sub-agent one mandate — attack this design, find what’s wrong with it — and it found the bug that mattered most: a success path that never marked its own work as done, so the next pass would silently redo it forever. Two ordinary review passes hadn’t caught it. The one whose only job was to find fault did.

Hand it something that already works. Reviewing a design in isolation is weaker than holding it next to a sibling that’s in production. I pointed an agent at another implementation of the same pattern and asked it to compare, not to review harder — and it found a real bug: the two designs kept their deduplication keys at different granularities, so one of them would silently drop legitimate work under load. Staring longer at the same file wouldn’t have caught that. A second working example did.

Ask explicitly for blind spots. The AI catches some incidentally, just by working through a task, but it’s far better at it when sent looking on purpose. Blind-spot detection isn’t a side effect you can rely on — it’s a mode you have to invoke.

Planning didn’t get superseded

There’s a temptation, once the agent is fast enough, to skip straight to execution. Why write a plan when you can just say “do it” and watch code appear? The plan feels like the ceremony you were finally allowed to drop.

It isn’t. Going straight into execution mostly moves the planning to a worse place: you discover the scope in pieces, mid-implementation, one surprise at a time. Every discovery arrives as an interruption to work that’s already half-done, and each one either widens the change or forces you to unwind part of it. Scope extensions ad infinitum, found in the most expensive order possible.

And it wears on you. There’s a specific frustration in watching a task you thought was nearly done sprout another offshoot, then another — the finish line moving every time you get close to it. At that point it’s not a productivity problem, it’s a morale one. Discovering scope up front is tedious; discovering it one interruption at a time, forever, is corrosive.

The plan is where scope gets bounded before anything is built on top of it. That job didn’t get automated away — if anything the AI made it more valuable, because the AI will happily execute an under-specified request all the way to a large, confident, wrong diff.

And no persona saves you here. You can open with “you are my advisor, push back on me” all you like — if the actual request is go do this, it goes and does it. The framing doesn’t outrank the instruction. Wanting to be advised and then asking to be obeyed just gets you obeyed. If you want the plan, you have to ask for the plan.

The judgment doesn’t delegate

The AI is a very powerful token predictor, but it’s still just that. It doesn’t hold the whole system in its head unless you make it, and it won’t surface the edge case you never thought to ask about. Thinking at the systems level — what happens when these pieces are assembled, what breaks at the boundary, what the unhappy path looks like — is still the engineer’s job.

Two places where that shows up concretely.

Scope boundaries. Left to its own devices, the AI will jam a piece of logic into whatever place it roughly belongs rather than drawing a hard line. It isn’t being sloppy; it just has no strong prior about where a boundary should sit, because that’s a judgment call, not a prediction. I haven’t found a prompt that reliably fixes this. But the difference shows immediately when you ask for scope explicitly instead of hoping it holds: a rename once touched six places across a repo, and asked to scope it properly, the agent split that out into its own enumerated checklist rather than folding it silently into the change that triggered it — specifically so nothing quietly went un-updated. The hard line held because I asked for it.

Prior art. Always check what already exists in the repo first. Skipping this is how you end up with duplicate implementations, reinvented utilities, or a “new” design that quietly contradicts a pattern living three directories over. I’ve watched a ticket reopen a design question that had already been settled — the answer was sitting in the architecture docs the whole time, just not in the specific place the check had been told to look. Knowing where to look is the part that doesn’t delegate.

The upside, once you’re actually steering

Do all of that and you get something genuinely new in return: throwaway scenarios and what-ifs have become stupid cheap. Spinning up an agent to explore a branch you’re 90% sure you’ll discard used to not be worth the time. Now it’s close enough to free that you explore dead ends on purpose, just to confirm they’re actually dead rather than assuming it — which is exactly the kind of cheap evidence a plan wants.

The same cheapness applies to illustration, not just investigation. lavish-axi has been invaluable for letting the AI explain complex concepts beyond what plain Markdown can carry — diagrams, side-by-side comparisons, interactive artifacts you can annotate directly instead of reading a wall of prose.

Is prompting the new coding?

I’m not sure, but it feels like it somehow. Everything above — asking instead of asserting, giving sub-agents room to fail, asking explicitly for blind spots, refusing to let “do it” stand in for a plan — is a skill, and it’s specific to working with these systems rather than a general engineering skill wearing a new hat. Whether it’s a durable discipline or just this generation’s syntax, still figuring out.

It points at something bigger, too. Learning concepts in the AI age is a fundamentally different exercise: you can learn faster than before, or skip the learning altogether and let the agent apply its own judgment. Either way the old-school route — building understanding entirely by hand — isn’t competitive anymore. If you want the learning intact, the adversarial move works here as well as it does on designs: task agents with evaluating your comprehension and your plans directly, so the AI is a skeptic to argue with rather than an oracle to defer to.

That still leaves the question I haven’t resolved. How do you avoid becoming dependent on AI to learn, when the market is pushing everyone to lean on it as hard as possible just to stay relevant? There’s real tension between “use it constantly or fall behind” and “using it constantly is exactly what erodes the underlying skill.” Arguing with a skeptic is the closest thing to an answer I’ve got — and it trades away some of the speed that made delegation attractive in the first place.

The gated style

My harness splits a feature into four sequential phases — acceptance tests, implementation, human revision, docs — each ending in its own GitHub pull request. The reason is mundane and specific: the repo has no CI and no branch protection, so a human actually reading a diff is the only thing standing between “committed” and “merged.” Racing ahead to the next phase before that review happens defeats the point of having phases at all — phase 2 needs phase 1’s tests to be the reviewed version sitting in the branch, not a version the session merely remembers writing.

A few mechanisms fall out of that one constraint:

• Tests-first as a real contract. Acceptance tests are their own reviewed PR, red until implementation exists. Driving them to green is what “done” means, not a checklist someone can talk themselves out of.
• A phase no diff can replace. One phase exists purely to put a human in front of the running feature — copy quality, interaction feel, edge cases nobody wrote a test for. Code review can’t see any of that.
• An audit trail that survives reverted attempts. A running “Dev Log” and an append-only handoff doc record not just what shipped, but what was tried and walked back, and why — the kind of thing that normally evaporates the moment you git revert it.
• Ceremony that scales down on purpose. A named escape hatch for diagnose-and-fix work, and a two-tier inspection mode, so a one-line bug fix doesn’t pay the same toll as a new feature.

The cost is exactly what you’d expect: this only works because it’s fitted to one team’s stack (issue tracker, PR flow, workspace tooling, one repo). It isn’t going anywhere else without a rewrite.

The fluid style

OpenSpec’s own words for its philosophy: “enablers, not gates.” A change folder builds proposal → specs → design → tasks, but that arrow describes what becomes possible next, not what you’re forced to do next. Discover the design was wrong mid-implementation? Edit design.md and keep going. Nothing locks. The human review is front-loaded — read the whole artifact bundle once, before /opsx:apply — rather than gating every subsequent diff.

Its standout idea is delta specs: a change doesn’t rewrite a domain’s spec, it writes ADDED/MODIFIED/REMOVED blocks against it, and those deltas merge into a permanent, per-domain spec on archive. That’s a real answer to the brownfield problem my harness doesn’t have — a living, queryable “what does this system guarantee today” that a future planning session can read without archaeology through old tickets. In exchange, OpenSpec is a generic, installable CLI that works across 25+ editors and any stack — it trades per-repo depth for portability.

The actual axis

It’s tempting to call one of these “better,” but the difference isn’t quality — it’s where each one spends its trust. The gated style doesn’t trust an unreviewed diff, so it forces a human checkpoint at every phase boundary and treats “agreed but not yet reviewed” as an unfinished state. The fluid style trusts the human to keep steering after one upfront agreement, and optimizes for how cheaply that agreement can be revised. Neither is wrong; they’re just answers to different failure modes — a bad merge with no safety net, versus a stale plan nobody bothers to update.

Stealing the good idea without the philosophy

The interesting move isn’t picking a side — it’s grafting OpenSpec’s one structurally superior piece onto the gated harness without diluting the gates themselves. Concretely: keep every PR gate, keep the tests-first contract, keep the human-exercises-it phase — and add a docs/specs/<domain>/spec.md that acceptance tests write deltas against instead of one-off narrative docs. The docs-handoff phase — which already folds a feature’s history into permanent docs and deletes the scratch copy — is a natural place to also merge that delta into the living spec, the same way OpenSpec’s archive step does. Same audit discipline, same enforcement, one more source of truth that outlives a single feature.

Where I’ve landed

I’m keeping the homegrown harness where it lives — a real project, with real review gates, is where its maintenance cost actually pays for itself, and building it has taught me more about which mechanisms matter than adopting a tool would have. But I’m not planning to rebuild it from scratch on the next project either. The plan is to run both, on different projects, on purpose: the homegrown one where control and depth matter, OpenSpec (or something thinner) where portability or a teammate’s onboarding cost matters more — and to deliberately compare notes between them, the way this post did, rather than let two separate muscle memories form in isolation.

corpus-ticket: an advisor that can say no

corpus-ticket turns a rough idea into a Linear issue for Corpus’s team — but its actual job is refusing to write a bad ticket, not writing tickets fast. The system prompt is explicit about posture: “You are my advisor, not my assistant. Never open with agreement.” In practice that means every conversation runs through a fixed set of checkpoints before anything gets drafted:

1. React to the idea — push back if it’s vague, don’t open with “Great idea!”
2. Search Linear for duplicates/overlap. Find one → branch into “is this the same thing, related-but-distinct, or genuinely separate” instead of drafting.
3. Interrogate whatever’s unscoped: problem statement, acceptance criteria, explicit out-of-scope, priority-with-a-reason, labels, dependencies.
4. Right-size it — one ticket, a linked split, or an epic — judged by number of distinct outcomes, not effort. A single feature that touches a migration, two endpoints, and some UI is still one (big) ticket; several independently-shippable pieces get split and linked (blocks/relatedTo).
5. Place it in an existing Linear project or propose a new one — no ticket ships project-less.
6. Draft it, show it, get an explicit yes, create it.

Seven hard gates sit behind step 6 — duplicate search actually ran, every interrogation item has a real answer, an explicit yes came after seeing the draft — and none of them are satisfied by “the conversation implied it.” The draft itself gets rendered as a reviewable HTML artifact (via a separate lavish skill) rather than pasted as markdown, because a ticket the user is about to bless is a sign-off, not a status update.

corpus-work: status is the resume point

corpus-work takes a scoped ticket and drives it to a mergeable PR, one phase per invocation:

Todo/Backlog          → Phase 0: Planning
Testing Scenarios      → Phase 1: Acceptance tests (Playwright or integration)
Initial Implementation → Phase 2: Implementation, drive suite to green
Human Revision          → Phase 3: Human QA + fixes
Impl. Documents         → Phase 4: Fold into permanent docs
In Review               → feature-branch → main PR, awaiting merge

The root ticket’s Linear status is the save file — every invocation reads it, finds the child ticket in flight, and picks up exactly where the last run left off. That’s what makes “let’s work on COR-5” and “continue COR-5” the same command.

The one-phase-per-run rule exists for a boring, load-bearing reason: Corpus has no CI and no branch protection, so a human reading a diff is the only gate between “committed” and “in the feature branch.” Phase 2 needs phase 1’s tests to be the reviewed version sitting in the branch, not a version the session merely remembers writing — so every phase stops at an explicit hard gate before pushing, and racing ahead defeats the entire premise.

A few mechanisms carry that rule through the details:

• Two-tier logging. A ## Dev Log in docs/handoffs/<TICKET>.md records every branch created, in full detail — feature branch, base, why — because a later run parses that to know what’s in flight. A single Linear comment on the root ticket gets rewritten (never reposted) with one short line per phase, deliberately lighter, for a human skimming Linear. Same history, two resolutions, on purpose.
• An escape hatch. Not every ticket earns four phases — a Bug or a DX-only Improvement with no new user-observable surface can skip straight to a single PR, but only after an explicit propose-and-confirm, never an inferred guess.
• Inspection mode. Standard by default; Exhaustive (one sentence of why at every gate, not just a bare yes) for anything touching auth, money, data destruction, or a ticket that’s already looped back once.
• Loop-back, not silent patching. If driving tests to green in Phase 2 reveals the Phase 1 contract itself was wrong, or a human in Phase 3 finds a real gap, the root ticket goes back to “Testing Scenarios” for another round — logged as a round, not folded quietly into whatever phase found the gap.
• A PR comment monitor triages an automated review bot’s comments right after opening a PR, so that feedback is addressed before the human is asked to look.
• Docs that outlive the ticket. Phase 4 hands the whole handoff doc — the “why” behind every decision, not just the diff — to a separate no-mistakes skill, which folds it into two permanent docs/features/<Feature>-business.md / -technical.md files and deletes the scratch handoff. Corpus’s own DocumentUpload-technical.md is what that produces: architecture, an interface contract table (every data-testid and endpoint shape a test can target), failure modes, and a “Durable Decisions” section recording things like why a RAGDocument model got renamed to RagDocument mid-feature, tagged back to its source ticket (COR-5).

Where this bites

None of this is portable, and I’m not pretending otherwise. It’s fitted to one repo’s specific gap — no CI, no branch protection — and to a team of one where “the human” and “the person invoking the skill” are the same person. The ceremony (four child tickets, a handoff doc, a Dev Log at two resolutions) is a real tax, which is exactly why the escape hatch and inspection-mode dial exist: most of the overhead is supposed to scale down when the ticket doesn’t need it, not apply uniformly.

The most recent iteration (e484887) trimmed the Linear-facing Dev Log down to one line per phase instead of one per branch, after the fuller version turned out to be noise nobody actually read on that side. That’s the harness harness — I keep finding rules that were solving a problem the previous rule created, and cutting until only the load-bearing part is left.