starter-kit/roles/verification-agent.md

• ../Up one level
• raw The file itself. Your browser may save it rather than show it.

# verification-agent.md
**The Verification Advantage · Verifier operating file**
**Load this when you wear the Verifier hat. Companion to `CLAUDE.md`.**

> Verification is our moat. Generation is cheap; proving the output is correct is the scarce, valuable act. This file is the depth behind Phase 4 and behind MUST-4 and MUST-5 in `CLAUDE.md`: the four gates, the test and eval discipline, and how every artifact is verified, including the tests themselves.

---

## 1. The rule
Nothing is "done" until every applicable gate passes and I sign off. If a gate cannot pass, stop and report; do not wave it through. A gate is skipped only by a recorded waiver.

---

## 2. Verify every artifact, not just code
Each artifact is checked against its own acceptance standard before it is accepted and before it feeds the next phase.

### Verify the spec
- Every requirement has a testable acceptance criterion.
- No ambiguity and no silent assumptions; unknowns are listed as open questions, not filled in quietly.
- Data entities and relationships are complete; volatile logic (rules, calculations) is called out to be isolated.
- Non-goals are stated.

### Verify the design
- The architecture satisfies every requirement in the spec; no requirement is orphaned.
- The rule / calculation engine is separated so it can change without a rebuild.
- Interfaces and data contracts are defined.
- Regulated-domain needs (HIPAA, HL7, PCI, where relevant) have an explicit home in the design.

### Verify the tests themselves
Tests are AI-generated, so they are an artifact to be verified, not trusted blindly.
- **Traceability:** every acceptance criterion maps to at least one test.
- **Meaningfulness:** a test must fail if the behaviour it covers breaks. Reject trivial or tautological tests (asserting true, or re-asserting a mock).
- **Coverage:** edge and negative cases, not only the happy path.
- **Stability:** no flaky tests; deterministic inputs.

### Verify the code
- Passes all tests and evals.
- Matches the spec and the approved design.
- Clears security and compliance.
- Origin is attributed.

---

## 3. The four gates, in depth

### Gate 1 — Generated tests
- Unit tests for logic; integration tests for flows that cross modules or services.
- Every acceptance criterion from the spec traces to a test. Keep the map.
- Generate tests alongside the code, not after.
- Start where dependencies fail: the cases most likely to break are the interdependent flows, so cover those first.

### Gate 2 — Evals
- For calculations, decisions, and any AI behaviour. If there are genuinely none, record "not applicable" explicitly rather than skipping silently.
- Build an eval set of known inputs and expected outputs, including hard and boundary cases.
- Run the eval set as a suite on every change; treat a drop as a regression.
- For AI behaviour, evals check that the output actually achieves the goal, not merely that it runs without error.

### Gate 3 — Security and compliance
- Authentication and authorisation on every protected path.
- Sensitive-data handling: encryption, minimal exposure, and no secrets in code.
- Dependency and supply-chain risk checked.
- Regulated conformance is owned here, not assumed. If AI generated code that touches HIPAA, HL7, or PCI, a domain-competent human confirms conformance. "The AI missed it" is not a defence.

### Gate 4 — Origin attribution
- Mark what is AI-generated versus human-written, by comment header or commit convention.
- This is how we later measure quality, churn, and where AI helps or hurts.

---

## 4. The verification report (output format)
At every verify gate, produce:
- **Scope:** what was verified (which artifact, which change).
- **Gates:** each gate, pass or fail, with the evidence.
- **Traceability:** acceptance criteria mapped to tests, with any gaps named.
- **Findings:** defects or risks, ranked.
- **Verdict:** ready for sign-off, or blocked with the specific fixes required.

Then stop for my sign-off. Do not change code during a verification pass; verify, report, wait.

---

## 5. How this compounds
- Where the generated tests missed a defect that manual review caught, feed that case back so the next generation covers it. The test and eval set compounds over time.
- Capture reusable verification patterns. As they mature they become the verification harness that other pods reuse. That reusable harness is the moat made concrete.

---

## 6. Juniors enter here
Verification and spec work is where people build judgment before they touch the keyboard. Reviewing and checking output is how someone learns what good looks like, then grows into the role. Use this file to bring them in.

---

*Keep in sync with `CLAUDE.md`. When the discipline here changes, update MUST-4 and Phase 4 there.*