maintainability-agent docs
Decisions

ADR 013 — The hostile-audit prompt

The problem

This tool is built by hostile audits. An adversary reads a change and tries to make a stated promise false: a symlinked write that escapes the grant, an empty analyzer run priced as clean, a type check that never ran reported as a pass. Every accepted finding becomes a population-derived falsifier that fails without its fix, and the goal is the one product intent already states: "A hostile audit finds nothing that a stated invariant did not already cover." The promises P1–P8 are named with the thing that would falsify each.

That loop is the highest-leverage quality process the project has, and it is the only one with no artifact. Remediation has render_ai_prompt; agent standards have render_agent_instructions — deterministic prompts the tool emits so an LLM can do the non-deterministic part outside. The hostile audit has nothing. It depends on a human hand-writing a fresh prompt into a fresh session, re-deriving context the report already holds, with audit quality riding on whoever wrote the prompt that day. It is not repeatable, not seeded, and not comparable run to run.

The obvious shortcut is the wrong one. Determinism (P1) is the product's identity — the score is reproducible byte for byte — and an LLM red-team inside the audit would give a different answer every run: it could not gate, could not be compared, and "send code to an LLM by default" is a stated non-goal. The adversarial reasoning cannot live in the deterministic core.

Options considered

  1. Embed an LLM red-team in the audit. Rejected. Non-deterministic reasoning in the deterministic core breaks P1 outright; a hostile result that changes between identical runs is not evidence, and routing the tree to an LLM by default is a non-goal. This is the same line ADR 012 drew for builds: the core does not take on a capability that trades the product's strongest claim for one activity's output.

  2. Leave it ad-hoc. Honest but wasteful. The loop that finds the real holes is the only one with no repeatable, seeded artifact; each audit re-derives what the report already computed — what ran, what did not, which promises the change touches — and the result is only as good as the prompt a person improvised.

  3. Emit a deterministic hostile-audit prompt; delegate the reasoning outward. Seed the adversary from the report and the commit under audit; the LLM does the adversarial thinking outside the process, exactly as remediation delegates the fixing. Deterministic input, deterministic prompt text, non-deterministic work kept out of the core.

Resolution

Option 3. A render_hostile_audit_prompt(report) on the existing prompt seam — surfaced as a CLI --hostile-prompt output and an MCP prompt — that produces a bounded adversarial brief, deterministically, from what the audit already knows:

The prompt is text the tool returns — never a report it writes, never code it sends anywhere itself (P1, the MCP five-artifact write boundary). The boundary sentence:

The deterministic core seeds the hostile audit; it never performs it.

Consequences