Single adversarial falsification gate with ordinal verdicts
ADR-0004: Single adversarial falsification gate with ordinal verdicts
Section titled “ADR-0004: Single adversarial falsification gate with ordinal verdicts”Status
Section titled “Status”Accepted
Context
Section titled “Context”Background and Problem Statement
Section titled “Background and Problem Statement”Research findings are only as trustworthy as the scrutiny they survive. The
harness needs a verification step that treats each finding as a hypothesis under
test and actively seeks disconfirming evidence (falsification-analyst agent,
scripts/falsify.sh, design spec §6b). The question is how many passes run, what
verdict vocabulary they emit, and how remediation is applied.
Current Limitations
Section titled “Current Limitations”Verification can recurse forever, and multiple competing gates would produce contradictory verdicts. Without a defined vocabulary and a termination rule, verification is neither machine-actionable nor bounded.
Decision Drivers
Section titled “Decision Drivers”Primary Decision Drivers
Section titled “Primary Decision Drivers”- Verification must be adversarial: seek to refute, not to confirm.
- The process must terminate — no unbounded re-falsification recursion.
Secondary Decision Drivers
Section titled “Secondary Decision Drivers”- Verdicts must be machine-actionable and ordered, not free text.
- Remediation must be deterministic and proportionate to the verdict.
Considered Options
Section titled “Considered Options”Option 1: No gate
Section titled “Option 1: No gate”Description: Ship findings as gathered, with no verification pass.
- Advantages: Fastest path from gathering to publication.
- Disadvantages: Unverified claims ship; the harness has no defensible quality bar.
- Risk Assessment: technical low; schedule low; ecosystem high.
Option 2: Binary pass/fail gate
Section titled “Option 2: Binary pass/fail gate”Description: A single gate accepts or rejects each finding outright.
- Advantages: Simple to reason about.
- Disadvantages: Collapses real nuance — a claim that is narrowed by evidence is neither cleanly accepted nor cleanly rejected, forcing a lossy choice.
- Risk Assessment: technical low; schedule low; ecosystem medium.
Option 3: Single gate with ordinal verdicts
Section titled “Option 3: Single gate with ordinal verdicts”Description: One adversarial pass decomposes a finding into atomic claims, runs disconfirming web search, assigns an ordinal verdict, applies proportionate remediation, and enforces a one-round rule.
- Advantages: Verdicts are ordered (
falsified= ≥1 credible source contradicts;weakened= ≥1 source narrows;survived= queries ran, nothing disconfirming found;inconclusive= could not test); remediation is deterministic (falsified → quarantine, weakened → downgrade one level, survived/inconclusive → annotate only); the one-round rule stops recursion. - Disadvantages: The truthfulness of a
survivedverdict rests on agent discipline; no gate can prove disconfirming search was honestly performed. - Risk Assessment: technical low; schedule low; ecosystem low.
Decision
Section titled “Decision”Adopt Option 3: a single gate with ordinal verdicts. The
falsification-analyst is the one adversarial pass. It writes the verdict
through scripts/falsify.sh into extensions.harness.verification, applies
proportionate remediation, and enforces the one-round rule —
falsify.sh refuses to re-falsify a finding that already carries an
attempted_at verdict and logs exactly one gate-run line so a caller can assert
the gate ran once.
Consequences
Section titled “Consequences”Positive
Section titled “Positive”- Verdicts are ordered, machine-actionable, and lead to deterministic remediation; the gate fails closed on structural conformance.
- The one-round rule guarantees termination.
Negative
Section titled “Negative”- Verdict honesty is a residual agent-integrity assumption that no gate can prove.
Neutral
Section titled “Neutral”- Findings carry their verdict inline under
extensions.harness.verification, so the record travels with the finding.
Decision Outcome
Section titled “Decision Outcome”A single adversarial gate gives every finding an ordered, actionable verdict and proportionate remediation, while the one-round rule bounds the process. The residual trust in verdict honesty is stated plainly rather than implied away, and is the same assumption that applies to any agent-driven research step.
Related Decisions
Section titled “Related Decisions”More Information
Section titled “More Information”- Date: 2026-06-23
- Source:
scripts/falsify.sh, design spec §6b (falsification-analyst)
2026-06-23
Section titled “2026-06-23”Status: Compliant
| Finding | Files | Assessment |
|---|---|---|
| Gate substrate writes the verdict | scripts/falsify.sh | compliant |
| Verdict stored in the MIF extension | schemas/findings.schema.json (extensions.harness.verification) | compliant |
| One-round rule enforced | scripts/falsify.sh (attempted_at guard) | compliant |
Summary: scripts/falsify.sh implements the ordinal verdict model, writes to extensions.harness.verification, and enforces the one-round rule via the attempted_at guard.
Action Required: None