Quality and Reliability

Verification and the checker

Completion evidence must be tied to the actual destination and revision. A local write proves local preparation. A remote tool's success response proves what that API returned. A fresh destination read establishes what is actually visible there.

For mail, read the sent record in the correct thread and account. For publishing, inspect the public URL and intended content. For files, open the delivered version and verify audience access. For a fix, exercise the failing behavior again. For schedules, inspect the receipt for the relevant firing and detect a missing run.

Use an explicit negative result when a check is blocked. An unavailable endpoint is UNKNOWN, not clean. Keep the failing check, owner and next action visible.

A later checker can reopen missing proof, escalate repeated misses to the accountable function and reserve human escalation for the decision that needs it. Configure its cadence, recheck deadlines and authority. Do not let a checker mutate external customer tasks or widen access just because it can read them.

Failure modes and enforcement

The ten F1–F10 failure modes in the operator guide form the review checklist. Each needs an observable tell, a check, an owner and an honest enforcement state.

Static checks can catch some source drift, missing fields and unsafe substitutions. A preflight can confirm the environment or print reminders. Rendering can reveal visual defects. A destination read can test publication. None proves all the others.

For each control, record whether it is a documented practice, a source-inspected check, installed code, an observed run or a tested preventative gate. Test a deliberately failing case before relying on a passing report.

A review note should identify the artifact and revision, defect, evidence, consequence, required correction and disposition. Keep the stable F-number so later records can be compared. Do not infer universal coverage from a historical test count.

Congruency in detail

G1 Identity, G2 Containment, G3 Coverage, G4 Arithmetic, G5 Generation and G6 Installation are the six gates. Preserve their meanings across every summary, table and implementation.

The four invariants are one master per subject; a derived view points to that master; every view element is supported by the master; and required master content is reachable in the appropriate views. Audience restrictions may keep evidence private while the public view teaches the same method.

In a section-level merge, classify each element as promoted, duplicate, restricted, retained in a view or held for reconciliation. Record why. Keep the original evidence in its authorized record rather than inserting private material into the public view for the sake of apparent completeness.

Generation checks should be reproducible: rebuild, inspect the diff, and run again. An idempotent build should not change output merely because it ran twice. Test a known-bad input to prove that a gate can fail. UNKNOWN remains UNKNOWN until evidence resolves it.

A correction must reach every affected output

A corrected Knowledge Panel methodology failed to reach a final PDF. The lesson is simple: fixing the source is not enough. Trace the correction through every affected summary, table, recommendation, export and delivered copy.

1. Name the corrected canonical rule, revision and exact claim that changed. Keep earlier artifacts as historical versions.
2. List the affected outputs and their owners; mark each as updated, unaffected with evidence, or held.
3. Rebuild from the corrected source. Open and inspect the final HTML, PDF or other export, including the passage, scores and recommendations affected by the change.
4. Give the reviewer the exact exported revision and original correction. A source-file check cannot stand in for checking the final PDF.
5. Read the accepted version back at its delivery destination. Record its version and hash, reconcile competing copies, and close only when every affected output has a disposition.

Use the existing canonical audit map and its specialist skills for the subject; do not invent a competing rubric or average different exams. A contradictory finding or failed required check reopens the affected work and must be escalated with its evidence. The latest audit-method reconciliation is still being validated; this guide does not certify that release before its test and publication receipts exist.

Planned comparison pilot

The requested pilot is three organizer-selected attendees across Codex, Claude, Muse and Cursor: twelve reports. It has not started. Each run must use the same versioned inputs, selected canonical rubric, allowed sources, prompt and acceptance test. Record the actual app/runtime, source and rubric revisions, evidence, scores where applicable, missing data, failures, export hash, usage or UNKNOWN, and independent review.

Use existing subscriptions only, with no paid API or overages. Compare reports against the rubric and evidence, including whether corrections survive export; do not count agreement as truth. Participant selection, access and the actual reports remain with the pilot owner. This guide does not certify results before the pilot runs.

Reliability between scheduled ticks

A reliable design handles missing runs, expired claims, unavailable tools, interrupted handoffs and approval gaps. The nine mechanisms in the operator guide are the checklist; they are not a claim of continuous monitoring.

Define the next expected firing, grace period, success and failure receipt, missing-run alert, owner and permitted recovery for each schedule. Reconcile the destination before retrying work that may already have acted.

A fallback preserves scope, privacy, authority and total budget. During a coordination outage, continue only research or reversible isolated preparation that cannot duplicate an external action. Hold conflicting writes, sends, spending, publication and destructive changes.

If required approval is missing, the dependency remains held even when the person is silent. Continue only independent authorized work. A useful escalation states the missing decision and the safe fallback; it does not turn urgency into permission.

QA records and classes

The ladder is L0 self-check, L1 machine checks, L2 fresh-context review, L3 different-model review, L4 human owner sign-off and L5 scheduled audit. Use the same definitions on every surface.

Record the exact artifact revision, reviewer, date, result and evidence for each required rung. An L3 reviewer must be a different model and perform an independent read. An L4 authority record does not prove human quality review unless that happened too. L5 occurs later and is not a qa --level L5 command in the described packet tool.

The source class defaults are internal-note L0; repo-change L1; shared-rule L3; publish/partner/client L2 before external delivery and L3 before closure; money-credentials-delete L4 under the applicable action policy. The guide does not authorize any action merely by assigning its class.

In the documented tool, money-credentials-delete is not an external-delivery class; normal validation rejects that record. A recorded override can bypass the refusal but supplies no authority. An outgoing message needs its own appropriate deliverable packet, with separate recipient, authority and review checks.

A required rung remains required until satisfied or validly waived by an authorized party where waiver is allowed. Record FAIL and the corrective next click. Hold dependent delivery when its review or authority is missing. A passing machine check cannot substitute for judgment, permission or audience review.