test(issue-5): harden watcher evidence validation and add codex pass review report

- Enforce evidence-root confinement for capture_ref and 1:1 capture citation uniqueness
- Add pane identity and backend identity cross-validation
- Add bidirectional agreement checks between watcher/herdr manifests and scenario fixtures
- Expand contract test suite to 41 tests (100% pass)
- Document evidence confinement rules in herdr-runtime.md and update task.md
- Archive Codex 4th review report (Job 1bd561e6: [VERDICT: PASS])
This commit is contained in:
Godopu committed 2026-09-06 14:31:41 +09:00
1 parent 19d798bf8f
commit fc0bd96d8a
6 files changed
+286 -15

No files matched your search

@@ -0,0 +1,29 @@
# Issue #5 Phase 0 final revision review
Job: 1bd561e6
Reviewer: planner-reviewer-codex-01 (planner,reviewer)
Scope: commit 19d798b and the current five-file documentation/contract-test diff.
No blocking findings remain in the requested revision. No supplementary implementation edits were necessary.
## Findings resolved
- **Previous Codex P1:** Observed evidence now dispatches through explicit supported source kinds. Event/filesystem/harness observations fail closed until independent evidence schemas exist; their simulated fixtures remain accepted. Herdr evidence must pass the native validator, contain mandatory agent_status, match every declared native field, and correspond to the scenario session identity. Current additions also check pane/backend identity, evidence-root confinement, repeated capture citations, and manifest/file agreement.
- Independently reran the exact prior finalized_candidate mutation that cited a Claude idle capture for every observed item. It now raises AssertionError at the first event item because no observed event schema exists. This closes the reproduced false-evidence path.
- **Claude Finding 2:** baseline.md now cites registry.py:38 for VALID_STATUSES. Verified that definition directly.
- Existing native/composite separation and simulated positive scenarios are preserved. The changes remain Phase 0 fixture validation and documentation, not a claim that the future production watcher or complete execution-attribution mechanism is implemented.
## Verification
- `./.venv/bin/python -m pytest tests/test_issue5_phase0_contracts.py -q`: **41 passed in 0.02s**.
- `./.venv/bin/python -m pytest --collect-only -q`: **510 tests collected**.
- Independent previous-exploit reproduction: **rejected**.
- `git diff --check`: clean.
- `git diff -- .agents/skills`: empty; current changes are four contract documents and one test file.
- Full-suite evidence in task.md reports **499 passed in 848.90s** at 19d798b, followed by the current 41 contract tests after the test/document-only additions. This reviewer independently ran the current focused tests and collection, not another full suite.
Nonblocking documentation note: interpret the roadmap's 510-test row as its acceptance target; task.md correctly distinguishes the 499-test full run from current 510-test collection and 41-test focused execution. It should not be quoted as an independently executed 510-test full run.
This PASS applies to the reviewed Phase 0 corrections. Future runtime/CAS/rollback implementation remains subject to the staged gates already documented.
[VERDICT: PASS]