test(issue-5): harden watcher evidence validation and add codex pass review report
- Enforce evidence-root confinement for capture_ref and 1:1 capture citation uniqueness - Add pane identity and backend identity cross-validation - Add bidirectional agreement checks between watcher/herdr manifests and scenario fixtures - Expand contract test suite to 41 tests (100% pass) - Document evidence confinement rules in herdr-runtime.md and update task.md - Archive Codex 4th review report (Job 1bd561e6: [VERDICT: PASS])
This commit is contained in:
1 parent
19d798bf8f
commit
fc0bd96d8a
6 files changed
+286
-15
No files matched your search
@@ -0,0 +1,29 @@
|
||||
# Issue #5 Phase 0 final revision review
|
||||
|
||||
Job: 1bd561e6
|
||||
Reviewer: planner-reviewer-codex-01 (planner,reviewer)
|
||||
Scope: commit 19d798b and the current five-file documentation/contract-test diff.
|
||||
|
||||
No blocking findings remain in the requested revision. No supplementary implementation edits were necessary.
|
||||
|
||||
## Findings resolved
|
||||
|
||||
- **Previous Codex P1:** Observed evidence now dispatches through explicit supported source kinds. Event/filesystem/harness observations fail closed until independent evidence schemas exist; their simulated fixtures remain accepted. Herdr evidence must pass the native validator, contain mandatory agent_status, match every declared native field, and correspond to the scenario session identity. Current additions also check pane/backend identity, evidence-root confinement, repeated capture citations, and manifest/file agreement.
|
||||
- Independently reran the exact prior finalized_candidate mutation that cited a Claude idle capture for every observed item. It now raises AssertionError at the first event item because no observed event schema exists. This closes the reproduced false-evidence path.
|
||||
- **Claude Finding 2:** baseline.md now cites registry.py:38 for VALID_STATUSES. Verified that definition directly.
|
||||
- Existing native/composite separation and simulated positive scenarios are preserved. The changes remain Phase 0 fixture validation and documentation, not a claim that the future production watcher or complete execution-attribution mechanism is implemented.
|
||||
|
||||
## Verification
|
||||
|
||||
- `./.venv/bin/python -m pytest tests/test_issue5_phase0_contracts.py -q`: **41 passed in 0.02s**.
|
||||
- `./.venv/bin/python -m pytest --collect-only -q`: **510 tests collected**.
|
||||
- Independent previous-exploit reproduction: **rejected**.
|
||||
- `git diff --check`: clean.
|
||||
- `git diff -- .agents/skills`: empty; current changes are four contract documents and one test file.
|
||||
- Full-suite evidence in task.md reports **499 passed in 848.90s** at 19d798b, followed by the current 41 contract tests after the test/document-only additions. This reviewer independently ran the current focused tests and collection, not another full suite.
|
||||
|
||||
Nonblocking documentation note: interpret the roadmap's 510-test row as its acceptance target; task.md correctly distinguishes the 499-test full run from current 510-test collection and 41-test focused execution. It should not be quoted as an independently executed 510-test full run.
|
||||
|
||||
This PASS applies to the reviewed Phase 0 corrections. Future runtime/CAS/rollback implementation remains subject to the staged gates already documented.
|
||||
|
||||
[VERDICT: PASS]
|
||||
Reference in new issue
Block a user