release(v4.8.7): Issue #17 follow-ups - archive the review reports and bump the lockstep version
Merge-workspace review loop (--all-reviewer): round 1 (@8241bd5) claude grok PASS, agy NOT PASS (two static wrapper-literal tests did not follow _decided/_exit_for), codex NOT PASS (R1: a yes/no decision that could not run read as "no"); fixed in541c3feand65b78b6. Round 2 (@65b78b6) 4/4 PASS. Full suite at65b78b6on macOS / Python 3.9 / herdr 0.9.3: 1730 passed / 0 failed / 56 skipped; 18 of the skips were the uninitialized nats-docker submodule in the fresh worktree, and with it tests/test_deploy_freshness.py is 31 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
This commit is contained in:
1 parent
65b78b6ec4
commit
cd82ab6e72
28 files changed
+594
-25
No files matched your search
@@ -0,0 +1,59 @@
|
||||
# Issue #17 follow-ups — Codex review
|
||||
|
||||
Job: 70d302c2
|
||||
Loop: loop-20261007T224638Z-cf1efb
|
||||
Base: 56d0bfda54e4f4fe1ccc138cc262edfb14cfb4a3
|
||||
Head: 8241bd53b179cf8332541264c3bf12b8e3eca44a
|
||||
Tree: efef640d36f6666f107e188d554c39f05400d4ac
|
||||
Validation path: /var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261007T224638Z-cf1efb/1
|
||||
|
||||
## Finding R1 — subscriber-up decision failures can still report successful submission (P2)
|
||||
|
||||
Evidence: .agents/skills/multi-agent-mux-delegate-job/multi-agent-mux-delegate-job:384-395 (direct), :556-567 (loop/discuss); .agents/skills/lib_py/delegation/wrapper.py:332-333.
|
||||
|
||||
The new failure handling only protects printed answers. An execution failure in the subscriber-up predicate is consumed as a normal false answer. If the subscriber has already exited 0, the error branch maps that code to completed, finalize-begin keeps it, and _exit_for exits 0. This violates N2's requirement that a failed decision call ends the wrapper with 1, never 0.
|
||||
|
||||
I read the exclusion rationale in commit a670ef5: it states that subscriber-up failure treats the subscriber as dead and reaches a fail-closed sub-status/exit-code path. Those later calls can succeed and report completed, so the rationale does not cover a failure confined to the earlier process invocation. This is the same per-decision failure-injection model used by the added regression tests, not an assumption that every later Python invocation must also fail.
|
||||
|
||||
Reproduction (no broker or agent session):
|
||||
1. Use tests/test_issue17_item4_decisions.py::_failing_py with decision="subscriber-up", mode="error" (or "gone").
|
||||
2. Set SUB_STANDIN=1 and SUB_RC=0. The shim makes only job_subscriber.py return 0 and only the targeted predicate fail; other Python calls remain real.
|
||||
3. Call _human_submit with a temporary HOME/workspace/registry and that shim as $PY.
|
||||
4. Expected: submit exits 1 with a decision-failure diagnostic. Actual: module traceback rc 1 -> submit rc 0; missing-interpreter rc 127 -> submit rc 0. The wrapper prints "subscriber died early (... exit=0 ...)" and a log path, then returns success.
|
||||
|
||||
Scripts: /tmp/70d302c2-probes.py and /tmp/70d302c2-predicate-errors.py. All created job records are in temporary probe directories, not the real registry.
|
||||
|
||||
Fix direction: distinguish a successfully computed boolean from inability to execute the decision. A printed boolean through the existing _decided machinery (possibly a new flag/subcommand retaining the old CLI contract) can return execution status 0 for both valid answers while preserving the normal true/false branches in Bash. Treat nonzero execution status or missing/invalid output as _decision_failed. Do not simply fail on every current rc 1, since false legitimately uses that code. At minimum, rc values outside 0/1 must never be silently treated as false, but that alone cannot fix the reproduced import-error rc 1 case. Add tests for both true and false predicates, plus selectively failing subscriber-up with subscriber rc 0, in both direct and loop/discuss call sites.
|
||||
|
||||
The same ambiguity exists at :718 for stall-watch: a probe returning 127 yielded caller rc 0 and continued execution ("survived"), silently disabling the watcher. The creator explicitly describes the watcher as optional/notify-only; this observation is related evidence, while the submission-success counterexample above is sufficient for NOT PASS.
|
||||
|
||||
## Other requirements
|
||||
|
||||
- Restore premise: tests/test_issue45_restore.py now accepts HOME or the original removed directory. The source docstring distinguishes herdr 0.9.1/0.9.3. The additional production handling for restore_error + pane_not_found is explained by commit f19f940: merely relaxing cwd exposed a placeholder with no process, so closing previously raised before reaching other candidates. The matching session, no-live-agent, non-default-server and normal foreground-shell filters remain.
|
||||
- Restore regression negative control: executed the new placeholder unit-test function against the base herdr_restore implementation using a stubbed _herdr; it raises CalledProcessError at base and passes at head.
|
||||
- Printed decisions: the introduced _decided/_exit_for checks cover exit-code, sub-status, finalize-begin, wait-rc and finalize-end process failures. Allowing empty finalize-end output is intentional for OwnerGone, whose CLI handler returns normally without a printed decision. Bash 3.2 dynamic scoping makes printf -v assign the intended caller variables.
|
||||
- Exit-code negative control: the exact base wrapper placed in a temporary dependency layout with the same failure shim exits 255, while head exits 1. This confirms the added failure test actually detects the original bug.
|
||||
- N1 accurately documents the package import path to paho; execution code did not change.
|
||||
- N3 accurately documents PATH python3 and missing-interpreter behavior; executable code did not change.
|
||||
- Six item-scoped commits, seven changed files, no version bump or VERSIONS entry. No further code defect was identified in the reviewed normal-path handling.
|
||||
|
||||
## Verification
|
||||
|
||||
- Python 3.9 from the designated issue17-followups/.venv; native /bin/bash 3.2.57.
|
||||
- With --noconftest, cache disabled, external basetemp and per-test temporary WORKSPACE_ROOT: tests/test_issue17_item4_decisions.py + tests/test_issue45_restore_unit.py -> **281 passed, 4 failed in 292.91s**. All newly added failure-handling and restore unit tests passed. Log: /tmp/70d302c2-selected.log.
|
||||
- The four failures are existing zombie-owner tests: test_u_an_exited_unreaped_owner_is_not_alive, both test_c_finalize_ends_with_an_unreaped_wrapper parameters, and test_c_an_unreaped_wrapper_leaves_the_real_record_alone. A separate owned-child probe confirmed ps execution raises PermissionError(1, Operation not permitted), causing the existing _alive fallback to return True for an unreaped exited child. The wrapper.py executable AST is identical to base (only its module docstring changed). These sandbox-caused failures are not a new finding against this PR.
|
||||
- Budget tests with --noconftest and selection "not test_b3 and not test_r3 and not test_b7": **13 passed, 7 deselected in 28.88s**. Log: /tmp/70d302c2-budget.log. This exercises budget propagation, child wait caps, remote deadline labels and closed-unconfirmed classification without needing a reachable broker.
|
||||
- Explicit loopback bind probe failed with PermissionError(1, Operation not permitted). Consequently no full broker suite result is claimed. Real-herdr restore integration was not run in this sandbox; the orchestrator retains that validation responsibility. Static inspection of its rig confirms disposable HOME/HOME_DIR, mamt45-* session names, scrubbed inherited server selectors and fake agent commands, rather than the user's server/config.
|
||||
- Both shell files pass /bin/bash -n. All five changed Python files pass AST parsing. git diff --check BASE HEAD passes.
|
||||
- HEAD and tree match the brief; final git status --porcelain is empty.
|
||||
- Independent probes reproduced subscriber-up failures returning submit 0 for both module-error rc 1 and interpreter-error rc 127, plus swallowed stall-watch rc 127.
|
||||
- Negative controls confirmed exit-code failure base 255 -> head 1 and placeholder regression base exception -> head pass.
|
||||
- The environment restrictions alone do not determine the verdict. R1 is an independently reproduced N2 requirement violation.
|
||||
|
||||
## Isolation and lifecycle
|
||||
|
||||
All verification ran from the specified review tree. Inherited HERDR_*, MAM_*, MQTT_*, SKS_* and control-plane bindings were cleared; temporary HOME/HOME_DIR/WORKSPACE_ROOT, XDG_CONFIG_HOME and an absent MAM_ENV_FILE were supplied. Tests sourcing lib.sh received a per-test temporary WORKSPACE_ROOT via an environment-only pytest plugin. Direct probes used synthetic jobs and temporary registries. No .mam.env was exported, no live herdr or agent session was used, and no creator/source files, commits or branches were changed. Base comparison artifacts and logs were outside the review tree.
|
||||
|
||||
The requested started publication was attempted first; MQTT exhausted three attempts with Operation not permitted and exited 2. The requested completed command was also executed after saving the report; all three MQTT attempts failed with Operation not permitted and it exited 2. The report exists, but broker delivery of started/completed is not confirmed.
|
||||
|
||||
[VERDICT: NOT PASS]
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"loop_id": "loop-20261007T224638Z-cf1efb",
|
||||
"base_sha": "56d0bfda54e4f4fe1ccc138cc262edfb14cfb4a3",
|
||||
"head_sha": "65b78b6ec44e44904afe34915563522d4631767f",
|
||||
"tree_sha": "b805d9f9e6639c7359115ea9336eac95d7ab85bc",
|
||||
"review_commit": "65b78b6ec44e44904afe34915563522d4631767f",
|
||||
"reviewer": "planner-reviewer-codex-01",
|
||||
"job_id": "fc553fcd",
|
||||
"verdict": "PASS",
|
||||
"transport": "herdr",
|
||||
"evidence": "review-tree",
|
||||
"isolation": "worktree",
|
||||
"verified_at": "2026-10-08T01:16:15Z"
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
# Issue #17 follow-ups, round 2 — Codex review
|
||||
|
||||
Job: fc553fcd
|
||||
Loop: loop-20261007T224638Z-cf1efb
|
||||
Base: 56d0bfda54e4f4fe1ccc138cc262edfb14cfb4a3
|
||||
Previous round: 8241bd53b179cf8332541264c3bf12b8e3eca44a
|
||||
Head: 65b78b6ec44e44904afe34915563522d4631767f
|
||||
Tree: b805d9f9e6639c7359115ea9336eac95d7ab85bc
|
||||
Validation path: /var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261007T224638Z-cf1efb/2
|
||||
|
||||
## Conclusion
|
||||
|
||||
PASS. The prior R1 failure-to-success path is closed. No new blocking defect was found in the cumulative change or the two follow-up commits. This review uses the brief's permitted network-free validation; it does not claim full broker or real-herdr integration execution in this sandbox.
|
||||
|
||||
## Previous finding closure
|
||||
|
||||
| Case | Previous round | Current head | Result |
|
||||
| --- | --- | --- | --- |
|
||||
| subscriber-up execution error, direct submit, subscriber rc 0 | submit exits 0 | submit exits 1 with decision-failure diagnostic | R1 closed |
|
||||
| Same error in loop submit | submit exits 0 | submit exits 1 | R1 closed |
|
||||
| stall-watch execution error | submit exits 0 | submit exits 1 | Related ambiguity closed |
|
||||
| Successful yes/no answers | Truth encoded in process status | --print emits yes/no with process status 0; Bash restores the same branches | Normal semantics preserved |
|
||||
| Invalid successful predicate stdout in if / OR context | No printed-answer validation | Exit 1, no continuation | Guard verified |
|
||||
|
||||
Independent negative controls executed the exact previous-round shell wrapper in a temporary dependency layout and the current wrapper against the same selective module-error shim. All three old/new pairs above were reproduced. The added tests also cover rc 127 and empty successful output. Probe source: /tmp/fc553fcd-probes.py.
|
||||
|
||||
Code evidence:
|
||||
- .agents/skills/lib_py/delegation/wrapper.py:301-309 adds optional --print; :335-353 separates yes/no output from execution status. Without --print, the existing predicate CLI still returns 0/1.
|
||||
- .agents/skills/multi-agent-mux-delegate-job/multi-agent-mux-delegate-job:112-130 checks execution status, nonempty output and the exact yes/no vocabulary. _decision_failed exits the calling shell explicitly, so the failure is not suppressed by Bash's conditional-context errexit rules.
|
||||
- Both subscriber branches now use _decided_yes (:396, :568), as does start_stall_watch (:730). The prior unchecked predicate paths are gone.
|
||||
|
||||
## Cumulative requirements and scope
|
||||
|
||||
The herdr cwd expectation accepts both documented versions; the placeholder handling retains session identity, live-agent and isolated-server filtering. Its unit cases include the valid placeholder, missing restore marker and another process-info error. The implementation and restore tests are unchanged since the prior review, where the new placeholder test was demonstrated to fail on base and pass on head.
|
||||
|
||||
Printed decision failures remain normalized to exit 1. The intentional empty finalize-end result for OwnerGone remains allowed. The prior base-vs-head exit-code negative control (255 -> 1) remains applicable because that helper is unchanged in this round.
|
||||
|
||||
N1's package-import/paho explanation and N3's PATH python3 explanation remain accurate, with no additional runtime changes. The two stale static test literals now match _decided/_exit_for; both corrected contract tests passed.
|
||||
|
||||
The round adds two commits and modifies five files relative to the prior head. AST comparison confirms wrapper.py's existing executable changes since that head are confined to _parser and main, plus the new _yes_no function; timing, owner checks, disk/event handling and finalization are unchanged. The cumulative diff contains nine files, no version bump or VERSIONS entry.
|
||||
|
||||
## Validation
|
||||
|
||||
- tests/test_issue17_item4_decisions.py and tests/test_issue45_restore_unit.py, --noconftest -k "not unreaped": **299 passed, 4 deselected in 262.30s**. Log: /tmp/fc553fcd-selected.log.
|
||||
- tests/test_issue33_budget.py, --noconftest -k "not test_b3 and not test_r3 and not test_b7": **13 passed, 7 deselected in 28.77s**. Log: /tmp/fc553fcd-budget.log.
|
||||
- The two modified static contract tests in test_issue5_phase1_delegation.py and test_tier1_unit.py: **2 passed in 0.16s**, using --noconftest and a minimal temporary mam_sandbox fixture.
|
||||
- Total executed pytest cases: **314 passed**; 11 explicitly deselected for the documented sandbox/integration limitations.
|
||||
- Independent previous/head predicate-failure comparisons and invalid-output probes: all assertions passed.
|
||||
- No product-test failures occurred among the executed cases.
|
||||
|
||||
All tests used the designated issue17-followups/.venv Python 3.9, disabled bytecode/cache writes and external basetemp directories. /bin/bash is the native 3.2 implementation. Both changed shell files pass bash -n; all changed Python files pass AST parsing; git diff --check passes. The review HEAD/tree matched the brief, and the final working tree was clean.
|
||||
|
||||
## Environment limits and isolation
|
||||
|
||||
The inherited sandbox prevents loopback socket binding and ps execution. As diagnosed in the prior round, the four unreaped-owner cases depend on ps; they were explicitly deselected here rather than reported as product failures. Broker-dependent budget cases and real-herdr restore integration remain for the orchestrator, as the brief permits. No claim is made that these omitted integration cases passed in this session.
|
||||
|
||||
Inherited HERDR_*, MAM_*, MQTT_*, SKS_* and control-plane binding variables were removed before test/probe execution. HOME, HOME_DIR, WORKSPACE_ROOT, XDG_CONFIG_HOME and absent MAM_ENV_FILE were redirected to temporary paths. The decisions suite used an environment-only pytest plugin to give every test its own WORKSPACE_ROOT. The two static contract tests used a minimal temporary fixture with a read-only link to the review skills and temporary registry data. No live herdr server, real agent session or .mam.env export was used. No source files, commits, branches, PRs or issue states were modified.
|
||||
|
||||
The requested started command ran first, but MQTT exhausted three attempts with Operation not permitted and exited 2. Completion publication is attempted after saving the report.
|
||||
|
||||
[VERDICT: PASS]
|
||||
Reference in new issue
Block a user