release(v4.8.7): Issue #17 follow-ups - archive the review reports and bump the lockstep version
Merge-workspace review loop (--all-reviewer): round 1 (@8241bd5) claude grok PASS, agy NOT PASS (two static wrapper-literal tests did not follow _decided/_exit_for), codex NOT PASS (R1: a yes/no decision that could not run read as "no"); fixed in541c3feand65b78b6. Round 2 (@65b78b6) 4/4 PASS. Full suite at65b78b6on macOS / Python 3.9 / herdr 0.9.3: 1730 passed / 0 failed / 56 skipped; 18 of the skips were the uninitialized nats-docker submodule in the fresh worktree, and with it tests/test_deploy_freshness.py is 31 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
This commit is contained in:
1 parent
65b78b6ec4
commit
cd82ab6e72
28 files changed
+594
-25
No files matched your search
@@ -0,0 +1,59 @@
|
||||
# Issue #17 follow-ups — Codex review
|
||||
|
||||
Job: 70d302c2
|
||||
Loop: loop-20261007T224638Z-cf1efb
|
||||
Base: 56d0bfda54e4f4fe1ccc138cc262edfb14cfb4a3
|
||||
Head: 8241bd53b179cf8332541264c3bf12b8e3eca44a
|
||||
Tree: efef640d36f6666f107e188d554c39f05400d4ac
|
||||
Validation path: /var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261007T224638Z-cf1efb/1
|
||||
|
||||
## Finding R1 — subscriber-up decision failures can still report successful submission (P2)
|
||||
|
||||
Evidence: .agents/skills/multi-agent-mux-delegate-job/multi-agent-mux-delegate-job:384-395 (direct), :556-567 (loop/discuss); .agents/skills/lib_py/delegation/wrapper.py:332-333.
|
||||
|
||||
The new failure handling only protects printed answers. An execution failure in the subscriber-up predicate is consumed as a normal false answer. If the subscriber has already exited 0, the error branch maps that code to completed, finalize-begin keeps it, and _exit_for exits 0. This violates N2's requirement that a failed decision call ends the wrapper with 1, never 0.
|
||||
|
||||
I read the exclusion rationale in commit a670ef5: it states that subscriber-up failure treats the subscriber as dead and reaches a fail-closed sub-status/exit-code path. Those later calls can succeed and report completed, so the rationale does not cover a failure confined to the earlier process invocation. This is the same per-decision failure-injection model used by the added regression tests, not an assumption that every later Python invocation must also fail.
|
||||
|
||||
Reproduction (no broker or agent session):
|
||||
1. Use tests/test_issue17_item4_decisions.py::_failing_py with decision="subscriber-up", mode="error" (or "gone").
|
||||
2. Set SUB_STANDIN=1 and SUB_RC=0. The shim makes only job_subscriber.py return 0 and only the targeted predicate fail; other Python calls remain real.
|
||||
3. Call _human_submit with a temporary HOME/workspace/registry and that shim as $PY.
|
||||
4. Expected: submit exits 1 with a decision-failure diagnostic. Actual: module traceback rc 1 -> submit rc 0; missing-interpreter rc 127 -> submit rc 0. The wrapper prints "subscriber died early (... exit=0 ...)" and a log path, then returns success.
|
||||
|
||||
Scripts: /tmp/70d302c2-probes.py and /tmp/70d302c2-predicate-errors.py. All created job records are in temporary probe directories, not the real registry.
|
||||
|
||||
Fix direction: distinguish a successfully computed boolean from inability to execute the decision. A printed boolean through the existing _decided machinery (possibly a new flag/subcommand retaining the old CLI contract) can return execution status 0 for both valid answers while preserving the normal true/false branches in Bash. Treat nonzero execution status or missing/invalid output as _decision_failed. Do not simply fail on every current rc 1, since false legitimately uses that code. At minimum, rc values outside 0/1 must never be silently treated as false, but that alone cannot fix the reproduced import-error rc 1 case. Add tests for both true and false predicates, plus selectively failing subscriber-up with subscriber rc 0, in both direct and loop/discuss call sites.
|
||||
|
||||
The same ambiguity exists at :718 for stall-watch: a probe returning 127 yielded caller rc 0 and continued execution ("survived"), silently disabling the watcher. The creator explicitly describes the watcher as optional/notify-only; this observation is related evidence, while the submission-success counterexample above is sufficient for NOT PASS.
|
||||
|
||||
## Other requirements
|
||||
|
||||
- Restore premise: tests/test_issue45_restore.py now accepts HOME or the original removed directory. The source docstring distinguishes herdr 0.9.1/0.9.3. The additional production handling for restore_error + pane_not_found is explained by commit f19f940: merely relaxing cwd exposed a placeholder with no process, so closing previously raised before reaching other candidates. The matching session, no-live-agent, non-default-server and normal foreground-shell filters remain.
|
||||
- Restore regression negative control: executed the new placeholder unit-test function against the base herdr_restore implementation using a stubbed _herdr; it raises CalledProcessError at base and passes at head.
|
||||
- Printed decisions: the introduced _decided/_exit_for checks cover exit-code, sub-status, finalize-begin, wait-rc and finalize-end process failures. Allowing empty finalize-end output is intentional for OwnerGone, whose CLI handler returns normally without a printed decision. Bash 3.2 dynamic scoping makes printf -v assign the intended caller variables.
|
||||
- Exit-code negative control: the exact base wrapper placed in a temporary dependency layout with the same failure shim exits 255, while head exits 1. This confirms the added failure test actually detects the original bug.
|
||||
- N1 accurately documents the package import path to paho; execution code did not change.
|
||||
- N3 accurately documents PATH python3 and missing-interpreter behavior; executable code did not change.
|
||||
- Six item-scoped commits, seven changed files, no version bump or VERSIONS entry. No further code defect was identified in the reviewed normal-path handling.
|
||||
|
||||
## Verification
|
||||
|
||||
- Python 3.9 from the designated issue17-followups/.venv; native /bin/bash 3.2.57.
|
||||
- With --noconftest, cache disabled, external basetemp and per-test temporary WORKSPACE_ROOT: tests/test_issue17_item4_decisions.py + tests/test_issue45_restore_unit.py -> **281 passed, 4 failed in 292.91s**. All newly added failure-handling and restore unit tests passed. Log: /tmp/70d302c2-selected.log.
|
||||
- The four failures are existing zombie-owner tests: test_u_an_exited_unreaped_owner_is_not_alive, both test_c_finalize_ends_with_an_unreaped_wrapper parameters, and test_c_an_unreaped_wrapper_leaves_the_real_record_alone. A separate owned-child probe confirmed ps execution raises PermissionError(1, Operation not permitted), causing the existing _alive fallback to return True for an unreaped exited child. The wrapper.py executable AST is identical to base (only its module docstring changed). These sandbox-caused failures are not a new finding against this PR.
|
||||
- Budget tests with --noconftest and selection "not test_b3 and not test_r3 and not test_b7": **13 passed, 7 deselected in 28.88s**. Log: /tmp/70d302c2-budget.log. This exercises budget propagation, child wait caps, remote deadline labels and closed-unconfirmed classification without needing a reachable broker.
|
||||
- Explicit loopback bind probe failed with PermissionError(1, Operation not permitted). Consequently no full broker suite result is claimed. Real-herdr restore integration was not run in this sandbox; the orchestrator retains that validation responsibility. Static inspection of its rig confirms disposable HOME/HOME_DIR, mamt45-* session names, scrubbed inherited server selectors and fake agent commands, rather than the user's server/config.
|
||||
- Both shell files pass /bin/bash -n. All five changed Python files pass AST parsing. git diff --check BASE HEAD passes.
|
||||
- HEAD and tree match the brief; final git status --porcelain is empty.
|
||||
- Independent probes reproduced subscriber-up failures returning submit 0 for both module-error rc 1 and interpreter-error rc 127, plus swallowed stall-watch rc 127.
|
||||
- Negative controls confirmed exit-code failure base 255 -> head 1 and placeholder regression base exception -> head pass.
|
||||
- The environment restrictions alone do not determine the verdict. R1 is an independently reproduced N2 requirement violation.
|
||||
|
||||
## Isolation and lifecycle
|
||||
|
||||
All verification ran from the specified review tree. Inherited HERDR_*, MAM_*, MQTT_*, SKS_* and control-plane bindings were cleared; temporary HOME/HOME_DIR/WORKSPACE_ROOT, XDG_CONFIG_HOME and an absent MAM_ENV_FILE were supplied. Tests sourcing lib.sh received a per-test temporary WORKSPACE_ROOT via an environment-only pytest plugin. Direct probes used synthetic jobs and temporary registries. No .mam.env was exported, no live herdr or agent session was used, and no creator/source files, commits or branches were changed. Base comparison artifacts and logs were outside the review tree.
|
||||
|
||||
The requested started publication was attempted first; MQTT exhausted three attempts with Operation not permitted and exited 2. The requested completed command was also executed after saving the report; all three MQTT attempts failed with Operation not permitted and it exited 2. The report exists, but broker delivery of started/completed is not confirmed.
|
||||
|
||||
[VERDICT: NOT PASS]
|
||||
Reference in new issue
Block a user