Files
multi-agent-mux/.agents/reports/planner-reviewer-codex-01/report-b880c1f2.md
T
GodopuandClaude Fable 5.1 cf5d1142e9 release(v4.7.0): bump framework and eight skills to 4.7.0 in lockstep; archive PR #29 reviews
Issue #21 items 4, 5, 7 (worktree helpers, review identity, acceptance
A1-A8), #20 test isolation, #27 .gitignore managed block, #28 host
portability. VERSIONS.md carries the entry; the eight review reports of
PR #29 (two rounds plus the codex re-review) are archived under
.agents/reports/<session>/.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
2026-09-20 21:43:25 +09:00

72 lines
11 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# PR #29 independent review
Job: b880c1f2. Reviewer: planner-reviewer-codex-01.
Review target: detached `multi-agent-mux-wt/pr-29` at `46ea4cc1e4f664ca35711c05c3fa71258e169697`, compared with main `1134d74b3884d5818a4f738f5c9441b80f3f8802`.
**NOT PASS.** Two reproduced correctness defects remain in the handling of hidden untracked submodule files. Preparation also fails the brief's explicit environment requirement. These are separate from the known pending 4.7.0 release bump. No broad redesign is necessary: the guard corrections are narrow and were verified in disposable fixtures.
## Blocking findings
### R1 — P1: removal can silently delete untracked submodule data
Locations: `.agents/skills/lib_py/worktree.py:167` and `.agents/skills/multi-agent-mux-stop/scripts/worktree_remove.sh:91` (the resulting force-removal path is at line 161).
Both cleanliness checks inherit Git's `status.showUntrackedFiles` configuration. `--porcelain` does not override that setting. In an initialized submodule with `git config status.showUntrackedFiles no`, a normal, non-ignored untracked file is hidden from both the parent status and the submodule `foreach` check. The helper consequently reports that all checks passed and executes `git worktree remove --force`, deleting the file.
I reproduced this using the existing `ctl_sub` sandbox fixture and the actual preparation/removal scripts, with no real worktree removal:
1. Prepare a temporary worktree with its local test submodule initialized.
2. Set `status.showUntrackedFiles=no` in that submodule and create `valuable-untracked.txt` there.
3. Parent `status --porcelain --ignore-submodules=none` returns an empty string; explicit submodule `status --porcelain --untracked-files=all` reports the file.
4. Removal returns **0**, prints `checks: locked/sessions/jobs/clean/submodules -> passed; removal method: single --force`, and the file no longer exists.
**Correction:** explicitly use `--untracked-files=all` in the parent and recursive submodule safety checks, retaining `--ignore-submodules=none`; do not let display preferences govern deletion eligibility. Add regressions with this configuration both before removal and for the post-stop recheck. I changed only the disposable fixture's copy of the submodule check to `git status --porcelain --untracked-files=all --ignore-submodules=none`: removal then returned **3** and preserved the file. The reviewed source remains unchanged.
### R2 — P1: the review snapshot can approve a dirty submodule it did not capture
Location: `.agents/skills/lib_py/loop/runner.py:717` (`_dirty_submodules`).
The review-identity guard has the same configuration dependency. With `status.showUntrackedFiles=no` inside a submodule, untracked content is not rejected. The snapshot records only the submodule gitlink, so that content is absent from the review tree and from the identity comparison. A stubbed PASS is then accepted and promoted, although the documented dirty-submodule refusal should have prevented any review.
Using the existing temporary `wt_repo` and `_drive` fixtures, I added a clean submodule, set this configuration, and created a non-ignored untracked file. The actual controller returned **0**, dispatched **one review**, and wrote **one PASS identity sidecar**. This demonstrates the controller admission/promotion defect; it does not represent a real reviewer's assessment.
**Correction:** add `--untracked-files=all` to `_dirty_submodules` so both snapshot admission and subsequent stale/pre-promotion checks reject these files. An in-memory override of only that status command made the same probe return **1**, dispatch zero reviews, and write no sidecar. Add configuration-sensitive tests for both initial and mid-round untracked changes.
### R3 — P2: preparation still aliases the control environment
Location: `.agents/skills/multi-agent-mux-create/scripts/worktree_prepare.sh:111`–118; related assertion: `tests/test_issue21_phase2_loop_a.py:264`.
The brief explicitly asks for a real worktree venv rather than a symlink, or explicit interpreter-path handling. The helper instead executes `ln -s "$MAIN/.venv" "$WT/.venv"`; its test requires that link, and the design/skill documents describe it. This is an implementation-and-test requirement gap, not a failing existing test. A temporary fixture confirmed that writing through the worktree `.venv` changes the control root's environment. The PR review checkout's own real venv does not establish what this helper creates.
**Correction:** create an independent environment with the intended dependencies, or use the brief's explicit interpreter-path alternative without creating a misleading local environment alias. Update the W1 assertion and corresponding preparation documentation. The loop's review prompt already provides an absolute interpreter path, but that does not change preparation behavior for creators.
## Other review results
- **R1–R6 removal defenses:** the post-stop tree recheck, stash checks, retained/deinitialized submodule repository checks, partial-repository refusals, and traversal of HEAD plus local refs including tags are present. The existing regressions cover those cases and passed. They do not exercise `status.showUntrackedFiles=no`, which explains the additional R1 failure above. Initialization rollback retains preexisting submodule repositories/configuration in the tested cases; initialization over a nonempty uninitialized path is refused.
- **Resume and bindings:** the original-cwd check uses canonical paths, runs before session resolution, rejects missing/mismatching recorded cwd with exit 2, and leaves rows without cwd compatible. Single-workspace resume at its recorded path and already-running binding persistence passed. Create now starts immediate reconcile/watchdog from the control runtime, and preparation supplies `MAM_NO_SUBSCRIBER_DAEMON=1`.
- **Identity workflow:** snapshots use a temporary index, include ordinary untracked source files, pin the diff to the snapshot, and generate detached review worktrees. Stale rounds discard approvals; promotion rechecks identity. Failure to create a review worktree does not silently lower isolation. Remote verdict recipients are refused before delegation unless explicitly admitted as diff-only; truncated/unavailable evidence is refused for those recipients. Self-review uses the identity path. These behaviors passed focused tests, subject to R2. With no worktree/base options, existing delegation prompts and the two-argument diff collector remain compatible; output is not literally byte-identical everywhere because loop-id logging/checkpoint data are new.
- **#20 isolation:** ambient bindings are recorded/dropped before collection, and function-scoped defaults route registry/audit state to tmp_path. The #15 default-resolution tests explicitly remove overrides where necessary and check their own job/session identities. The guard watches recorded control-plane paths, not only the review checkout. Direct directory snapshots found **zero added entries** in main and review checkout `.mam/jobs` and `.mam/delegate_job_logs` during this review's tests.
- **Acceptance A1–A8:** these are sandbox/mock integration tests, not proof of environment delivery inside a real herdr pane. The documentation now states that boundary. The tests exercise caller arguments, routing, stale review rejection, cleanup, and loop locking; all passed in the focused run. A3 uses a deliberately unreachable broker, so failure timing is bounded but the suite is not uniformly instantaneous. Real-herdr acceptance remains separate evidence to obtain before claiming complete operational validation.
- **acceptance5 changes:** unique session/secret values remove the old fixed-session collision, retained-terminal delivery is a valid acknowledgement path, and product `started --heartbeat` behavior is unchanged. The test passed in this run. Caution: the new skip predicate at `tests/test_issue18_liveness_contract.py:1183` checks local subscription plus absence of observed events; it does not independently establish the topic actually published by the runner. Its claim that a wrong publish topic must still fail is stronger than its evidence. A controlled broker or assertions against publisher topic/outcome evidence would make this classification more reliable. Do not treat a skipped acceptance run as a PASS.
- **Documentation/follow-ups:** the brief's four nonblocking follow-ups are reasonable: strengthen A4 beyond status/path equality to compare untracked contents; make shared info/exclude mutation and its pre-seeding in A4 explicit; scope fallback prune to owned review metadata; deduplicate submodule refusal output. A4 currently proves only its stated tuple invariants. Tracking these under the proposed #30 follow-up is appropriate, but I did not independently inspect the external issue's registration/content. R1/R2 above must be fixed before merge rather than deferred as those polish items.
- **Release:** 4.7.0 lockstep changes and release notes remain a known merge prerequisite. Version-consistency tests passing on 4.6.0 do not fulfill it. `git diff --check 1134d74...HEAD` passed; `bash -n` passed for all eight changed shell files.
## Verification record
All pytest runs used the requested worktree's own `.venv/bin/python`, `PYTHONDONTWRITEBYTECODE=1`, and `-p no:cacheprovider`. Additional probes were injected in memory through pytest collection and used existing temporary fixtures; only the fix-direction probe altered a fixture's disposable source copy.
| Run | Result |
|---|---|
| `test_issue21_acceptance.py`, `test_issue21_phase2_loop_a.py`, `test_issue21_loop_b.py`, `test_issue21_worktree_prereqs.py`, `test_issue20_state_isolation.py`, `test_issue15_registry_isolation.py`, `test_version_consistency.py` | **118 passed, 1 skipped**, 289.64 s. Skip is the outer collection of the deliberately nested S6 probe. |
| `test_issue14_delivery_ack.py`, `test_issue18_liveness_contract.py`, `test_issue6_turn_handshake.py` | **56 passed, 1 failed**, 331.60 s. |
| Isolated wall-clock regression rerun | **1 failed**, 31.24 s; close delay 28.5585 s against 28 s. |
| R1/R2 safety assertions against unmodified implementation | Both failed as described above, reproducing the blockers. |
| R1 disposable-copy correction / R2 in-memory correction | Each passed: removal preserved the file; identity refused before review. |
| Preparation alias probe | Confirmed the symlink and cross-environment write. |
The regression-suite failure was `test_wrapper_idle_grace_never_outlives_the_wall_clock`, `tests/test_issue14_delivery_ack.py:513`: the expected error status and return code were observed, but close delay was **28.7779 s**, exceeding the **28 s** allowance. The isolated rerun also failed at **28.5585 s**. The test body and wrapper are unchanged by this PR; the changed #14 code is its separate in-process publication helper. This timing failure remains unresolved in this environment and is not the evidence for R1/R2. I did not run the base suite or independently classify #28's 33 failures; these results do not justify a blanket environmental explanation.
No product files, source refs, production sessions, or monitor state were changed. Main remained on `1134d74`; the reviewed worktree remained on `46ea4cc`. The verdict concerns this exact tree, not later orchestrator release/fix commits.
[VERDICT: NOT PASS]