Files
multi-agent-mux/.agents/reports/reviewer-agy-01/report-f3699a96.md
T
GodopuandClaude Opus 5.5 bf8a583afb docs(v4.8.5): archive PR #48 reviews (three rounds) and record the merge-workspace review
PR #48 already carries the 4.8.5 lockstep bump. Merge-workspace review:
round 1 (@35935f9) claude agy grok NOT PASS (test portability: tomllib on
Python 3.9, /proc and /tmp on macOS), codex no verdict (model at capacity,
budget spent); fixed in 36f0b8d and e8d9151. Round 2 (@e8d9151) codex NOT
PASS (unrunnable config check passed an unchecked merge, odd pids closed a
pane, nan dropped the user's settings); fixed in ab93776. Round 3
(@ab93776) 4/4 PASS. Full suite at ab93776 on macOS / Python 3.9: 1456
passed / 0 failed / 38 skipped. Reports and the round-3 identity sidecars
go to .agents/reports/<session>/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
2026-10-05 18:56:38 +09:00

108 lines
7.5 KiB
Markdown

# Final Review Report: Job f3699a96 (PR #48 / Issue #45)
## 1. Review Identity & Metadata
- **Loop ID**: `loop-20261005T064728Z-a4e9c8`
- **Job ID**: `f3699a96`
- **Reviewer**: agy (session: `herdr:reviewer-agy-01`)
- **Base Commit**: `c5c7eff6f758c76ac37a138dbc3c301f3f7e10cf` (main, v4.8.4)
- **Head Commit**: `35935f9e8a052ca9e9d9b806a62d17eb72afea78` (10 commits)
- **Tree Hash**: `dff9db13f3e018528d04a024acda942f501513ff`
- **Verification Path**: `/var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1`
- **Git Status**: Clean (working tree clean, no uncommitted changes)
- **Whitespace Check (`git diff base...HEAD --check`)**: Clean (0 errors)
---
## 2. Review Findings & Reasons for NOT PASS
During verification with the specified test command (`AGENT_PYTHON_BIN=... python -m pytest ...`), multiple critical test failures occurred on macOS / Python < 3.11:
### 1) Unconditional `import tomllib` at Module Level in Test Files
- **Failure 1**: [`tests/test_issue45_herdr_config.py:16`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/tests/test_issue45_herdr_config.py#L16)
```python
import tomllib
```
- **Error**: `ModuleNotFoundError: No module named 'tomllib'`
- **Impact**: Pytest fails collection immediately with exit code 2. The entire test module cannot be run on Python < 3.11 (such as Python 3.9 in the project virtualenv).
- **Context**: While [`lib_py/herdr_config.py:152-155`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/.agents/skills/lib_py/herdr_config.py#L152-L155) guards the import with `try ... except ImportError: tomllib = None`, the test module itself imports `tomllib` unconditionally at line 16 and in test functions.
- **Failure 2**: [`tests/test_issue9_server_env.py:1592`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/tests/test_issue9_server_env.py#L1592)
```python
def test_bootstrap_hands_the_server_a_no_restore_config(mam_sandbox):
import tomllib
```
- **Error**: `ModuleNotFoundError: No module named 'tomllib'`
- **Impact**: Test fails immediately on Python < 3.11.
---
### 2) Non-Portable Linux Assumptions in Real Herdr Server Test (`test_issue45_restore.py`)
Four out of 5 tests in [`tests/test_issue45_restore.py`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/tests/test_issue45_restore.py) fail on macOS:
- **Failure 3 (Linux `/proc` filesystem hardcoding)**: [`tests/test_issue45_restore.py:227`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/tests/test_issue45_restore.py#L227)
```python
environ = Path(f"/proc/{rig.server_pids()[0]}/environ").read_bytes().split(b"\0")
```
- **Error**: `FileNotFoundError: [Errno 2] No such file or directory: '/proc/26854/environ'`
- **Impact**: Tests `test_a_mam_server_does_not_relaunch_saved_agents[plain-user-config]` and `test_a_mam_server_does_not_relaunch_saved_agents[user-config-with-session-table]` both crash. macOS (Darwin) does not have a `/proc` filesystem. Reading `/proc/<pid>/environ` is non-portable.
- **Failure 4 (macOS `/tmp` vs `/private/tmp` symlink mismatch)**: [`tests/test_issue45_restore.py:263`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/tests/test_issue45_restore.py#L263)
```python
assert _pane_with(rig, ids["Y"])["cwd"] == str(rig.root)
```
- **Error**: `AssertionError: assert '/private/tmp/m45phdbspj1' == '/tmp/m45phdbspj1'`
- **Impact**: `test_b_only_the_resumed_sessions_restored_shell_is_closed` fails. On macOS, `/tmp` is a symlink to `/private/tmp`. Herdr canonicalizes working directories to `/private/tmp/...`. Exact string comparison against uncanonicalized `str(rig.root)` fails. Comparing with `Path(cwd).resolve()` or `os.path.samefile` is required.
- **Failure 5 (Pane lookup failure due to uncanonicalized `/tmp` path)**: [`tests/test_issue45_restore.py:325`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/tests/test_issue45_restore.py#L325)
```python
pane = next(p for p in rig.panes() if p.get("cwd") == str(ws))
```
- **Error**: `StopIteration`
- **Impact**: `test_b_resume_leaves_no_restored_shell_behind` fails. `ws` is constructed under `/tmp/...`, while `p.get("cwd")` is reported as `/private/tmp/...`. The generator finds no match and raises `StopIteration`.
---
## 3. Evaluation of Brief Review Questions
1. **A's config merging**:
- On Python 3.11+, parsing and rewriting logic in `lib_py/herdr_config.py` preserves tables, dotted keys, and verifies with `tomllib.loads(text) == expected` followed by `herdr config check`.
- On Python < 3.11 (e.g. system Python 3.9), `tomllib` is absent, causing `server_config_text()` to emit a warning and fall back to `MINIMAL` (`[session]\nresume_agents_on_restore = false`), thereby losing user-defined herdr settings on Python 3.9 systems.
2. **Coverage across server boot paths**:
- `lib.sh:230-236` runs `python3 -m lib_py.herdr_config server-config` during headless bootstrap.
- Accurately documented that manually attached/started servers outside MAM do not pick up `mam-server-config.toml`.
3. **B's pane closing conditions**:
- Clean fail-safe in `lib_py/herdr_restore.py`: checks `HERDR_SESSION_NAME` is not default, ensures no live agent is running in the pane, checks matching `agent_session`, and verifies foreground process group is the shell. If herdr cannot be queried, it fails-safe by leaving panes untouched and emitting a warning in `resume_session.sh:614`.
4. **Test isolation & portability**:
- Test suites properly use temporary HOME and isolated session names, preventing mutation of live user servers or configs.
- However, portability on macOS is broken due to hardcoded `/proc` access and uncanonicalized `/tmp` symlink path comparisons.
5. **Version and Inventory**:
- Framework and 8 skills were updated to `4.8.5`.
- `tests/data/skill_code_blocks.json` updated line ranges for `create` and `resume` while preserving classification.
6. **PR purge failure note**:
- Baseline purge test passes 23/23 on macOS and is unrelated.
7. **Boundary (#17)**:
- Complex logic is implemented in Python (`herdr_config.py`, `herdr_restore.py`), while shell scripts delegate to them.
---
## 4. Test Suite Execution Summary
Executed in verification worktree (`/var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1`):
| Test Suite | Result | Details |
|---|---|---|
| `tests/test_issue45_herdr_config.py` | **FAILED** | Collection error: `ModuleNotFoundError: No module named 'tomllib'` at line 16 |
| `tests/test_issue9_server_env.py` | **FAILED** | `test_bootstrap_hands_the_server_a_no_restore_config` failed at line 1592 (`ModuleNotFoundError`) |
| `tests/test_issue45_restore.py` | **FAILED** (4/5 failed) | Lines 227 (`/proc` FileNotFoundError), 263 (`/tmp` vs `/private/tmp` AssertionError), 325 (`StopIteration`) |
| `tests/test_issue23_24_startup_path.py` | 27 passed | Clean |
| `tests/test_issue42_skill_blocks.py` | 37 passed, 1 skipped | Clean |
| `tests/test_version_consistency.py` | 3 passed | Clean |
---
## 5. Conclusion
Due to the critical test failures on macOS and Python < 3.11 (`ModuleNotFoundError: tomllib` in test suites, Linux `/proc` hardcoding, and macOS `/private/tmp` path matching failures in `test_issue45_restore.py`), the review verdict is NOT PASS.
[VERDICT: NOT PASS]