PR #48 already carries the 4.8.5 lockstep bump. Merge-workspace review: round 1 (@35935f9) claude agy grok NOT PASS (test portability: tomllib on Python 3.9, /proc and /tmp on macOS), codex no verdict (model at capacity, budget spent); fixed in36f0b8dande8d9151. Round 2 (@e8d9151) codex NOT PASS (unrunnable config check passed an unchecked merge, odd pids closed a pane, nan dropped the user's settings); fixed inab93776. Round 3 (@ab93776) 4/4 PASS. Full suite atab93776on macOS / Python 3.9: 1456 passed / 0 failed / 38 skipped. Reports and the round-3 identity sidecars go to .agents/reports/<session>/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
108 lines
7.5 KiB
Markdown
108 lines
7.5 KiB
Markdown
# Final Review Report: Job f3699a96 (PR #48 / Issue #45)
|
|
|
|
## 1. Review Identity & Metadata
|
|
- **Loop ID**: `loop-20261005T064728Z-a4e9c8`
|
|
- **Job ID**: `f3699a96`
|
|
- **Reviewer**: agy (session: `herdr:reviewer-agy-01`)
|
|
- **Base Commit**: `c5c7eff6f758c76ac37a138dbc3c301f3f7e10cf` (main, v4.8.4)
|
|
- **Head Commit**: `35935f9e8a052ca9e9d9b806a62d17eb72afea78` (10 commits)
|
|
- **Tree Hash**: `dff9db13f3e018528d04a024acda942f501513ff`
|
|
- **Verification Path**: `/var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1`
|
|
- **Git Status**: Clean (working tree clean, no uncommitted changes)
|
|
- **Whitespace Check (`git diff base...HEAD --check`)**: Clean (0 errors)
|
|
|
|
---
|
|
|
|
## 2. Review Findings & Reasons for NOT PASS
|
|
|
|
During verification with the specified test command (`AGENT_PYTHON_BIN=... python -m pytest ...`), multiple critical test failures occurred on macOS / Python < 3.11:
|
|
|
|
### 1) Unconditional `import tomllib` at Module Level in Test Files
|
|
- **Failure 1**: [`tests/test_issue45_herdr_config.py:16`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/tests/test_issue45_herdr_config.py#L16)
|
|
```python
|
|
import tomllib
|
|
```
|
|
- **Error**: `ModuleNotFoundError: No module named 'tomllib'`
|
|
- **Impact**: Pytest fails collection immediately with exit code 2. The entire test module cannot be run on Python < 3.11 (such as Python 3.9 in the project virtualenv).
|
|
- **Context**: While [`lib_py/herdr_config.py:152-155`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/.agents/skills/lib_py/herdr_config.py#L152-L155) guards the import with `try ... except ImportError: tomllib = None`, the test module itself imports `tomllib` unconditionally at line 16 and in test functions.
|
|
|
|
- **Failure 2**: [`tests/test_issue9_server_env.py:1592`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/tests/test_issue9_server_env.py#L1592)
|
|
```python
|
|
def test_bootstrap_hands_the_server_a_no_restore_config(mam_sandbox):
|
|
import tomllib
|
|
```
|
|
- **Error**: `ModuleNotFoundError: No module named 'tomllib'`
|
|
- **Impact**: Test fails immediately on Python < 3.11.
|
|
|
|
---
|
|
|
|
### 2) Non-Portable Linux Assumptions in Real Herdr Server Test (`test_issue45_restore.py`)
|
|
Four out of 5 tests in [`tests/test_issue45_restore.py`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/tests/test_issue45_restore.py) fail on macOS:
|
|
|
|
- **Failure 3 (Linux `/proc` filesystem hardcoding)**: [`tests/test_issue45_restore.py:227`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/tests/test_issue45_restore.py#L227)
|
|
```python
|
|
environ = Path(f"/proc/{rig.server_pids()[0]}/environ").read_bytes().split(b"\0")
|
|
```
|
|
- **Error**: `FileNotFoundError: [Errno 2] No such file or directory: '/proc/26854/environ'`
|
|
- **Impact**: Tests `test_a_mam_server_does_not_relaunch_saved_agents[plain-user-config]` and `test_a_mam_server_does_not_relaunch_saved_agents[user-config-with-session-table]` both crash. macOS (Darwin) does not have a `/proc` filesystem. Reading `/proc/<pid>/environ` is non-portable.
|
|
|
|
- **Failure 4 (macOS `/tmp` vs `/private/tmp` symlink mismatch)**: [`tests/test_issue45_restore.py:263`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/tests/test_issue45_restore.py#L263)
|
|
```python
|
|
assert _pane_with(rig, ids["Y"])["cwd"] == str(rig.root)
|
|
```
|
|
- **Error**: `AssertionError: assert '/private/tmp/m45phdbspj1' == '/tmp/m45phdbspj1'`
|
|
- **Impact**: `test_b_only_the_resumed_sessions_restored_shell_is_closed` fails. On macOS, `/tmp` is a symlink to `/private/tmp`. Herdr canonicalizes working directories to `/private/tmp/...`. Exact string comparison against uncanonicalized `str(rig.root)` fails. Comparing with `Path(cwd).resolve()` or `os.path.samefile` is required.
|
|
|
|
- **Failure 5 (Pane lookup failure due to uncanonicalized `/tmp` path)**: [`tests/test_issue45_restore.py:325`](file:///var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1/tests/test_issue45_restore.py#L325)
|
|
```python
|
|
pane = next(p for p in rig.panes() if p.get("cwd") == str(ws))
|
|
```
|
|
- **Error**: `StopIteration`
|
|
- **Impact**: `test_b_resume_leaves_no_restored_shell_behind` fails. `ws` is constructed under `/tmp/...`, while `p.get("cwd")` is reported as `/private/tmp/...`. The generator finds no match and raises `StopIteration`.
|
|
|
|
---
|
|
|
|
## 3. Evaluation of Brief Review Questions
|
|
|
|
1. **A's config merging**:
|
|
- On Python 3.11+, parsing and rewriting logic in `lib_py/herdr_config.py` preserves tables, dotted keys, and verifies with `tomllib.loads(text) == expected` followed by `herdr config check`.
|
|
- On Python < 3.11 (e.g. system Python 3.9), `tomllib` is absent, causing `server_config_text()` to emit a warning and fall back to `MINIMAL` (`[session]\nresume_agents_on_restore = false`), thereby losing user-defined herdr settings on Python 3.9 systems.
|
|
2. **Coverage across server boot paths**:
|
|
- `lib.sh:230-236` runs `python3 -m lib_py.herdr_config server-config` during headless bootstrap.
|
|
- Accurately documented that manually attached/started servers outside MAM do not pick up `mam-server-config.toml`.
|
|
3. **B's pane closing conditions**:
|
|
- Clean fail-safe in `lib_py/herdr_restore.py`: checks `HERDR_SESSION_NAME` is not default, ensures no live agent is running in the pane, checks matching `agent_session`, and verifies foreground process group is the shell. If herdr cannot be queried, it fails-safe by leaving panes untouched and emitting a warning in `resume_session.sh:614`.
|
|
4. **Test isolation & portability**:
|
|
- Test suites properly use temporary HOME and isolated session names, preventing mutation of live user servers or configs.
|
|
- However, portability on macOS is broken due to hardcoded `/proc` access and uncanonicalized `/tmp` symlink path comparisons.
|
|
5. **Version and Inventory**:
|
|
- Framework and 8 skills were updated to `4.8.5`.
|
|
- `tests/data/skill_code_blocks.json` updated line ranges for `create` and `resume` while preserving classification.
|
|
6. **PR purge failure note**:
|
|
- Baseline purge test passes 23/23 on macOS and is unrelated.
|
|
7. **Boundary (#17)**:
|
|
- Complex logic is implemented in Python (`herdr_config.py`, `herdr_restore.py`), while shell scripts delegate to them.
|
|
|
|
---
|
|
|
|
## 4. Test Suite Execution Summary
|
|
|
|
Executed in verification worktree (`/var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T064728Z-a4e9c8/1`):
|
|
|
|
| Test Suite | Result | Details |
|
|
|---|---|---|
|
|
| `tests/test_issue45_herdr_config.py` | **FAILED** | Collection error: `ModuleNotFoundError: No module named 'tomllib'` at line 16 |
|
|
| `tests/test_issue9_server_env.py` | **FAILED** | `test_bootstrap_hands_the_server_a_no_restore_config` failed at line 1592 (`ModuleNotFoundError`) |
|
|
| `tests/test_issue45_restore.py` | **FAILED** (4/5 failed) | Lines 227 (`/proc` FileNotFoundError), 263 (`/tmp` vs `/private/tmp` AssertionError), 325 (`StopIteration`) |
|
|
| `tests/test_issue23_24_startup_path.py` | 27 passed | Clean |
|
|
| `tests/test_issue42_skill_blocks.py` | 37 passed, 1 skipped | Clean |
|
|
| `tests/test_version_consistency.py` | 3 passed | Clean |
|
|
|
|
---
|
|
|
|
## 5. Conclusion
|
|
|
|
Due to the critical test failures on macOS and Python < 3.11 (`ModuleNotFoundError: tomllib` in test suites, Linux `/proc` hardcoding, and macOS `/private/tmp` path matching failures in `test_issue45_restore.py`), the review verdict is NOT PASS.
|
|
|
|
[VERDICT: NOT PASS]
|