PR #48 already carries the 4.8.5 lockstep bump. Merge-workspace review: round 1 (@35935f9) claude agy grok NOT PASS (test portability: tomllib on Python 3.9, /proc and /tmp on macOS), codex no verdict (model at capacity, budget spent); fixed in36f0b8dande8d9151. Round 2 (@e8d9151) codex NOT PASS (unrunnable config check passed an unchecked merge, odd pids closed a pane, nan dropped the user's settings); fixed inab93776. Round 3 (@ab93776) 4/4 PASS. Full suite atab93776on macOS / Python 3.9: 1456 passed / 0 failed / 38 skipped. Reports and the round-3 identity sidecars go to .agents/reports/<session>/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
116 lines
7.7 KiB
Markdown
116 lines
7.7 KiB
Markdown
# PR #48 (fix/issue-45, Closes #45) Round 3 Delta Review Report
|
|
|
|
- **Reviewer Agent**: `agy` (session: `herdr:reviewer-agy-01`)
|
|
- **Job ID**: `3e6f0467`
|
|
- **Verification Directory**: `/var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T085016Z-c0f6eb/1`
|
|
- **Base Commit**: `e8d9151226d8139c7b5688e83d035a3a747aeb2d` (Round 2 head)
|
|
- **Head Commit**: `ab93776d6d1b8ca534c14b6be53f61dc3a0fa2a9` (Round 3 head)
|
|
- **Delta Under Inspection**: `git diff e8d9151 ab93776` (Commit `ab93776`)
|
|
|
|
---
|
|
|
|
## 1. Executive Summary
|
|
|
|
In Round 2, `claude`, `agy`, and `grok` reached a `PASS` verdict after verifying that Python 3.9 macOS environment and portability blockers were eliminated. However, `codex` reported three valid edge-case issues in the product code (`R1`: unchecked merge pass-through on `herdr config check` execution failure; `R2`: unsafe type coercion in `_shell_in_foreground` comparing non-PIDs with `==`; `R3`: IEEE 754 `NaN != NaN` dropping valid TOML configurations containing `nan`).
|
|
|
|
Commit `ab93776` surgically addresses all three findings (R1, R2, R3) in `lib_py/herdr_config.py` and `lib_py/herdr_restore.py`, accompanied by targeted unit tests in `tests/test_issue45_herdr_config.py` and `tests/test_issue45_restore_unit.py`.
|
|
|
|
All review criteria have been audited, analyzed, and verified across both Python 3.9 and Python 3.11 test environments. The modifications are minimal, robust, fail-safe, and fully within scope.
|
|
|
|
---
|
|
|
|
## 2. Review Findings (R1 ~ R3) Resolution Matrix
|
|
|
|
| # | Item | Defect Description | Resolution in `ab93776` | Status |
|
|
|---|---|---|---|---|
|
|
| **R1** | Unchecked merge pass-through on check failure | If `herdr config check` failed to execute (timeout, `OSError`), `_write_checked` previously treated the merged file as valid and returned `True`. If the unvalidated merge contained a schema error, `herdr` would silently discard the file on startup and re-enable restore, causing Issue #45 to recur. | In `lib_py/herdr_config.py:_write_checked`, an execution error during candidate validation skips any unchecked merged body and advances to `MINIMAL`. If `MINIMAL` also encounters a check execution error, it writes `MINIMAL` with a warning rather than returning no config. | **CLOSED** (Verified) |
|
|
| **R2** | Loose equality in `_shell_in_foreground` | `_shell_in_foreground` compared `shell` and `foreground_process_group_id` directly with `==`, causing matching non-PIDs (`0 == 0`, `False == False`, `"" == ""`, `-1 == -1`) to evaluate to `True` and mistakenly close active panes. | Added `_is_pid(v)` helper: `isinstance(v, int) and not isinstance(v, bool) and v > 0`. `_shell_in_foreground` now requires both `shell` and `fg` to be valid positive integer PIDs. | **CLOSED** (Verified) |
|
|
| **R3** | `NaN` equality dropping valid configs | Standard Python equality `tomllib.loads(text) == expected` fails for valid TOML `nan` values because `float('nan') != float('nan')`, causing all user settings to be discarded in favor of minimal config. | Implemented `_same_doc(a, b)` which recursively verifies dictionary keys and list items while treating `math.isnan(a) and math.isnan(b)` as equal and enforcing strict type equality (`type(a) is type(b)`). | **CLOSED** (Verified) |
|
|
|
|
---
|
|
|
|
## 3. Deep-Dive on Specific Review Criteria
|
|
|
|
### 3.1. Assessment of R1: Unchecked MINIMAL Fallback vs. Returning None
|
|
- **Context & Design Trade-off**: Codex suggested that if validating `MINIMAL` fails or cannot be executed, `write_server_config` should return `None` (no path). The Creator instead decided to emit a warning and return the `MINIMAL` config.
|
|
- **Analysis**:
|
|
- If `write_server_config` returns `None`, `lib.sh` starts `herdr` without setting `HERDR_CONFIG_PATH`. Herdr's built-in default is `resume_agents_on_restore = true`. When the server starts, every previously saved agent session would be relaunched without permissions, resurrecting the primary vulnerability of Issue #45.
|
|
- In contrast, `MINIMAL` is a static, immutable, two-line string:
|
|
```toml
|
|
[session]
|
|
resume_agents_on_restore = false
|
|
```
|
|
This fixed content has zero dynamic variables and has been rigorously verified against `herdr 0.9.1+` (`test_real_herdr_accepts_every_result`).
|
|
- If `herdr config check` cannot be executed due to a transient execution error (e.g., subprocess timeout or file descriptor exhaustion), deploying `MINIMAL` provides the safest available defense-in-depth against unauthorized agent auto-restoration.
|
|
- **Verdict on Criterion 2**: The Creator's decision is sound, defensible, and superior in fail-safety compared to returning `None`.
|
|
|
|
### 3.2. Assessment of R3: Equivalence of `_same_doc`
|
|
- `_same_doc(a, b)` enforces:
|
|
1. `math.isnan(a) and math.isnan(b)` returns `True` for floats.
|
|
2. Dict comparison checks `a.keys() == b.keys()` and recurses on all values.
|
|
3. List comparison checks `len(a) == len(b)` and recurses element-wise in sequence, strictly preserving list order.
|
|
4. Primitive comparison checks `type(a) is type(b) and a == b`.
|
|
- **Type Discrimination**: In Python, `0 == False`, `1 == True`, and `1 == 1.0` evaluate to `True`. Because TOML maintains distinct types for boolean, integer, and float, the added check `type(a) is type(b)` ensures that serialization does not inadvertently morph types (e.g. integer `1` into float `1.0`).
|
|
- **Verdict on Criterion 3**: `_same_doc` preserves identical behavior to `==` for all non-nan types while providing strictly higher fidelity TOML AST verification.
|
|
|
|
### 3.3. Scope and Surgical Precision
|
|
- Modified lines:
|
|
- `lib_py/herdr_config.py`: +31 lines, -2 lines.
|
|
- `lib_py/herdr_restore.py`: +9 lines, -3 lines.
|
|
- `tests/test_issue45_herdr_config.py`: +38 lines.
|
|
- `tests/test_issue45_restore_unit.py`: +56 lines (new unit test file for stubbed PID verification).
|
|
- No extraneous changes or unrelated formatting edits were introduced.
|
|
|
|
---
|
|
|
|
## 4. Verification and Test Results
|
|
|
|
Tests were executed directly within the isolated review verification path:
|
|
`/var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261005T085016Z-c0f6eb/1`
|
|
|
|
### 4.1. Round 3 Target Test Suite (Python 3.9 venv)
|
|
```bash
|
|
AGENT_PYTHON_BIN=/Users/godopu16/PuKi/laa/canary_projects/multi-agent-mux-wt/pr-48/.venv/bin/python \
|
|
PYTHONDONTWRITEBYTECODE=1 \
|
|
/Users/godopu16/PuKi/laa/canary_projects/multi-agent-mux-wt/pr-48/.venv/bin/python \
|
|
-m pytest -p no:cacheprovider -q --basetemp=/tmp/pytest-pr48-r3 \
|
|
tests/test_issue45_herdr_config.py \
|
|
tests/test_issue45_restore_unit.py \
|
|
tests/test_issue45_restore.py \
|
|
tests/test_issue9_server_env.py
|
|
```
|
|
- **Exit Code**: `0`
|
|
- **Result**: `53 passed, 34 skipped in 72.58s (0:01:12)`
|
|
|
|
### 4.2. Python 3.11 Environment Verification
|
|
```bash
|
|
PYTHONPATH=.agents/skills \
|
|
/private/tmp/claude-501/-Users-godopu16-PuKi-laa-canary-projects-multi-agent-mux/dbe6dd10-a210-4b46-b613-58a1860499b6/scratchpad/venv311/bin/python \
|
|
-m pytest -p no:cacheprovider -q --basetemp=/tmp/pytest-pr48-r3-py311 \
|
|
tests/test_issue45_herdr_config.py \
|
|
tests/test_issue45_restore_unit.py
|
|
```
|
|
- **Exit Code**: `0`
|
|
- **Result**: `67 passed in 0.42s`
|
|
|
|
### 4.3. Full Auxiliary & Consistency Suite (Python 3.9 venv)
|
|
```bash
|
|
AGENT_PYTHON_BIN=/Users/godopu16/PuKi/laa/canary_projects/multi-agent-mux-wt/pr-48/.venv/bin/python \
|
|
PYTHONDONTWRITEBYTECODE=1 \
|
|
/Users/godopu16/PuKi/laa/canary_projects/multi-agent-mux-wt/pr-48/.venv/bin/python \
|
|
-m pytest -p no:cacheprovider -q --basetemp=/tmp/pytest-pr48-r3-full \
|
|
tests/test_issue23_24_startup_path.py \
|
|
tests/test_issue42_skill_blocks.py \
|
|
tests/test_version_consistency.py
|
|
```
|
|
- **Exit Code**: `0`
|
|
- **Result**: `110 passed, 1 skipped in 137.12s (0:02:17)`
|
|
|
|
---
|
|
|
|
## 5. Conclusion & Final Verdict
|
|
|
|
All findings from the previous round have been resolved completely, test coverage is thorough and passing across both Python 3.9 and 3.11, and no regressions or scope violations exist.
|
|
|
|
[VERDICT: PASS]
|