Merge-workspace review loop (--all-reviewer): round 1 (@8241bd5) claude grok PASS, agy NOT PASS (two static wrapper-literal tests did not follow _decided/_exit_for), codex NOT PASS (R1: a yes/no decision that could not run read as "no"); fixed in541c3feand65b78b6. Round 2 (@65b78b6) 4/4 PASS. Full suite at65b78b6on macOS / Python 3.9 / herdr 0.9.3: 1730 passed / 0 failed / 56 skipped; 18 of the skips were the uninitialized nats-docker submodule in the fresh worktree, and with it tests/test_deploy_freshness.py is 31 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
119 lines
6.6 KiB
Markdown
119 lines
6.6 KiB
Markdown
# PR Review Report: Issue #17 Followups (PR #51 Post-Merge Fixes)
|
|
|
|
- **Reviewer Agent**: `agy` (session: `herdr:reviewer-agy-01`)
|
|
- **Job ID**: `67a528ff`
|
|
- **Verification Directory**: `/var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261007T224638Z-cf1efb/1`
|
|
- **Base Commit**: `56d0bfda54e4f4fe1ccc138cc262edfb14cfb4a3` (main)
|
|
- **Head Commit**: `8241bd53b179cf8332541264c3bf12b8e3eca44a` (fix/issue17-followups)
|
|
- **Cumulative Commits**: 6 commits (`25c9970` ~ `8241bd5`)
|
|
|
|
---
|
|
|
|
## 1. Executive Summary
|
|
|
|
This review covers the Issue #17 follow-up work bundle addressing:
|
|
1. `herdr 0.9.3` compatibility for removed pane directories (`test_issue45_restore.py`, `lib_py/herdr_restore.py`, `test_issue45_restore_unit.py`).
|
|
2. N2: Fail-closed error handling for failed `_decide` calls (preventing `exit ""` -> rc 255 on bash 3.2).
|
|
3. N1: Docstring clarification in `wrapper.py` regarding package-level paho import.
|
|
4. N3: Comment update in `lib_int.sh` documenting `python3` PATH dependency.
|
|
|
|
While items 1, 3, and 4 are well-implemented and the new unit tests pass cleanly, **item 2 (commit `a670ef5`) introduces regressions in existing test suites** (`test_tier1_unit.py` and `test_issue5_phase1_delegation.py`). The commit replaced raw `_decide` subshell invocations with helper functions `_decided` and `_exit_for` in `multi-agent-mux-delegate-job`, but failed to update the frozen literal assertions in the existing test suite, causing 2 test failures.
|
|
|
|
Therefore, the verdict for this round is **NOT PASS**.
|
|
|
|
---
|
|
|
|
## 2. Evaluation of Task Requirements
|
|
|
|
| Item | Description | Implementation Status | Assessment |
|
|
|---|---|---|---|
|
|
| **1. herdr 0.9.3 Compatibility** | `test_b_only_the_resumed_sessions_restored_shell_is_closed` failed on herdr 0.9.3 because removed pane directories retain their old path rather than resetting to `$HOME`. `lib_py/herdr_restore.py` must close placeholders with `restore_error` and `pane_not_found`. | `tests/test_issue45_restore.py:275`, `.agents/skills/lib_py/herdr_restore.py:130-138`, `tests/test_issue45_restore_unit.py:424-463` | **PASS**: Correctly handles both herdr 0.9.1 and 0.9.3 behavior; unit tests and full restore test pass without touching live sessions. |
|
|
| **2. N2 Fail-Closed on `_decide` Failure** | When `_decide` fails, `exit "$(_decide exit-code ...)"` evaluated to `exit ""` (rc 255 in bash 3.2). The wrapper must exit 1 (fail-closed) on failure, never 255, 0, or empty string. | `multi-agent-mux-delegate-job:100-128`, `tests/test_issue17_item4_decisions.py:309-382` | **NOT PASS (Test Regression)**: Logic in `multi-agent-mux-delegate-job` is sound, but broke existing freeze tests in `test_tier1_unit.py` and `test_issue5_phase1_delegation.py`. |
|
|
| **3. N1 Docstring Clarification** | `wrapper.py` docstring claimed "Stdlib only", but running via `-m lib_py.delegation.wrapper` loads `lib_py/delegation/__init__.py` and transitively requires `paho`. | `lib_py/delegation/wrapper.py:21-29` | **PASS**: Accurate docstring explanation added. |
|
|
| **4. N3 `lib_int.sh` PATH Comment** | Document that `lib_int.sh` uses `python3` from PATH. | `lib_int.sh:9-11` | **PASS**: Comment added cleanly without code churn. |
|
|
|
|
---
|
|
|
|
## 3. Test Failures & Detailed Findings
|
|
|
|
Running the regression suite in the review verification directory (`/var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261007T224638Z-cf1efb/1`):
|
|
```bash
|
|
AGENT_PYTHON_BIN=/Users/godopu16/PuKi/laa/canary_projects/multi-agent-mux-wt/issue17-followups/.venv/bin/python \
|
|
PYTHONDONTWRITEBYTECODE=1 \
|
|
/Users/godopu16/PuKi/laa/canary_projects/multi-agent-mux-wt/issue17-followups/.venv/bin/python \
|
|
-m pytest -p no:cacheprovider -q --basetemp=/tmp/pytest-failed-inspect \
|
|
tests/test_tier1_unit.py::test_g10_delegate_job_rc3_not_mistaken_for_error \
|
|
tests/test_issue5_phase1_delegation.py::test_wrapper_exit_code_mapping_literals_are_frozen
|
|
```
|
|
|
|
### Failure 1: `tests/test_tier1_unit.py:847`
|
|
- **Test**: `test_g10_delegate_job_rc3_not_mistaken_for_error`
|
|
- **Failure**:
|
|
```python
|
|
> assert '_decide sub-status "$sub_rc"' in delegate_script.read_text()
|
|
E AssertionError: assert '_decide sub-status "$sub_rc"' in ...
|
|
```
|
|
- **Cause**: Commit `a670ef5` replaced line 405 (and line 574) of `multi-agent-mux-delegate-job`:
|
|
```bash
|
|
# Before (base 56d0bfd):
|
|
job_status="$(_decide sub-status "$sub_rc" --job "$JOB_ID" --registry-dir "$REGISTRY_DIR")"
|
|
|
|
# After (commit a670ef5):
|
|
_decided job_status sub-status "$sub_rc" --job "$JOB_ID" --registry-dir "$REGISTRY_DIR"
|
|
```
|
|
Because `_decide sub-status "$sub_rc"` no longer appears literally in `multi-agent-mux-delegate-job`, the assertion in `test_tier1_unit.py:847` fails.
|
|
|
|
### Failure 2: `tests/test_issue5_phase1_delegation.py:390`
|
|
- **Test**: `test_wrapper_exit_code_mapping_literals_are_frozen`
|
|
- **Failure**:
|
|
```python
|
|
> for literal in ('_decide sub-status "$sub_rc"', '_decide exit-code "$job_status"', "--sub-rc \"$sub_rc\""):
|
|
> assert literal in text, literal
|
|
E AssertionError: _decide sub-status "$sub_rc"
|
|
```
|
|
- **Cause**: Commit `a670ef5` replaced all occurrences of:
|
|
- `_decide sub-status "$sub_rc"` with `_decided job_status sub-status "$sub_rc"`
|
|
- `exit "$(_decide exit-code "$job_status")"` with `_exit_for "$job_status"`
|
|
Neither literal is present in the wrapper script anymore, causing the freeze verification in `test_issue5_phase1_delegation.py:390` to fail.
|
|
|
|
---
|
|
|
|
## 4. Required Fixes for Next Round
|
|
|
|
In worktree `/Users/godopu16/PuKi/laa/canary_projects/multi-agent-mux-wt/issue17-followups`:
|
|
|
|
1. **Update `tests/test_tier1_unit.py:847`**:
|
|
Change:
|
|
```python
|
|
assert '_decide sub-status "$sub_rc"' in delegate_script.read_text()
|
|
```
|
|
To match the new helper signature:
|
|
```python
|
|
assert '_decided job_status sub-status "$sub_rc"' in delegate_script.read_text()
|
|
```
|
|
|
|
2. **Update `tests/test_issue5_phase1_delegation.py:390`**:
|
|
Change:
|
|
```python
|
|
for literal in ('_decide sub-status "$sub_rc"', '_decide exit-code "$job_status"', "--sub-rc \"$sub_rc\""):
|
|
assert literal in text, literal
|
|
```
|
|
To match the new helper signatures:
|
|
```python
|
|
for literal in ('_decided job_status sub-status "$sub_rc"', '_exit_for "$job_status"', "--sub-rc \"$sub_rc\""):
|
|
assert literal in text, literal
|
|
```
|
|
|
|
3. **Verify Full Test Suite**:
|
|
Run:
|
|
```bash
|
|
AGENT_PYTHON_BIN=.venv/bin/python PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -m pytest -p no:cacheprovider -q tests/test_tier1_unit.py tests/test_issue5_phase1_delegation.py tests/test_issue17_item4_decisions.py tests/test_issue45_restore.py tests/test_issue45_restore_unit.py tests/test_issue33_budget.py
|
|
```
|
|
Confirm all tests pass without errors.
|
|
|
|
---
|
|
|
|
## 5. Final Verdict
|
|
|
|
[VERDICT: NOT PASS]
|