feat(hermes): modernize hermes agent adapter and skills support
- Add --yolo and --accept-hooks headless auto-approval flags to spawn_spec and resume_spec
- Define Hermes TUI input delimiters (input_prompt='❯', input_rule_pattern='─{10,}')
- Refactor verify_artifact and discover in hermes.py using session started_at timestamps
- Fix HERDR_EPOCH capture timing before spawn in create_session.sh to prevent epoch race
- Update reconcile.sh for hermes drift-C multi-candidate and sibling claimed exclusion
- Add Hermes contract, epoch filtering, and C-ambiguous unit tests (140 passed)
- Add agent evaluation report and final review reports for Hermes support
This commit is contained in:
@@ -0,0 +1,22 @@
|
||||
# Report: Job a7a2da9f — Refined Plan (Rev.2) per `creator-agy-01` Challenge
|
||||
|
||||
**Durable output (updated in place)**: [.agents/reports/planner-reviewer-claude-01/plan-28f9b565.md](../../../.agents/reports/planner-reviewer-claude-01/plan-28f9b565.md)
|
||||
|
||||
## Summary
|
||||
|
||||
`creator-agy-01` filed a formal architectural challenge (job `cd64ae0b`) against Rev.1's hermes audit plan, identifying that **Rev.1's own proposed §2.4 fix for the `C-ambiguous` bug was itself regressive**: it widened `reconcile.sh`'s candidate query but still relied on `adapters/hermes.py::verify_artifact()`, which checks the shared `~/.hermes/state.db` file's mtime rather than the individual session row's `started_at`. Since all hermes sessions across all workspaces share one `state.db` file, any recent write makes the file-level epoch check pass for essentially every historical session row — meaning Rev.1's fix would have turned "never detects `C-ambiguous`" into "permanently false-positives `C-ambiguous` in any workspace with prior hermes history." A second, lower-severity point noted that `hermes --resume` auto-`cd`s into its recorded `cwd`, risking divergence from MAM's symlink-canonicalized path model, and recommended adding `--no-restore-cwd`.
|
||||
|
||||
**Disposition**: I independently re-verified both claims against the actual code (`hermes.py::verify_artifact`'s file-mtime check; `verify_session.py`'s `epoch = row.get("herdr_session_epoch", 0)`) before accepting — both are correct. **Both accepted — no `[REBUT:]` filed.**
|
||||
|
||||
## What changed in Rev.2
|
||||
|
||||
- Added §0 changelog cross-referencing each challenge point.
|
||||
- §2.4 replaced (not appended) with a two-layer fix: `verify_artifact()` now checks the session row's own `started_at` instead of the shared file's mtime, and `reconcile.sh`'s query adds a `started_at >= ?` SQL-level filter alongside the existing candidate-widening + sibling-exclusion logic from Rev.1.
|
||||
- §2.1's canonical `resume_spec()` and §2.6 updated to add `--no-restore-cwd` alongside `--yolo --accept-hooks`.
|
||||
- Comparison table and Definition of Done updated; the regression test requirement now explicitly covers **both** directions (under-detection and the Rev.1 over-detection regression), not just the original under-detection case.
|
||||
|
||||
Full Rev.2 text, including both corrected code diffs, is in the durable report linked above.
|
||||
|
||||
**Process note**: Per `MULTI_AGENT_RULES.md` §3.1, this Rev.2 still requires a fresh `[VERDICT: PASS]` before the §2.4 fix — the highest-risk change in this plan — is merged.
|
||||
|
||||
[VERDICT: N/A — planning artifact, review pending]
|
||||
@@ -0,0 +1,96 @@
|
||||
# 🔍 Cross-Code Review — Hermes Agent Support Implementation, Follow-Up (Job fcb16ed6)
|
||||
|
||||
- **Reviewer**: `planner-reviewer-claude-01`
|
||||
- **Target diff**: revised implementation of `plan-28f9b565.md`, superseding the diff reviewed in job `685bb381` (`[VERDICT: PASS]`, with 2 non-blocking findings). This diff addresses both of those findings plus adds two proactive fixes not previously flagged.
|
||||
- **Method**: same as the prior review — read every changed file's live post-diff state directly (branch `support-hermes`), traced call chains, ran `bash -n` syntax checks, and ran the actual test suite.
|
||||
|
||||
---
|
||||
|
||||
## 1. Disposition of my Previous Review's Findings (job 685bb381)
|
||||
|
||||
| Finding | Status this round |
|
||||
|---|---|
|
||||
| §4.1: `multi-agent-mux-resume/SKILL.md` doc example was stale (didn't include `--no-restore-cwd --yolo --accept-hooks`) | ✅ **Fixed.** Now reads `hermes) CMD_FULL="hermes --resume $UUID --no-restore-cwd --yolo --accept-hooks" ;;` — verified it matches `resume_session.sh`'s real fallback string exactly (modulo the illustrative literal binary name, consistent with how the doc renders every other agent's row). |
|
||||
| §4.2: `_sibling_claimed_uuids` set on the hermes row in `reconcile.sh` but never read by `hermes.py::verify_artifact()` (unlike `agy`'s adapter-level pattern) | **Not touched directly** — `hermes.py::verify_artifact()` still doesn't read `ctx.row`. This is unchanged from the prior diff and remains a minor architectural-parity note, not a live bug (the explicit loop-level `if uuid in sibling_claimed: continue` in `reconcile.sh` still enforces it correctly on its own, confirmed again this round — see §3). Still non-blocking. |
|
||||
|
||||
---
|
||||
|
||||
## 2. New Changes Beyond the Prior Diff (not requested by my previous review — found and verified independently)
|
||||
|
||||
### 2.1 `hermes.py::discover()` rewritten to return multiple candidates (previously `LIMIT 1`)
|
||||
|
||||
```python
|
||||
def discover(self, ctx: DiscoveryContext) -> list:
|
||||
...
|
||||
if ctx.epoch:
|
||||
rows = conn.execute("SELECT id FROM sessions WHERE cwd=? AND started_at >= ? ORDER BY started_at DESC LIMIT 20", ...)
|
||||
else:
|
||||
rows = conn.execute("SELECT id FROM sessions WHERE cwd=? ORDER BY started_at DESC LIMIT 20", ...)
|
||||
...
|
||||
candidates = []
|
||||
for (cand,) in rows:
|
||||
if cand and self.verify_artifact(cand, ctx):
|
||||
candidates.append(cand)
|
||||
return candidates
|
||||
```
|
||||
|
||||
This is a genuine, previously-unflagged fix: the old `discover()` used `fetchone()`/`LIMIT 1`, making hermes the only adapter whose `discover()` could never surface more than one candidate — inconsistent with `grok.py`/`agy.py`/`cline.py`, whose `discover()` methods already return every verified candidate. **Checked for regression risk**: grepped every call site of `.discover(` across the repo — the only real caller is `workspace_uuid.py:81` (`for cand in adapter.discover(ctx):`), which already iterates rather than indexing, so it handles 0/1/many candidates identically regardless of agent. No caller assumes a single-element list. This change makes hermes consistent with the rest of the framework rather than introducing risk.
|
||||
|
||||
The new `test_hermes_verify_artifact_spawn_epoch_timing_f1` test specifically exercises `discover()` with a mix of old (`started_at < epoch`) and fresh rows and asserts only the fresh one survives — I re-ran it directly, passes.
|
||||
|
||||
### 2.2 `create_session.sh`: `HERDR_EPOCH` capture moved earlier, before `spawn()`
|
||||
|
||||
```diff
|
||||
+HERDR_EPOCH=$(date +%s)
|
||||
+NOW_ISO=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
|
||||
+
|
||||
spawn
|
||||
...
|
||||
-HERDR_EPOCH=$(date +%s)
|
||||
-NOW_ISO=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
|
||||
```
|
||||
|
||||
This closes a real timing edge case in Rev.2's own epoch-based fix that neither my review nor the plan itself had caught: previously `HERDR_EPOCH` was captured **after** `spawn()`, `wait_for_tui_ready`, and pane-metadata resolution — all of which can take multiple seconds. Since `HERDR_EPOCH` becomes `herdr_session_epoch` in the registry and flows into `ctx.epoch` for the new `started_at >= ctx.epoch` check, a late-captured epoch could end up **later** than the freshly-spawned session's own `started_at`, causing the epoch guard to falsely reject the very session it was meant to validate — self-defeating. Capturing it immediately before `spawn()` guarantees `HERDR_EPOCH <= started_at` for the session about to be created.
|
||||
|
||||
**Verified this is safe for every agent, not just hermes**: `HERDR_EPOCH` is a single shared line (not gated by `$AGENT`), consumed generically by `verify_session_uuid`'s `epoch = row.get("herdr_session_epoch", 0)` for all agents. For claude/agy/grok, whose `verify_artifact()` implementations use `os.path.getmtime(path) < ctx.epoch`, an earlier epoch can only make that check *more* lenient (smaller epoch → less likely to be `>` a file's mtime), never *more* restrictive — so this is a strictly safe, general robustness improvement, not a hermes-only special case. `grep -rn "HERDR_EPOCH"` confirms its only consumer (`create_session.sh:346`, feeding `lib.sh`'s YAML-write step) is unaffected by the earlier capture point — it only needs the variable to exist by the time it's read, which it still does.
|
||||
|
||||
---
|
||||
|
||||
## 3. Re-Verification of Everything from the Prior (Already-PASSed) Review
|
||||
|
||||
Re-traced and re-confirmed unchanged/still-correct (identical diff hash `974a367` for `reconcile.sh` vs. the prior review — no regression risk here, but re-verified functionally with the fuller test run below):
|
||||
- Core `C-ambiguous` epoch fix (`verify_artifact` row-level `started_at` check + `reconcile.sh`'s `started_at >= ?` SQL filter + sibling-exclusion loop) — still correct, still passes its dual-direction regression test.
|
||||
- `--yolo --accept-hooks` / `--no-restore-cwd` wiring across `spawn_spec`/`resume_spec`/`create_session.sh`/`resume_session.sh`/both `SKILL.md` files — consistent everywhere, no drift.
|
||||
- `input_prompt`/`input_placeholder`/`input_rule_pattern`/`ready_tokens` — unchanged, still correct.
|
||||
|
||||
`bash -n` on `create_session.sh` (re-checked after the epoch-timing edit) — valid.
|
||||
|
||||
---
|
||||
|
||||
## 4. Test Suite — Functional Verification
|
||||
|
||||
```
|
||||
.venv/bin/python -m pytest tests/test_a4_adapter_contract.py -q -v
|
||||
→ 19 passed (17 from before + 2 new: test_hermes_verify_artifact_spawn_epoch_timing_f1, test_hermes_reconcile_full_block_integration)
|
||||
|
||||
.venv/bin/python -m pytest tests/ -q
|
||||
→ 441 passed in 625.89s (0:10:25), exit code 0
|
||||
```
|
||||
|
||||
I ran the full suite myself (not the diff's own claims) rather than sampling only the adapter-contract file, given this diff touches session-creation timing (`create_session.sh`) which is broader-surface than the previous diff. **441 passed vs. 439 in the previous review — the +2 matches exactly the two new hermes regression tests added in this diff, and there are zero failures.**
|
||||
|
||||
---
|
||||
|
||||
## 5. Findings Carried Forward (non-blocking, unchanged from job 685bb381)
|
||||
|
||||
- `_sibling_claimed_uuids` is still set-but-unread at the `hermes.py` adapter level (§1 above). Still recommend either removing the unused field or moving the check into `verify_artifact()` for architectural parity with `agy`'s pattern — purely a consistency/future-proofing item, not a live defect.
|
||||
|
||||
No new findings beyond that one. No design rework is warranted — this round demonstrably improved on the already-PASSed implementation rather than introducing regressions.
|
||||
|
||||
---
|
||||
|
||||
## 6. Verdict
|
||||
|
||||
Both items from my previous review are now closed (one fully fixed, one confirmed still non-blocking and unchanged), and the two additional changes in this diff (`discover()` multi-candidate parity, `HERDR_EPOCH` pre-spawn capture) are correct, well-targeted fixes to real edge cases in the underlying epoch-based design — verified independently via call-site tracing and the full test suite, not accepted on the diff's own say-so. No correctness defects found; no escalation warranted.
|
||||
|
||||
[VERDICT: PASS]
|
||||
Reference in New Issue
Block a user