feat(issue-18): decide liveness and delivery by the event protocol, not the screen
Contract (no new event types): heartbeat every N s (automatic timer), unresponsive after K x N s without an event, `started` within T s of the injection, retained per-job state, and a remote transport. - registry: heartbeat_sec / heartbeat_misses / ack_timeout_sec / transport fields; idle_timeout_sec derives K x N unless explicit (defaults 30/4/60 keep the historical 120 s idle window). - heartbeat.py + publish_event --heartbeat N: detached fixed-cadence timer publishing signed `progress` heartbeats (threaded wire publish, local record first); stopped by the terminal event via pidfile, by re-dispatch and by lifetime. Terminal events refuse --heartbeat. - publish_event: retained `<prefix>/state` (status, last_event, seq, HMAC) sent on the same connection after every event (--no-state to skip); publisher.publish_raw takes extra messages. - job_subscriber: subscribes the state topic, recovers terminal status from retained state, resets idle on any accepted message, records seq_gap and reconnected in the audit log, idle budget before the first event is T + K x N (--ack-timeout), --remote mirrors started/terminal events into the registry through the transition boundary. - lib.sh send_keys_safe: an accepted-but-unconfirmed submission is confirmed by agent_status working or the job's own `started` (_sks_job_started); pane change and busy token are no longer delivery signals. - wrapper: --heartbeat/--heartbeat-misses/--ack-timeout (MAM_* env), injected start command arms the timer, brief states the contract, _wait_started (registry / events.log / subscriber log, bounded by T and the wall clock) replaces the 10 s inject grace, R1 grace uses only the on-disk event age (herdr/pane logged once as a diagnostic), remote:<name> sessions skip preflight/idle/pane and dispatch over the broker (remote_dispatch). - remote.py (+ scripts shim): signed retained dispatch (MAM_DISPATCH_SECRET) and a runner that mirrors the job into its own registry with the orchestrator's identity, publishes started --heartbeat, runs --exec and publishes the terminal event. - loop runner: passes MAM_HEARTBEAT_SEC / MAM_HEARTBEAT_MISSES / MAM_ACK_TIMEOUT_S through; YAML rows with transport: remote delegate as remote:<name>. - docs: MESSAGING.md 2.5 contract + remote transport, delegate-job SKILL.md, registry.md, loop SKILL.md. - tests: tests/test_issue18_liveness_contract.py (16); issue 14 pane-signal tests flipped to started-based; issue 6 working-agent tests acknowledge with started; CLI snapshot/help captures refreshed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
This commit is contained in:
GodopuandClaude Fable 5.1 committed
1 parent
139bdd4b4c
commit
897b505266
21 files changed
+1989
-129
No files matched your search
+45
-22
@@ -2280,6 +2280,39 @@ except Exception:
|
||||
' <<< "$raw" 2>/dev/null || true
|
||||
}
|
||||
|
||||
# _sks_job_started <job_id> — Issue #18 delivery signal: the job's own
|
||||
# `started` (registry status left `pending`, or a started line in the job's
|
||||
# events.log). Reads the registry the wrapper exports (SKS_REGISTRY_DIR), else
|
||||
# MAM_JOBS_DIR, else $WORKSPACE_ROOT/.mam/jobs. "adhoc" (no job) is never started.
|
||||
_sks_job_started() {
|
||||
local job_id="${1:-adhoc}" rd
|
||||
[ -n "$job_id" ] && [ "$job_id" != "adhoc" ] || return 1
|
||||
rd="${SKS_REGISTRY_DIR:-${MAM_JOBS_DIR:-${WORKSPACE_ROOT:-$PWD}/.mam/jobs}}"
|
||||
"$(_delegate_py_bin)" - "$rd" "$job_id" <<'PYEOF'
|
||||
import json, os, sys
|
||||
rd, jid = sys.argv[1], sys.argv[2]
|
||||
rec = os.path.join(rd, jid + ".json")
|
||||
try:
|
||||
st = json.load(open(rec)).get("status")
|
||||
if st and st != "pending":
|
||||
sys.exit(0)
|
||||
except Exception:
|
||||
pass
|
||||
log = os.path.join(rd, jid + ".events.log")
|
||||
try:
|
||||
with open(log) as fh:
|
||||
for line in fh:
|
||||
try:
|
||||
if json.loads(line).get("event") == "started":
|
||||
sys.exit(0)
|
||||
except Exception:
|
||||
continue
|
||||
except OSError:
|
||||
pass
|
||||
sys.exit(1)
|
||||
PYEOF
|
||||
}
|
||||
|
||||
# _wait_session_gone <sess> [max_sec=5]
|
||||
# Reactive loop to wait until a herdr session goes away (happy path returns early).
|
||||
_wait_session_gone() {
|
||||
@@ -2494,9 +2527,6 @@ send_keys_safe() {
|
||||
# Native herdr 0.8+ fast path: agent prompt handles atomic submission
|
||||
if [ "${SKS_PROMPT_WAIT:-1}" = "1" ]; then
|
||||
local p_out="" p_rc=0
|
||||
# ISSUE-14: pre-submit snapshot for the delivery-confirmation poll below.
|
||||
local rpc_pre_pane=""
|
||||
[ "$_q_rc" != "2" ] && rpc_pre_pane=$(_pane_capture "$sess")
|
||||
local _p_tmp
|
||||
_p_tmp=$(mktemp 2>/dev/null || echo "/tmp/sks_p_${sess}_$$")
|
||||
if _sks_herdr agent prompt "$agent_target" "$text" --wait --until working --timeout 5000 >"$_p_tmp" 2>&1; then
|
||||
@@ -2534,29 +2564,22 @@ send_keys_safe() {
|
||||
if [ "$(_sks_agent_status "$sess" "$agent_target")" = "working" ]; then
|
||||
return 0
|
||||
fi
|
||||
# ISSUE-14: `timeout` / `agent_prompt_stalled` mean herdr ACCEPTED the
|
||||
# submission and sent the input - only its post-submit status watch came
|
||||
# up empty. An agent whose agent_status never leaves `idle` mid-turn
|
||||
# (codex under herdr 0.9.0) therefore failed every delivered prompt here:
|
||||
# the job was closed as agent_blocked and the agent's own `started` /
|
||||
# `completed`, published seconds later, were refused as late. For an
|
||||
# accepted submission keep observing for SKS_DELIVERY_GRACE_S (default
|
||||
# 15) and take any positive signal: agent_status working, a busy token
|
||||
# the pane did not show before the prompt, or the pane moving on from
|
||||
# its pre-submit snapshot. No signal at all still fails closed (rc 4).
|
||||
# ISSUE-14 / ISSUE-18: `timeout` / `agent_prompt_stalled` mean herdr
|
||||
# ACCEPTED the submission and sent the input - only its post-submit
|
||||
# status watch came up empty (codex under herdr 0.9.0 never reports
|
||||
# `working`). For an accepted submission keep observing for
|
||||
# SKS_DELIVERY_GRACE_S (default 15) and take a *protocol* signal only:
|
||||
# agent_status working, or the job's own `started` in the registry
|
||||
# (_sks_job_started). Pane changes and busy tokens are no longer
|
||||
# delivery evidence (Issue #18: they are CLI-version-specific screen
|
||||
# scraping that a headless or remote agent never produces). No signal
|
||||
# still fails closed (rc 4); the wrapper then waits for `started` up to
|
||||
# the job's ack timeout T before finalising the inject as failed.
|
||||
if printf '%s\n' "$p_out" | grep -Eq '"code": *"(timeout|agent_prompt_stalled)"'; then
|
||||
local g_deadline=$(( $(date +%s) + ${SKS_DELIVERY_GRACE_S:-15} ))
|
||||
local pre_busy_rpc=0 cur_pane
|
||||
printf '%s\n' "$rpc_pre_pane" | grep -Fq "esc to interrupt" && pre_busy_rpc=1
|
||||
while [ "$(date +%s)" -lt "$g_deadline" ]; do
|
||||
[ "$(_sks_agent_status "$sess" "$agent_target")" = "working" ] && return 0
|
||||
if [ "$_q_rc" != "2" ]; then
|
||||
cur_pane=$(_pane_capture "$sess")
|
||||
if [ "$pre_busy_rpc" -eq 0 ] && printf '%s\n' "$cur_pane" | grep -Fq "esc to interrupt"; then
|
||||
return 0
|
||||
fi
|
||||
[ -n "$cur_pane" ] && [ "$cur_pane" != "$rpc_pre_pane" ] && return 0
|
||||
fi
|
||||
_sks_job_started "$job_id" && return 0
|
||||
sleep 1
|
||||
done
|
||||
fi
|
||||
|
||||
Reference in new issue
Block a user