fix(issue-18): default ack timeout T = 180 s
In the v4.5.0 review round grok read the brief and published `started` 70 s after the injection; with T = 60 the wrapper had already closed the job as agent_blocked and refused its later PASS. T bounds delivery, not work, so a generous default costs nothing when the agent answers fast. Code, docs and the CLI snapshot updated together. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
This commit is contained in:
1 parent
d09c1627ce
commit
77f2829b8e
10 files changed
+15
-13
No files matched your search
@@ -168,7 +168,7 @@ To ensure that agents running in TMUX environments do not lose debug logs or pre
|
|||||||
- **Identity Verification**: The guard validates process liveness via `pid` + `lstart` to prevent livelocks on PID rollover.
|
- **Identity Verification**: The guard validates process liveness via `pid` + `lstart` to prevent livelocks on PID rollover.
|
||||||
|
|
||||||
### ⏱️ Timeout Configuration & Alignment Rules
|
### ⏱️ Timeout Configuration & Alignment Rules
|
||||||
- **Job Execution Limits (`timeout_sec` & `idle_timeout_sec`)**: Each job independently manages its overall execution timeout (`timeout_sec`, default 3600s) and idle timeout without receiving messages (`idle_timeout_sec`). Since v4.5.0 (Issue #18) the idle timeout is the liveness contract's K x N: the agent heartbeats every `heartbeat_sec` (N, default 30 — automatic once it publishes `started --heartbeat N`), and `heartbeat_misses` (K, default 4) missed heartbeats close the job as `deadline_timeout` (default 120 s, `--idle-timeout` overrides). `started` must arrive within `ack_timeout_sec` (T, default 60) of the injection or the job fails as `agent_blocked` without re-injection. herdr status and pane text are never used for these decisions.
|
- **Job Execution Limits (`timeout_sec` & `idle_timeout_sec`)**: Each job independently manages its overall execution timeout (`timeout_sec`, default 3600s) and idle timeout without receiving messages (`idle_timeout_sec`). Since v4.5.0 (Issue #18) the idle timeout is the liveness contract's K x N: the agent heartbeats every `heartbeat_sec` (N, default 30 — automatic once it publishes `started --heartbeat N`), and `heartbeat_misses` (K, default 4) missed heartbeats close the job as `deadline_timeout` (default 120 s, `--idle-timeout` overrides). `started` must arrive within `ack_timeout_sec` (T, default 180) of the injection or the job fails as `agent_blocked` without re-injection. herdr status and pane text are never used for these decisions.
|
||||||
- **Heartbeat timer and agent life**: the automatic timer is bound to the agent process that ran the start command (the nearest agent-CLI ancestor, or `--watch-pid`) and stops within one period once that process is gone; a crashed agent is therefore closed after K x N like any silent one, while an agent that is alive but stuck keeps heartbeating (process liveness, not work liveness).
|
- **Heartbeat timer and agent life**: the automatic timer is bound to the agent process that ran the start command (the nearest agent-CLI ancestor, or `--watch-pid`) and stops within one period once that process is gone; a crashed agent is therefore closed after K x N like any silent one, while an agent that is alive but stuck keeps heartbeating (process liveness, not work liveness).
|
||||||
- **Monitor Idle Waiting (`SUB_IDLE_TIMEOUT`)**: The idle timeout for the monitor script (`reconcile.sh`), `SUB_IDLE_TIMEOUT`, must always be set generously to `3600s` (1 hour) or more to align with the maximum job budget. This prevents the monitor from terminating early due to idle detection, which would lose control over background tasks before they finish.
|
- **Monitor Idle Waiting (`SUB_IDLE_TIMEOUT`)**: The idle timeout for the monitor script (`reconcile.sh`), `SUB_IDLE_TIMEOUT`, must always be set generously to `3600s` (1 hour) or more to align with the maximum job budget. This prevents the monitor from terminating early due to idle detection, which would lose control over background tasks before they finish.
|
||||||
|
|
||||||
|
|||||||
@@ -52,10 +52,12 @@ def generate_job_id(bits: int = 32) -> str:
|
|||||||
# heartbeats (K x N seconds since the last event) as unresponsive; `started`
|
# heartbeats (K x N seconds since the last event) as unresponsive; `started`
|
||||||
# must arrive within T seconds of the prompt injection. K x N with these
|
# must arrive within T seconds of the prompt injection. K x N with these
|
||||||
# defaults equals the historical 120 s idle timeout, so a caller that never
|
# defaults equals the historical 120 s idle timeout, so a caller that never
|
||||||
# asked for the contract sees the same idle window as before.
|
# asked for the contract sees the same idle window as before. T is generous:
|
||||||
|
# a real agent reads the brief and reasons before it runs the start command
|
||||||
|
# (grok took ~70 s in the v4.5.0 review round), and T only bounds delivery.
|
||||||
DEFAULT_HEARTBEAT_SEC = 30
|
DEFAULT_HEARTBEAT_SEC = 30
|
||||||
DEFAULT_HEARTBEAT_MISSES = 4
|
DEFAULT_HEARTBEAT_MISSES = 4
|
||||||
DEFAULT_ACK_TIMEOUT_SEC = 60
|
DEFAULT_ACK_TIMEOUT_SEC = 180
|
||||||
|
|
||||||
|
|
||||||
def liveness_window(heartbeat_sec: int, heartbeat_misses: int) -> int:
|
def liveness_window(heartbeat_sec: int, heartbeat_misses: int) -> int:
|
||||||
|
|||||||
@@ -40,7 +40,7 @@ multi-agent-mux-delegate-job submit \
|
|||||||
--agent-session herdr:<session_name> \
|
--agent-session herdr:<session_name> \
|
||||||
--prompt "Task description or instructions here" \
|
--prompt "Task description or instructions here" \
|
||||||
--role <Worker|Planner|Reviewer> \
|
--role <Worker|Planner|Reviewer> \
|
||||||
--timeout 3600 [--heartbeat 30 --heartbeat-misses 4 --ack-timeout 60] [--idle-timeout <sec>]
|
--timeout 3600 [--heartbeat 30 --heartbeat-misses 4 --ack-timeout 180] [--idle-timeout <sec>]
|
||||||
# Liveness contract (Issue #18): the agent heartbeats every N s (automatic),
|
# Liveness contract (Issue #18): the agent heartbeats every N s (automatic),
|
||||||
# the job is unresponsive after K x N s (--idle-timeout overrides), `started`
|
# the job is unresponsive after K x N s (--idle-timeout overrides), `started`
|
||||||
# must arrive within T s of the injection or the inject fails closed.
|
# must arrive within T s of the injection or the inject fails closed.
|
||||||
|
|||||||
@@ -86,7 +86,7 @@ multi-agent-mux-delegate-job <command> [options]
|
|||||||
# Liveness contract: the agent heartbeats every --heartbeat N s
|
# Liveness contract: the agent heartbeats every --heartbeat N s
|
||||||
# (default 30, automatic), the job is unresponsive after
|
# (default 30, automatic), the job is unresponsive after
|
||||||
# --heartbeat-misses K x N s (default 4 x 30 = 120; --idle-timeout
|
# --heartbeat-misses K x N s (default 4 x 30 = 120; --idle-timeout
|
||||||
# overrides), `started` must arrive within --ack-timeout T s (60).
|
# overrides), `started` must arrive within --ack-timeout T s (180).
|
||||||
status --job <id> [--registry-dir <dir>]
|
status --job <id> [--registry-dir <dir>]
|
||||||
list [--registry-dir <dir>]
|
list [--registry-dir <dir>]
|
||||||
verify --job <id> --validate <script> [--registry-dir <dir>]
|
verify --job <id> --validate <script> [--registry-dir <dir>]
|
||||||
@@ -99,7 +99,7 @@ EOF
|
|||||||
AGENT="claude-code"; PROMPT=""; WORKDIR="$(pwd)"; AGENT_SESSION="herdr:claude"
|
AGENT="claude-code"; PROMPT=""; WORKDIR="$(pwd)"; AGENT_SESSION="herdr:claude"
|
||||||
TIMEOUT=3600; IDLE_TIMEOUT=""; VALIDATE=""; DRY_RUN=0
|
TIMEOUT=3600; IDLE_TIMEOUT=""; VALIDATE=""; DRY_RUN=0
|
||||||
# Issue #18 liveness contract (N / K / T). An unset --idle-timeout derives K x N.
|
# Issue #18 liveness contract (N / K / T). An unset --idle-timeout derives K x N.
|
||||||
HEARTBEAT_SEC="${MAM_HEARTBEAT_SEC:-30}"; HEARTBEAT_MISSES="${MAM_HEARTBEAT_MISSES:-4}"; ACK_TIMEOUT="${MAM_ACK_TIMEOUT_S:-60}"
|
HEARTBEAT_SEC="${MAM_HEARTBEAT_SEC:-30}"; HEARTBEAT_MISSES="${MAM_HEARTBEAT_MISSES:-4}"; ACK_TIMEOUT="${MAM_ACK_TIMEOUT_S:-180}"
|
||||||
JOB_ID=""; REGISTRY_DIR="$REGISTRY_DIR_DEFAULT"; DELEGATE_ROLE="Worker"
|
JOB_ID=""; REGISTRY_DIR="$REGISTRY_DIR_DEFAULT"; DELEGATE_ROLE="Worker"
|
||||||
TYPE="direct"; REVIEWER="hermes"; REVIEWER_SESSION="herdr:hermes"; MAX_ITERATIONS=5
|
TYPE="direct"; REVIEWER="hermes"; REVIEWER_SESSION="herdr:hermes"; MAX_ITERATIONS=5
|
||||||
DEFAULT_COUNTERPART_ROLE="Reviewer"
|
DEFAULT_COUNTERPART_ROLE="Reviewer"
|
||||||
|
|||||||
@@ -201,7 +201,7 @@ $PY scripts/registry.py logs --list # every job, live status
|
|||||||
| `heartbeat_sec` | N: the agent publishes `progress` at least every N s (the `--heartbeat N` timer does it) | 30 |
|
| `heartbeat_sec` | N: the agent publishes `progress` at least every N s (the `--heartbeat N` timer does it) | 30 |
|
||||||
| `heartbeat_misses` | K: missed heartbeats before the job is unresponsive | 4 |
|
| `heartbeat_misses` | K: missed heartbeats before the job is unresponsive | 4 |
|
||||||
| `idle_timeout_sec` | K x N unless `--idle-timeout` was given | 120 |
|
| `idle_timeout_sec` | K x N unless `--idle-timeout` was given | 120 |
|
||||||
| `ack_timeout_sec` | T: `started` must arrive within T s of the injection | 60 |
|
| `ack_timeout_sec` | T: `started` must arrive within T s of the injection | 180 |
|
||||||
| `transport` | `herdr` (pane injection) or `remote` (signed retained dispatch over the broker) | `herdr` |
|
| `transport` | `herdr` (pane injection) or `remote` (signed retained dispatch over the broker) | `herdr` |
|
||||||
|
|
||||||
`<registry>/<job>.heartbeat.pid` / `.heartbeat.log` belong to the running heartbeat timer; a terminal publish removes the pidfile.
|
`<registry>/<job>.heartbeat.pid` / `.heartbeat.log` belong to the running heartbeat timer; a terminal publish removes the pidfile.
|
||||||
@@ -47,7 +47,7 @@ the injection. Tune it for a whole loop with environment variables, which the
|
|||||||
runner passes through as `--heartbeat` / `--heartbeat-misses` / `--ack-timeout`:
|
runner passes through as `--heartbeat` / `--heartbeat-misses` / `--ack-timeout`:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
MAM_HEARTBEAT_SEC=30 MAM_HEARTBEAT_MISSES=4 MAM_ACK_TIMEOUT_S=60 \
|
MAM_HEARTBEAT_SEC=30 MAM_HEARTBEAT_MISSES=4 MAM_ACK_TIMEOUT_S=180 \
|
||||||
bash .agents/skills/multi-agent-mux-loop/scripts/run_loop.sh --creator creator-claude-01 --all-reviewer --task "..."
|
bash .agents/skills/multi-agent-mux-loop/scripts/run_loop.sh --creator creator-claude-01 --all-reviewer --task "..."
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -198,7 +198,7 @@ orchestrator's disk, take part on equal terms.
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| **Heartbeat obligation** | While working, the agent publishes `progress` at least every **N** s (`heartbeat_sec`, default 30). `publish_event.py --event started --heartbeat N` starts a detached timer that publishes `progress` (`detail: heartbeat`) every N s with the same `dispatch_token`/`attempt_id`, so the agent does not have to remember. The timer stops itself at the terminal event (which also kills it through `<registry>/<job>.heartbeat.pid`), on re-dispatch, after `timeout_sec`, and within one period of the agent process it watches going away (`--watch-pid`, default the nearest agent-CLI ancestor of the publishing command; a remote runner watches itself) - a crashed agent stops heartbeating, an alive-but-stuck one does not. | `lib_py/delegation/heartbeat.py`, `publish_event.py --heartbeat` |
|
| **Heartbeat obligation** | While working, the agent publishes `progress` at least every **N** s (`heartbeat_sec`, default 30). `publish_event.py --event started --heartbeat N` starts a detached timer that publishes `progress` (`detail: heartbeat`) every N s with the same `dispatch_token`/`attempt_id`, so the agent does not have to remember. The timer stops itself at the terminal event (which also kills it through `<registry>/<job>.heartbeat.pid`), on re-dispatch, after `timeout_sec`, and within one period of the agent process it watches going away (`--watch-pid`, default the nearest agent-CLI ancestor of the publishing command; a remote runner watches itself) - a crashed agent stops heartbeating, an alive-but-stuck one does not. | `lib_py/delegation/heartbeat.py`, `publish_event.py --heartbeat` |
|
||||||
| **Unresponsive** | The orchestrator treats a job as unresponsive after **K x N** s without any event (`heartbeat_misses`, default 4 → 120 s; `--idle-timeout` overrides). The subscriber's idle clock counts every accepted message (`progress` and retained state included); the wrapper's post-idle grace re-reads only the job's events.log (`_recent_disk_heartbeat`, written before every wire publish). herdr status and the pane are logged once as a *diagnostic* (`liveness: ... herdr/pane says busy=yes/no (diagnostic only)`) and never decide. | `multi-agent-mux-delegate-job` (`finalize_unfinished`), `job_subscriber.py` |
|
| **Unresponsive** | The orchestrator treats a job as unresponsive after **K x N** s without any event (`heartbeat_misses`, default 4 → 120 s; `--idle-timeout` overrides). The subscriber's idle clock counts every accepted message (`progress` and retained state included); the wrapper's post-idle grace re-reads only the job's events.log (`_recent_disk_heartbeat`, written before every wire publish). herdr status and the pane are logged once as a *diagnostic* (`liveness: ... herdr/pane says busy=yes/no (diagnostic only)`) and never decide. | `multi-agent-mux-delegate-job` (`finalize_unfinished`), `job_subscriber.py` |
|
||||||
| **Delivery** | After the prompt is handed over (herdr accepted the submission, or the remote dispatch was published), the agent's `started` must arrive within **T** s (`ack_timeout_sec`, default 60): the registry leaves `pending`, a `started` line lands in events.log, or the subscriber prints one. No `started` within T → the inject fails closed: rc 1, registry `error` (`agent_blocked`), and the prompt is **never re-injected**. `send_keys_safe` confirms an accepted-but-unconfirmed submission only by `agent_status == working` or the job's `started`; pane changes and busy tokens are no longer signals. | `_wait_started`, `lib.sh: _sks_job_started` |
|
| **Delivery** | After the prompt is handed over (herdr accepted the submission, or the remote dispatch was published), the agent's `started` must arrive within **T** s (`ack_timeout_sec`, default 180; a real agent reads the brief before it runs the start command): the registry leaves `pending`, a `started` line lands in events.log, or the subscriber prints one. No `started` within T → the inject fails closed: rc 1, registry `error` (`agent_blocked`), and the prompt is **never re-injected**. `send_keys_safe` confirms an accepted-but-unconfirmed submission only by `agent_status == working` or the job's `started`; pane changes and busy tokens are no longer signals. | `_wait_started`, `lib.sh: _sks_job_started` |
|
||||||
| **Termination** | Only `completed` / `error` (and an operator `cancelled`) end a job. Terminal immutability (Issue #6 CAS) refuses anything that arrives after `deadline_timeout` closed the job. | `engine.py` |
|
| **Termination** | Only `completed` / `error` (and an operator `cancelled`) end a job. Terminal immutability (Issue #6 CAS) refuses anything that arrives after `deadline_timeout` closed the job. | `engine.py` |
|
||||||
| **Wire durability** | Every event (QoS 1) is followed, on the same connection, by a **retained** `<topic_prefix>/state` message (`event: state`, `status`, `last_event`, `seq`, signed like an event). A subscriber that connects or reconnects receives it at once: a terminal status ends the wait, any state resets the idle window, and a `seq` beyond the last one seen is recorded as `seq_gap` in the audit log (also for gaps between events). paho reconnects on its own; `on_connect` re-subscribes both topics and logs `reconnected`. | `publish_event.py: build_state_payload`, `job_subscriber.py: _on_state` |
|
| **Wire durability** | Every event (QoS 1) is followed, on the same connection, by a **retained** `<topic_prefix>/state` message (`event: state`, `status`, `last_event`, `seq`, signed like an event). A subscriber that connects or reconnects receives it at once: a terminal status ends the wait, any state resets the idle window, and a `seq` beyond the last one seen is recorded as `seq_gap` in the audit log (also for gaps between events). paho reconnects on its own; `on_connect` re-subscribes both topics and logs `reconnected`. | `publish_event.py: build_state_payload`, `job_subscriber.py: _on_state` |
|
||||||
| **Credentials** | Each host connects with its own `.mam.env` (Issue #12: the password is never in a record). Every event and state message carries the job's HMAC (`auth_token`). A remote dispatch is signed with the pre-shared `MAM_DISPATCH_SECRET`. | `mqtt_common.py`, `remote.py` |
|
| **Credentials** | Each host connects with its own `.mam.env` (Issue #12: the password is never in a record). Every event and state message carries the job's HMAC (`auth_token`). A remote dispatch is signed with the pre-shared `MAM_DISPATCH_SECRET`. | `mqtt_common.py`, `remote.py` |
|
||||||
|
|||||||
+2
-2
@@ -40,12 +40,12 @@
|
|||||||
|
|
||||||
### 🚀 `v4.5.0` — Issue #18 생존·전달 판정을 프로토콜 계약으로 완결: heartbeat 자동화, K×N 무응답, T 내 `started`, retained 상태, 원격 전송 (2026-09-13)
|
### 🚀 `v4.5.0` — Issue #18 생존·전달 판정을 프로토콜 계약으로 완결: heartbeat 자동화, K×N 무응답, T 내 `started`, retained 상태, 원격 전송 (2026-09-13)
|
||||||
|
|
||||||
> **마이너 마일스톤 (MINOR Release)**: 에이전트의 생존·전달 판정을 TUI 스크래핑(herdr `agent_status`, pane 문자열, 화면 변화)에서 떼어내어, 에이전트가 스스로 발행하는 이벤트(`started`/`progress`/`completed`/`error`)에 **의무·주기·타임아웃 규칙이 있는 계약**을 부여했다(Issue #18, #17 항목 1). 새 이벤트 종류는 없다. 원격 호스트의 에이전트(pane·디스크 공유 없음)를 같은 계약으로 다루는 `remote:` 전송을 추가했다. 래퍼 옵션 3개(`--heartbeat`, `--heartbeat-misses`, `--ack-timeout`)와 `publish_event --heartbeat`, `job_subscriber --remote`, `remote.py` CLI 가 추가되어 MINOR 로 올린다. 기본값(N=30, K=4 → idle 120 s, T=60)은 종전 idle 기본값과 같으므로 기존 호출은 동작이 유지된다.
|
> **마이너 마일스톤 (MINOR Release)**: 에이전트의 생존·전달 판정을 TUI 스크래핑(herdr `agent_status`, pane 문자열, 화면 변화)에서 떼어내어, 에이전트가 스스로 발행하는 이벤트(`started`/`progress`/`completed`/`error`)에 **의무·주기·타임아웃 규칙이 있는 계약**을 부여했다(Issue #18, #17 항목 1). 새 이벤트 종류는 없다. 원격 호스트의 에이전트(pane·디스크 공유 없음)를 같은 계약으로 다루는 `remote:` 전송을 추가했다. 래퍼 옵션 3개(`--heartbeat`, `--heartbeat-misses`, `--ack-timeout`)와 `publish_event --heartbeat`, `job_subscriber --remote`, `remote.py` CLI 가 추가되어 MINOR 로 올린다. 기본값(N=30, K=4 → idle 120 s, T=180)은 종전 idle 기본값과 같으므로 기존 호출은 동작이 유지된다.
|
||||||
|
|
||||||
#### 🌟 주요 개선 사항
|
#### 🌟 주요 개선 사항
|
||||||
1. **heartbeat 의무의 자동화 (`lib_py/delegation/heartbeat.py`, `publish_event.py --heartbeat N`)**: `started --heartbeat N` 이 분리 세션의 타이머를 띄워 N초마다 `progress`(detail `heartbeat`)를 같은 `dispatch_token`/`attempt_id` 로 서명·발행한다. 터미널 이벤트가 pidfile(`<registry>/<job>.heartbeat.pid`)로 타이머를 먼저 멈추고(pid 재사용 대비 cmdline 확인), 타이머는 레코드 터미널·재dispatch·`timeout_sec` 초과·**감시 대상 에이전트 프로세스 소멸**(`--watch-pid`, 기본은 시작 명령을 실행한 가장 가까운 에이전트 CLI 조상; 원격 러너는 자기 자신) 시 스스로 끝난다. 죽은 에이전트는 heartbeat 를 멈추고 K×N 뒤 닫히며, 살아 있지만 멈춘 에이전트는 계속 heartbeat 를 낸다(프로세스 생존 계약). 각 beat 는 와이어 이전에 events.log 에 기록되므로 디스크 생존 판정에도 그대로 쓰인다. 래퍼가 주입하는 시작 명령이 `--heartbeat N` 을 포함하고, 브리프에 계약(N/K/T)이 명시된다.
|
1. **heartbeat 의무의 자동화 (`lib_py/delegation/heartbeat.py`, `publish_event.py --heartbeat N`)**: `started --heartbeat N` 이 분리 세션의 타이머를 띄워 N초마다 `progress`(detail `heartbeat`)를 같은 `dispatch_token`/`attempt_id` 로 서명·발행한다. 터미널 이벤트가 pidfile(`<registry>/<job>.heartbeat.pid`)로 타이머를 먼저 멈추고(pid 재사용 대비 cmdline 확인), 타이머는 레코드 터미널·재dispatch·`timeout_sec` 초과·**감시 대상 에이전트 프로세스 소멸**(`--watch-pid`, 기본은 시작 명령을 실행한 가장 가까운 에이전트 CLI 조상; 원격 러너는 자기 자신) 시 스스로 끝난다. 죽은 에이전트는 heartbeat 를 멈추고 K×N 뒤 닫히며, 살아 있지만 멈춘 에이전트는 계속 heartbeat 를 낸다(프로세스 생존 계약). 각 beat 는 와이어 이전에 events.log 에 기록되므로 디스크 생존 판정에도 그대로 쓰인다. 래퍼가 주입하는 시작 명령이 `--heartbeat N` 을 포함하고, 브리프에 계약(N/K/T)이 명시된다.
|
||||||
2. **무응답 판정 = 마지막 이벤트로부터 K×N (`registry.py`, 래퍼 `finalize_unfinished`)**: 레지스트리 레코드에 `heartbeat_sec`·`heartbeat_misses`·`ack_timeout_sec`·`transport` 가 추가되고 `idle_timeout_sec` 은 명시하지 않으면 K×N 으로 유도된다. 래퍼의 R1 유예는 events.log 의 최근 이벤트(K×N 이내)만을 증거로 삼고, herdr 상태·pane busy 토큰은 진입 시 한 번 `liveness: ... (diagnostic only)` 로 기록만 한다. 헤드리스 에이전트(항상 idle, 빈 pane)도 heartbeat 만으로 K×N 을 넘겨 살아 있고, heartbeat 가 끊기면 `deadline_timeout` 으로 닫히며 이후 `completed` 는 CAS 가 거부한다.
|
2. **무응답 판정 = 마지막 이벤트로부터 K×N (`registry.py`, 래퍼 `finalize_unfinished`)**: 레지스트리 레코드에 `heartbeat_sec`·`heartbeat_misses`·`ack_timeout_sec`·`transport` 가 추가되고 `idle_timeout_sec` 은 명시하지 않으면 K×N 으로 유도된다. 래퍼의 R1 유예는 events.log 의 최근 이벤트(K×N 이내)만을 증거로 삼고, herdr 상태·pane busy 토큰은 진입 시 한 번 `liveness: ... (diagnostic only)` 로 기록만 한다. 헤드리스 에이전트(항상 idle, 빈 pane)도 heartbeat 만으로 K×N 을 넘겨 살아 있고, heartbeat 가 끊기면 `deadline_timeout` 으로 닫히며 이후 `completed` 는 CAS 가 거부한다.
|
||||||
3. **전달 확인 = T 내 `started` (`_wait_started`, `lib.sh: _sks_job_started`)**: 주입 후 T초(`--ack-timeout`, 기본 60; 벽시계 상한) 안에 레지스트리가 `pending` 을 벗어나거나 events.log(이번 attempt 의 dispatch_token 을 가진 `started` 만)/구독자 로그에 `started` 가 없으면 rc 1·`agent_blocked` 로 fail-closed 하고 재주입하지 않는다. `send_keys_safe` 는 accepted-but-unconfirmed 제출을 `agent_status working` 또는 job 의 `started` 로만 확인하며 pane 변화·busy 토큰 신호는 제거됐다(무신호 rc 4 는 유지). `MAM_INJECT_ACK_GRACE_S` 는 T 의 override 로 남는다.
|
3. **전달 확인 = T 내 `started` (`_wait_started`, `lib.sh: _sks_job_started`)**: 주입 후 T초(`--ack-timeout`, 기본 180 — 실제 에이전트는 브리프를 읽고 나서 시작 명령을 실행하므로 여유 있게; 벽시계 상한) 안에 레지스트리가 `pending` 을 벗어나거나 events.log(이번 attempt 의 dispatch_token 을 가진 `started` 만)/구독자 로그에 `started` 가 없으면 rc 1·`agent_blocked` 로 fail-closed 하고 재주입하지 않는다. `send_keys_safe` 는 accepted-but-unconfirmed 제출을 `agent_status working` 또는 job 의 `started` 로만 확인하며 pane 변화·busy 토큰 신호는 제거됐다(무신호 rc 4 는 유지). `MAM_INJECT_ACK_GRACE_S` 는 T 의 override 로 남는다.
|
||||||
4. **와이어 내구성 (`publish_event.py: build_state_payload`, `job_subscriber.py`)**: 모든 이벤트 뒤에 같은 연결로 retained `<prefix>/state`(status·last_event·seq, HMAC 서명)를 발행한다. 구독자는 state 토픽도 구독해 (재)접속 즉시 터미널 상태를 복구하고, 모든 메시지(progress·state 포함)로 idle 창을 리셋하며, seq 점프를 감사 로그 `seq_gap` 으로 남기고, 재접속을 `reconnected` 로 기록한다. `--no-state` 로 끌 수 있다.
|
4. **와이어 내구성 (`publish_event.py: build_state_payload`, `job_subscriber.py`)**: 모든 이벤트 뒤에 같은 연결로 retained `<prefix>/state`(status·last_event·seq, HMAC 서명)를 발행한다. 구독자는 state 토픽도 구독해 (재)접속 즉시 터미널 상태를 복구하고, 모든 메시지(progress·state 포함)로 idle 창을 리셋하며, seq 점프를 감사 로그 `seq_gap` 으로 남기고, 재접속을 `reconnected` 로 기록한다. `--no-state` 로 끌 수 있다.
|
||||||
5. **원격 전송 (`lib_py/delegation/remote.py`, `scripts/remote.py`, 래퍼 `remote:<name>`)**: `--agent-session remote:<name>` 이면 preflight·idle 핸드셰이크·pane 주입을 건너뛰고 `MAM_DISPATCH_SECRET` 로 서명한 retained `<prefix>/dispatch`(브리프·auth_token·dispatch_token·attempt_id·계약)를 발행한 뒤 T 내 `started` 를 기다린다. 다른 호스트의 `remote.py run --session <name> [--exec CMD] [--once]` 러너가 자기 레지스트리에 같은 id·identity 로 잡을 미러링하고, dispatch 를 retire 하고, `started --heartbeat N` 후 명령을 실행해 터미널 이벤트를 발행한다. 구독자의 `--remote`(transport=remote 잡은 자동)가 `started`·터미널을 오케스트레이터 레지스트리에 반영한다. 서명 없는 dispatch 는 거부된다. 루프 러너는 YAML 행의 `transport: remote` 를 `remote:<name>` 으로 넘기고, `MAM_HEARTBEAT_SEC`/`MAM_HEARTBEAT_MISSES`/`MAM_ACK_TIMEOUT_S` 를 계약 옵션으로 전달한다.
|
5. **원격 전송 (`lib_py/delegation/remote.py`, `scripts/remote.py`, 래퍼 `remote:<name>`)**: `--agent-session remote:<name>` 이면 preflight·idle 핸드셰이크·pane 주입을 건너뛰고 `MAM_DISPATCH_SECRET` 로 서명한 retained `<prefix>/dispatch`(브리프·auth_token·dispatch_token·attempt_id·계약)를 발행한 뒤 T 내 `started` 를 기다린다. 다른 호스트의 `remote.py run --session <name> [--exec CMD] [--once]` 러너가 자기 레지스트리에 같은 id·identity 로 잡을 미러링하고, dispatch 를 retire 하고, `started --heartbeat N` 후 명령을 실행해 터미널 이벤트를 발행한다. 구독자의 `--remote`(transport=remote 잡은 자동)가 `started`·터미널을 오케스트레이터 레지스트리에 반영한다. 서명 없는 dispatch 는 거부된다. 루프 러너는 YAML 행의 `transport: remote` 를 `remote:<name>` 으로 넘기고, `MAM_HEARTBEAT_SEC`/`MAM_HEARTBEAT_MISSES`/`MAM_ACK_TIMEOUT_S` 를 계약 옵션으로 전달한다.
|
||||||
6. **문서**: MESSAGING.md §2.5 계약 표·원격 전송, delegate-job SKILL.md(옵션·종료 코드·프로토콜), registry.md(계약 필드), loop SKILL.md(환경변수·원격 행).
|
6. **문서**: MESSAGING.md §2.5 계약 표·원격 전송, delegate-job SKILL.md(옵션·종료 코드·프로토콜), registry.md(계약 필드), loop SKILL.md(환경변수·원격 행).
|
||||||
|
|||||||
@@ -133,7 +133,7 @@ def test_register_derives_the_idle_window_from_the_contract(tmp_path, monkeypatc
|
|||||||
reg = tmp_path / "jobs"
|
reg = tmp_path / "jobs"
|
||||||
jid = registry.register_job("contract", registry_dir=str(reg))
|
jid = registry.register_job("contract", registry_dir=str(reg))
|
||||||
rec = _job(reg, jid)
|
rec = _job(reg, jid)
|
||||||
assert (rec["heartbeat_sec"], rec["heartbeat_misses"], rec["ack_timeout_sec"]) == (30, 4, 60)
|
assert (rec["heartbeat_sec"], rec["heartbeat_misses"], rec["ack_timeout_sec"]) == (30, 4, 180)
|
||||||
assert rec["idle_timeout_sec"] == 120 == registry.liveness_window(30, 4)
|
assert rec["idle_timeout_sec"] == 120 == registry.liveness_window(30, 4)
|
||||||
assert rec["transport"] == "herdr"
|
assert rec["transport"] == "herdr"
|
||||||
|
|
||||||
|
|||||||
@@ -343,7 +343,7 @@ def test_registry_cli_contract_snapshot(mam_sandbox):
|
|||||||
assert reg["--idle-timeout"][2] is None
|
assert reg["--idle-timeout"][2] is None
|
||||||
assert reg["--heartbeat-sec"][2] == 30
|
assert reg["--heartbeat-sec"][2] == 30
|
||||||
assert reg["--heartbeat-misses"][2] == 4
|
assert reg["--heartbeat-misses"][2] == 4
|
||||||
assert reg["--ack-timeout"][2] == 60
|
assert reg["--ack-timeout"][2] == 180
|
||||||
assert reg["--transport"][1] == ("herdr", "remote")
|
assert reg["--transport"][1] == ("herdr", "remote")
|
||||||
assert registry.liveness_window(reg["--heartbeat-sec"][2], reg["--heartbeat-misses"][2]) == 120
|
assert registry.liveness_window(reg["--heartbeat-sec"][2], reg["--heartbeat-misses"][2]) == 120
|
||||||
assert reg["--bits"][2] == 32
|
assert reg["--bits"][2] == 32
|
||||||
|
|||||||
Reference in new issue
Block a user