Files
multi-agent-mux/.agents/skills/multi-agent-mux-delegate-job/SKILL.md
T
Godopu ec61511699 fix(issue-6): fail-closed registry timeout and TUI turn handshake
- Introduce finalize_unfinished in multi-agent-mux-delegate-job for fail-closed atomic transitions
- Enforce CAS dispatch_token and attempt_id pinning across engine, CLI, and lifecycle scripts
- Integrate Herdr wait_for_agent_idle turn handshake to prevent prompt loss and race conditions
- Resolve bash 3.2 set -u empty array expansion compatibility in lib.sh
- Preserve Issue #5 R2 contract in CAS transition engine (engine.py, publish_event.py, service.py)
- Add comprehensive regression suites (test_issue6_fail_closed.py, test_issue6_turn_handshake.py)
- Bump lockstep framework and skills to v4.4.1 across lib.sh, eager.py, VERSIONS.md, 8 SKILL.md
- Synchronize documentation (IMPROVEMENTS.md, MESSAGING.md, state-transitions.md, task.md, phase5-e2e.md)
- Unanimous PASS achieved across 4 reviewer team leaders (Claude, Codex, OpenCode, Grok)

Closes #6
2026-09-09 14:02:47 +09:00

9.6 KiB

name, description, version, author, license, platforms, environments, metadata
name description version author license platforms environments metadata
multi-agent-mux-delegate-job Delegate a unit of work to any autonomous agent (claude-code, hermes, agy, grok-build, codex, or a human) and observe it asynchronously over an MQTT event channel. Supported roles include orchestrator, worker, and reviewer. 4.4.1 godopu MIT
linux
macos
windows
terminal
herdr
hermes
tags related_skills prereq_skills
agent
herdr
multi-agent
delegate
mqtt
async
job
multi-agent-mux-create
multi-agent-mux-resume
multi-agent-mux-loop
multi-agent-mux-create

multi-agent-mux-delegate-job — Async Job Delegation over MQTT

Delegate a unit of work to any autonomous agent, then observe it asynchronously instead of blocking. Every job gets a unique ID and a registry record. The worker agent publishes lifecycle events (started, permission_required, progress, completed, error) to a per-job MQTT topic, and the delegator/orchestrator subscribes to verify the final state.

This skill allows any agent (claude-code, hermes, agy, grok-build, etc.) to play any role: Orchestrator/Delegator, Worker/Implementer, or Reviewer.


Roles in Multi-Agent Mux

  • Orchestrator (Delegator): Initiates the job, coordinates other agents, handles loops and reviews, and commits final changes.
  • Worker (Implementer): Receives the brief file or task prompt, performs the implementation, and emits started/completed/error events.
  • Reviewer: Evaluates git diffs or artifacts produced by the worker, and responds with a completed event containing "PASS" or feedback.

Core Commands (CLI)

The multi-agent-mux-delegate-job bash wrapper handles job registration, subscriber management, agent session targeting, and validation hooks:

# 1) Submit a new job to a targeted agent session (e.g. herdr session name 'demo')
multi-agent-mux-delegate-job submit \
  --agent <claude-code|hermes-agent|agy-agent|grok-build|human> \
  --agent-session herdr:<session_name> \
  --prompt "Task description or instructions here" \
  --role <Worker|Planner|Reviewer> \
  --timeout 3600 --idle-timeout 120

# 2) Submit a job with a feedback loop (Worker-Reviewer Loop)
multi-agent-mux-delegate-job submit \
  --agent <worker_agent> --agent-session herdr:<worker_session> \
  --type loop --reviewer <reviewer_agent> --reviewer-session herdr:<reviewer_session> \
  --prompt "Task description"

# 3) Check job status and audit logs
multi-agent-mux-delegate-job status --job <JOB_ID>
multi-agent-mux-delegate-job logs   <JOB_ID>            # Chronological log of events
multi-agent-mux-delegate-job list                       # Summary of all registered jobs

# 4) Verify job artifacts with a validation script
multi-agent-mux-delegate-job verify --job <JOB_ID> --validate ./validate.sh

Task Delegation Types

Supported job types include:

  • direct (default): Single agent execution (direct tasking).
  • loop (Worker-Reviewer Loop): Alternates worker execution and reviewer evaluation until reviewer approves (PASS) or iterations run out.
  • discuss (Research & Discussion): Collaboration between two agents to reach a consensus (e.g., agreeing on a design or plan).

For detailed state machine diagrams and configurations, see DELEGATION_TYPES.md.


Exit Codes & Failure Handling

Under fail-closed operation (MAM_DELEGATE_FAIL_CLOSED=1, default), the CLI returns deterministic exit codes:

Exit Code Meaning Final Registry Status Failure Behavior & Resilience
0 Success completed Job succeeded and completed event received or verified on disk with approval (PASS/AGREE).
1 Failure error Execution failure / Agent injection error (inject_agent).
2 Timeout error (deadline_timeout) Subscriber timeout (total budget or idle timeout reached without terminal event). R1 Working-Agent Resilience: If the agent in Herdr is still actively working, the wrapper does not immediately fail-close; it re-observes disk status at intervals up to the wall-clock --timeout deadline before fail-closing.
3 Broker Error error (deadline_timeout) Broker unavailable (MQTT_BROKER_UNAVAILABLE) and terminal status not found on disk during deadline.
5 Session Busy cancelled (session_busy) Precondition failed / session_busy: Preflight check (preflight_agent) or TUI idle handshake wait failed (prompt never delivered, retryable).

In all execution paths, the final line of standard output prints the persistent audit log directory path (<logs_root>/<job_id>), fulfilling the parsing contract for orchestrator runners.


Where the Implementation Lives (v4.3.0)

The implementation moved to the lib_py.delegation package in issue #5 phase 1. scripts/{mqtt_common,registry,publish_event,job_subscriber}.py remain as compatibility shims and their paths are unchanged -- every command in this document still works verbatim. Each shim binds the lib_py.delegation package to the tree next to itself, once per process, without touching sys.path, so import mqtt_common yields the same module object as lib_py.delegation.mqtt_common.

Which tree wins, by execution context:

Context Resolved by Tree that wins
Loop process and its children (frozen lib.sh sourced) the helper's explicit bootstrap from SKILL_DIR the frozen snapshot
Agent session running a frozen script path while PYTHONPATH still points live the shim, from its own location the frozen snapshot (B-13 preserved)
Live helpers (create/stop/resume/reconcile) PYTHONPATH or the shim (same tree) the live tree
Standalone, no PYTHONPATH the shim's fallback append the shim's own tree

lib.sh calls the package through _delegation_py, which inserts its own SKILL_DIR at the front of the import path. It deliberately does not use python -m: for -m the current directory precedes PYTHONPATH, so a frozen loop whose working directory happened to contain a lib_py package would have silently executed live code. The working directory is left untouched, because the registry and audit-log paths are relative to it.

Setting MAM_DELEGATION_PY=0 switches the shell helpers back to their preserved legacy bodies with no redeploy.

One in-process caveat: the package binds once per process and is never rebound, so do not import a frozen shim after a live lib_py.delegation is already bound inside the same interpreter. Production never does this — each delegated job is a fresh interpreter — but a test harness could.


The Event Protocol Contract

Every agent participating in the delegation contract must follow the same lifecycle publishing protocol using publish_event.py:

  1. On Start: Publish started event. python3 .agents/skills/multi-agent-mux-delegate-job/scripts/publish_event.py --job "$JOB_ID" --event started
  2. On Tool/Permission Prompt: Publish permission_required event. python3 ... --job "$JOB_ID" --event permission_required --detail "<tool>:<reason>"
  3. On Progress Update (Optional): Publish progress event. python3 ... --job "$JOB_ID" --event progress --detail "<status_update>"
  4. On Success: Publish completed event. python3 ... --job "$JOB_ID" --event completed --detail "<summary>" (Reviewer should include "PASS" in the detail to approve).
  5. On Failure/Feedback: Publish error event. python3 ... --job "$JOB_ID" --event error --detail "<reason_or_feedback>"

Audit Logs

Job lifecycle execution events are persistently mirrored to an append-only log under .mam/delegate_job_logs/<job_id>/ (containing meta.json, events.ndjson, and status.json). Use multi-agent-mux-delegate-job logs <job_id> to view the timeline.


Best Practices and Pitfalls

  • Subscribe-Before-Publish: The subscriber must be running before the agent starts publishing. The submit command handles this automatically by launching the subscriber in the background first.
  • Fresh job_id Propagation: Make sure the worker agent receives the correct JOB_ID generated for the current run, rather than reusing stale IDs from previous sessions.
  • Brief delivery via file path: For long or complex prompts, write the instructions to a file (e.g. /tmp/task-brief.md) and pass a short prompt pointing to the file path to prevent terminal buffer overflows.
  • Prompts injected into a live agent session MUST be English, ASCII-only, and short — this is exactly what --prompt/the instructions string sent to run_agent() end up as. Any Korean (or other non-ASCII) content the task needs to convey must go in a markdown brief file (e.g. .mam/jobs/<id>/brief.md, written in Korean is fine) that the injected prompt merely tells the agent to read. Two independent bugs in send_keys_safe's paste-verification (in lib.sh) made this matter in practice: (a) its marker was taken with a byte-based tail -c 24, which can slice a multi-byte UTF-8 (e.g. Korean) character in half; (b) the rendered pane soft-wraps long lines at the terminal width, which can split the marker across two visual lines. Both are now fixed at the source (character-safe truncation + newline-stripped matching before comparison), but keeping injected prompts short/English/file-referencing is still the cheapest way to avoid ever exercising this edge case at all — it's also simply what submit's own default instruction template already does (see Core Commands above).
  • Batch Grouping: Group non-overlapping tasks into batches to parallelize execution across multiple agent sessions, reducing overhead.