feat(issue-5): auto-completion gate and durable pending notification
The gate does not re-decide anything. Phase 2a already decides, fail-closed and against contract-pinned vectors; this supplies real evidence to that decision and commits only if three independent gates agree: the global switch is on, the backend has observed evidence that it reaches a quiescent state, and the job itself opted in. Default is off. The backend allowlist is evidence-based rather than optimistic. Inferring "finished" from stillness requires stillness to have been observed, and only three of six backends have that. An operator may narrow the list; a name that was never observed is dropped rather than trusted, so it cannot be widened by configuration. Everything else holds. An unreadable manifest, a report rewritten after finalization, missing evidence for a declared side effect, a policy that raises: all refuse. "Could not decide" is never "done". A late explicit terminal wins, because the boundary refuses the inferred one -- that is the boundary working, not an error in the gate. Notification intent is committed in the same atomic write as the state, so a crash between committing and publishing leaves a record that still says the world has not been told. Recovery re-sends; duplicates are already dropped by sequence. A failed send deliberately leaves the intent pending, because an unsent notification that looks sent is worse than one that is retried. Two fixes this surfaced. Settling used a request that terminal records refuse, which made an owed notification impossible to ever settle; settling is now a status-free bookkeeping request, valid in any state. And the phase 2a guard against shell references was matching hashlib.sha256 as a substring, so it now requires a word boundary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
This commit is contained in:
1 parent
3ea5757cfe
commit
803ba43884
5 files changed
+513
-4
No files matched your search
@@ -0,0 +1,97 @@
|
||||
"""Durable notification intent, carried in the job record.
|
||||
|
||||
The contract forbids a separate outbox store, so the intent rides in the record
|
||||
the transition already writes: one atomic commit records both the new state and
|
||||
the fact that the world has not been told yet. Publishing then happens outside
|
||||
the registry lock, and settling the intent is itself an ordinary transition.
|
||||
|
||||
That ordering is the point. If the process dies between the commit and the
|
||||
publish, the record still says a notification is owed, and startup recovery
|
||||
re-sends it. Duplicates are safe: the subscriber already drops any event whose
|
||||
sequence is not greater than the highest it has accepted.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from lib_py.delegation import mqtt_common
|
||||
|
||||
logger = logging.getLogger("delegate_job.notifier")
|
||||
|
||||
FIELD = "pending_notification"
|
||||
|
||||
|
||||
def intent_for(event: str, seq: int, timestamp: str, detail: str = "") -> Dict[str, Any]:
|
||||
return {"event": event, "seq": seq, "timestamp": timestamp, "detail": detail}
|
||||
|
||||
|
||||
def pending(job_id: str, registry_dir: str) -> Optional[Dict[str, Any]]:
|
||||
try:
|
||||
return mqtt_common.load_job(job_id, registry_dir).get(FIELD)
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
|
||||
|
||||
def jobs_with_pending(registry_dir: str) -> List[str]:
|
||||
"""Records still owing a notification. The input to restart recovery."""
|
||||
from lib_py.delegation import registry # local: keeps module import order flat
|
||||
|
||||
out = []
|
||||
for record in registry.list_jobs(registry_dir):
|
||||
if record.get(FIELD):
|
||||
out.append(record["job_id"])
|
||||
return out
|
||||
|
||||
|
||||
def settle(job_id: str, registry_dir: str, source: str = "notifier") -> bool:
|
||||
"""Clear the intent once the event is on the wire.
|
||||
|
||||
Goes through the transition boundary like every other record write, using a
|
||||
request that carries no status: settling is about a terminal, so a request
|
||||
the terminal state would refuse could never be used to settle one.
|
||||
"""
|
||||
from lib_py.delegation import engine
|
||||
|
||||
result = engine.transition(job_id, "notification_settled", registry_dir=registry_dir,
|
||||
fields={FIELD: None}, source=source)
|
||||
return result.committed
|
||||
|
||||
|
||||
def flush(job_id: str, registry_dir: str, publish=None, attempts: int = 3) -> bool:
|
||||
"""Publish an owed notification and settle it. Safe to call repeatedly.
|
||||
|
||||
Returns True when nothing is owed afterwards. A failure leaves the intent in
|
||||
place on purpose: an unsent notification that looks sent is worse than one
|
||||
that is retried.
|
||||
"""
|
||||
intent = pending(job_id, registry_dir)
|
||||
if not intent:
|
||||
return True
|
||||
|
||||
if publish is None:
|
||||
from lib_py.delegation import publish_event as _pe
|
||||
|
||||
def publish(job, intent_, registry_dir_):
|
||||
return _pe.main(["--registry-dir", registry_dir_, "--job", job,
|
||||
"--event", intent_["event"], "--detail",
|
||||
intent_.get("detail", ""), "--attempts", str(attempts)])
|
||||
|
||||
try:
|
||||
rc = publish(job_id, intent, registry_dir)
|
||||
except Exception as exc:
|
||||
logger.warning("notification publish failed for %s: %s", job_id, exc)
|
||||
return False
|
||||
if rc != 0:
|
||||
logger.warning("notification for %s not accepted (rc=%s); leaving it pending",
|
||||
job_id, rc)
|
||||
return False
|
||||
return settle(job_id, registry_dir)
|
||||
|
||||
|
||||
def recover(registry_dir: str, publish=None) -> Dict[str, bool]:
|
||||
"""Re-send every owed notification. Called at startup after a crash."""
|
||||
results = {}
|
||||
for job_id in jobs_with_pending(registry_dir):
|
||||
results[job_id] = flush(job_id, registry_dir, publish=publish)
|
||||
return results
|
||||
Reference in new issue
Block a user