Phase 3: VISUAL feed and M-Files drawings in the worker

Migration 0004 (VISUAL and drawing caches, go-live ignored list, resource map, job tables, order-derived columns with provenance; work order key includes the operation sequence). The real worker on APScheduler with settings re-read each fire, pause, run now, advisory locks, job_runs and a heartbeat. The visual-feed job (five VisualKit reads, COPY and MERGE cache, suspect-batch guard, the apply in one transaction: drafts, change orders, retire-moved, work order gone, closed and cancelled observed, projects and units from order links, auto-release) and the fingerprint. The drawing-lookup job (queue, budget, download on version change, sha256 into assets, decision table, staging for rev approval, drawing-origin change orders). Clerical From VISUAL panel, flags, rev approval, hand uploads, change-order review; Admin Feed settings and resource map. Synthetic fixture generator and a fixture-driven worker in CI with a Playwright scenario. Docs updated. Verified: 3249 server tests and 13 Playwright scenarios on the docker-1 CI stack, 76 frontend tests, doctor at 0004; one feed and one drawing run against VISUAL DEV and the vault from the developer machine (counts in the changelog). Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-16 14:45:38 -06:00
parent 16950055e3
commit 591fe90076
103 changed files with 16063 additions and 151 deletions
@@ -0,0 +1,63 @@
+"""The background worker: the job scheduler plus a heartbeat (beat file and worker_heartbeat row).
+
+`run(beat_file, build, stop=None, interval=BEAT_SECONDS, settings=None)` blocks until SIGTERM, SIGINT or `stop`.
+With settings it builds and starts a Scheduler; without (the CLI tests) it only beats the file.
+"""
+
+from __future__ import annotations
+
+import logging
+import signal
+import threading
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+from .heartbeat import BEAT_SECONDS, STALE_SECONDS, beat_age, touch
+
+if TYPE_CHECKING:
+    from ..settings import Settings
+
+log = logging.getLogger("packetstation.worker")
+
+__all__ = ["BEAT_SECONDS", "STALE_SECONDS", "beat_age", "run", "touch"]
+
+
+def run(
+    beat_file: Path,
+    build: str,
+    stop: threading.Event | None = None,
+    interval: float = BEAT_SECONDS,
+    settings: Settings | None = None,
+) -> int:
+    stop = stop or threading.Event()
+
+    def _stop(signum: int, _frame: object) -> None:
+        log.info("worker stopping on signal %s", signum)
+        stop.set()
+
+    if threading.current_thread() is threading.main_thread():
+        signal.signal(signal.SIGTERM, _stop)
+        signal.signal(signal.SIGINT, _stop)
+    log.info("worker started", extra={"ps": {"build": build, "beat_file": str(beat_file)}})
+    sched = None
+    if settings is not None:
+        from .scheduler import Scheduler
+
+        sched = Scheduler(settings, build=build)
+        sched.start()
+    try:
+        if sched is not None:
+            sched.serve(beat_file, stop, interval)
+        else:
+            while not stop.is_set():
+                try:
+                    touch(beat_file)
+                except OSError as e:
+                    log.error("can't touch beat file %s: %s", beat_file, e.strerror)
+                log.info("heartbeat", extra={"ps": {"build": build}})
+                stop.wait(interval)
+    finally:
+        if sched is not None:
+            sched.shutdown(wait=True)
+    log.info("worker stopped")
+    return 0