Phase 9: Admin completion, reconcile, notifications, integrity, backups

Admin: Departments (name and stations; add shows the three groups IT must
create), Profiles (welder stamp, station limits, active) with a CSV import
that previews and applies only when clean, Checklists (weld and plain),
Security (session, lockout and rate-limit settings with bounds, alerts for
the last 24 hours, the integrity result), Access log with filters and a
guarded CSV, Who is in, Stamp check, Notifications, and the employeeID
flags. Every Admin change is audited.

Worker: ad-reconcile (nightly and on "refresh from AD now"), session-refresh
(15 minutes), integrity (seals, asset hashes, missing and orphan files, the
audit chain), backup-check, notify-send and notify-digest. Notifications
route per event type to AD groups, users, mailboxes or the department
Supervisor group, immediately or in a daily digest, over STARTTLS verified
against the PESCO CA, with dry run on the test stack.

Backups: scripts/backup/ps-backup.sh and ps-restore.sh with a systemd timer
at 02:00 America/Denver. Restore rehearsed on the test stack 2026-10-21.
CI: pip-audit and npm audit. pypdf 6.21.1 and certifi 2026.10.19.

Migration 0010. Closes the app part of I6, M3, O1, O2 and S3.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

docs/SCHEMA.md

@@ -1,6 +1,6 @@
 # Packet Station schema
 
-Status: phases 1 to 4 built (identity, organisation, projects, packets, change orders, documents, jobs, VISUAL and drawing caches, the warehouse columns in use, the `notification_events` placeholder). Phase 5 adds the shop columns, the checklist templates and the clock rules (migration 0006). Phase 6 adds the Q.C. columns, the Q.C. check items, the Q.C. folder metadata, the ITP header, line and block details and the ITP log (migration 0007). Phase 7 adds the print log and the data book on the seal (migration 0008). Phase 8 adds no table and no migration: the PM report exports use new `print_log` kinds and presence uses two new session events. Phase 9 adds the notification rules and delivery. The DDL is in `server/packetstation/migrations/`; this file explains it. See [PLAN.md](PLAN.md), "Schema".
+Status: phases 1 to 4 built (identity, organisation, projects, packets, change orders, documents, jobs, VISUAL and drawing caches, the warehouse columns in use, the `notification_events` placeholder). Phase 5 adds the shop columns, the checklist templates and the clock rules (migration 0006). Phase 6 adds the Q.C. columns, the Q.C. check items, the Q.C. folder metadata, the ITP header, line and block details and the ITP log (migration 0007). Phase 7 adds the print log and the data book on the seal (migration 0008). Phase 8 adds no table and no migration: the PM report exports use new `print_log` kinds and presence uses two new session events. Phase 9 adds the notification rules and delivery, the integrity findings and the Admin profile switch (migration 0010). The DDL is in `server/packetstation/migrations/`; this file explains it. See [PLAN.md](PLAN.md), "Schema".
 
 ## Conventions
 
@@ -339,7 +339,7 @@ A unit folder shows its own current rows and inherits the project's current rows
 
 These tables have the `ps_append_only` row and TRUNCATE triggers, and `ps_app` holds only SELECT and INSERT on them:
 
-`audit_events`, `signin_attempts`, `signatures`, `project_renumbers`, `short_receipts`, `packet_transitions`, `qc_history`, `packet_log`, `itp_rejects`, `itp_log` (0007), `print_log` (0008), `co_verifications`.
+`audit_events`, `signin_attempts`, `signatures`, `project_renumbers`, `short_receipts`, `packet_transitions`, `qc_history`, `packet_log`, `itp_rejects`, `itp_log` (0007), `print_log` (0008), `notifications_sent` and `integrity_findings` (0010), `co_verifications`.
 
 The app role is refused by privilege first. The migrate role owns the tables, so the trigger refuses it. The views `v_units` and `v_open_cos` are SELECT only for `ps_app`.
 
@@ -352,7 +352,7 @@ Phase 3, migration 0004:
 
 - `job_requests`: one row per "Run now". `job`, `requested_at`, `requested_by`, `note`, `claimed_run_id` and `claimed_at` (set by the run that took it). Index on `job` where `claimed_at` is null. The app inserts the row and NOTIFYs `ps_job_request` in the same transaction.
 - `job_runs`: one row per run. `job`, `trigger` (schedule, request, chain, cli), `requested_by` uuid[] (the requests claimed), `started_at`, `finished_at`, `status` (running, ok, degraded, failed, skipped, abandoned), `counts` jsonb, `errors` jsonb (a list of redacted strings), `build_version`, `visual_target` (`"<server> / <database>"`), `host`. Index on (`job`, `id DESC`).
-- `job_schedule`: one row per job (`job` PK). `next_run_at`, `interval_seconds`, `paused`, `updated_at`. Written by the worker after each run for the Admin tab. Seeded for `visual-feed`, `drawing-lookup` and `housekeeping`.
+- `job_schedule`: one row per job (`job` PK). `next_run_at`, `interval_seconds`, `paused`, `updated_at`. Written by the worker after each run for the Admin tab. Seeded for `visual-feed`, `drawing-lookup` and `housekeeping`; 0010 adds `ad-reconcile`, `session-refresh`, `integrity`, `backup-check`, `notify-send` and `notify-digest`.
 - `worker_heartbeat`: a single row (`id` = 1 by CHECK). `at`, `build`, `host`, `pid`, `running` text[] (the jobs running now). Upserted every 30 seconds.
 
 ## Notifications (phases 4 and 9)
@@ -361,7 +361,22 @@ Phase 4, migration 0005:
 
 - `notification_events`: one row per alert. `id` bigserial, `kind` (text, for example `assets_quota`), `severity` (`info`, `warning` or `critical` by CHECK, default `warning`), `subject` (what the alert is about, for example `assets`), `detail` jsonb (for the quota alert, the volume usage: `total`, `used`, `free`, `pct`, `warn`), `created_at`, `resolved_at` (set when the condition clears), `acknowledged_at` and `acknowledged_by` (an Admin saw it). The unique index `notification_events_open` on (`kind`, `coalesce(subject, '')`) where `resolved_at` is null keeps one open alert per kind and subject, so a check that runs again does not stack rows. The table is not append-only: the app sets `resolved_at` and the acknowledgement. 0005 also grants `ps_app` USAGE and SELECT on every sequence in the schema.
 
-Phase 9 builds the delivery on this table: rules, recipients, exclusions, outbox, sent log, state.
+Phase 9, migration 0010, builds the delivery on this table:
+
+- `notification_events` gains `event_type` (the catalog key; `kind` holds the same text), `title` (the mail subject after the prefix), `body`, `dept_code` (for the department events, so the department's Supervisor group can be the recipient), `raised_count` and `last_raised_at` (a condition raised again while open only counts), `recovery_of` (the row a "Cleared" notice follows), `dispatched_at` and `dispatch_result` (set by `notify-send`: sent, dry_run, failed, no_recipients, queued for the digest, rule_off, or no_rule; `directory_retry:N` marks an event left pending while AD was down, and the third failure dispatches it failed). The phase 4 quota rows were marked dispatched with `before_phase_9`, so they are never mailed late. A standing condition (feed failed, backup stale, quota, integrity failure, group missing, drawing failing, change order waiting) stays open until it clears, and the open index is the dedupe. A one-shot event (security alert, sign-in burst, reject, reconcile report, version deployed) is inserted already resolved. The plan's separate `notification_state` table is not needed: the open row is the state.
+- `notification_rules`: one row per event type (`event_type` PK). `enabled`, `delivery` (enum `notify_delivery`: immediate or digest), `digest_time` (site time, default 07:00), `param` (N hours for drawing_failing, N days for co_waiting), `version` (compare and set for the Admin tab), `updated_at`, `updated_by`. The code seeds a missing rule enabled, immediate, with PS-Admins as its recipient, and never changes a rule that exists.
+- `notification_recipients`: `event_type`, `kind` (enum `notify_recipient_kind`: group, user, mailbox, dept_supervisor), `group_dn` and `group_name` (any AD group under LDAP_BASE_DN), `user_id`, `address`. CHECKs make each kind carry its own column. Groups are resolved to their members' mail when a message goes, not stored.
+- `notification_exclusions`: (`event_type`, `user_id`) PK, `self` (set by the person on Account settings; never allowed on the security event types), `created_at`, `created_by`.
+- `notification_outbox`: events waiting for their rule's digest. `event_id`, `event_type`, `queued_at`, `sent_at`, `sent_id` (the `notifications_sent` row).
+- `notifications_sent`: append-only. One row per message: `at`, `event_type` (`digest` for a digest), `event_ids`, `digest`, `test`, `subject`, `recipients` jsonb (`[{address, user_id, why}]`, as resolved), `result` (sent, dry_run, failed, no_recipients), `error`, `requested_by` (for a test send).
+
+## Admin and integrity (phase 9, migration 0010)
+
+- `users.profile_active`: the Admin's switch on a person's profile, default true. `users.active` stays the reconcile's (disabled or gone in AD). An inactive profile drops out of the welder picker and the worker filter. It does not change what the person may do.
+- `departments.created_at` and `created_by`: set when an Admin adds a department in the app. Seeded departments have neither. `code` and `slug` stay immutable by trigger.
+- `assets.hash_checked_at` and `hash_ok`: when the integrity job last rehashed the file and whether it matched `sha256`.
+- `integrity_findings`: append-only. One row per thing the integrity job found wrong: `run_id` (no foreign key, since housekeeping prunes `job_runs`), `at`, `kind` (seal, asset_hash, asset_missing, blob_orphan, audit_chain), `subject` (the packet, asset, file path or audit id), `detail` jsonb.
+- Settings rows used by phase 9: `security` (the Admin's overrides of the session, lockout and rate-limit settings; missing keys fall back to the environment), `checklist_templates` (now edited on the Admin Checklists tab), `notify.last_build` (the build the worker last announced).
 
 ## VISUAL and drawing caches (phase 3)