# Packet Station feed and drawings

Status: phase 9. This file says how the worker reads VISUAL and M-Files, what it does with what it reads, and the other jobs it runs (phase 9: the AD reconcile, the role refresh, the integrity check, the backup check and the notifications). The design is in [PLAN.md](PLAN.md), "Worker, feed and drawings". When this file and the code disagree, the code is the rule and this file is out of date.

## Purpose

The worker is the only part of Packet Station that talks to VISUAL and M-Files. It reads VISUAL through VisualKit and M-Files through MFilesKit, both read-only, on a schedule. It stores what it read in cache tables, turns the rows into projects, units, draft packets and change orders, and works the drawing queue. Screens read PostgreSQL only.

There are three reasons for a scheduled job instead of reads per screen:

- VISUAL production must not be queried on page loads. One modest query every few minutes from one place is fine. One per user per screen is not.
- Turning VISUAL rows into packets is a state change. It runs in one place, in one transaction, with an audit row.
- Drawing lookups and downloads are slow. They belong in the background.

The worker never writes to VISUAL or M-Files.

## Worker process

`packetstation worker` starts the worker. It runs the stack guard, builds the scheduler and blocks. The main thread writes the heartbeat and waits for a stop signal.

### Scheduler

- APScheduler 3 with a pool of 4 threads. Each job has `coalesce` on, `max_instances` 1 and a misfire grace of 300 seconds.
- Times are scheduled in site time (`SITE_TZ`, default `America/Denver`) and stored in UTC.
- Jobs are registered in `worker/jobs/__init__.py`. Each job has a name, a run function, a cadence function and a `pausable` flag. A job can name others to chain after it: `visual-feed` chains `drawing-lookup`, so every feed run is followed by a drawing run.
- The cadence function returns the seconds to the next run, from the feed settings and the current site time.

### Settings re-read

The feed settings live in the database (see "Settings"), not in the env file. The worker keeps one connection that listens on two channels:

- `ps_settings`: the app sends it when an Admin saves the Feed tab. The worker reloads the settings and recomputes the next run of every job. A new interval or the pause switch applies at once, without a restart.
- `ps_job_request`: see "Run now" below.

The listening connection reconnects with backoff when it drops.

### Pause

When `paused` is on, scheduled fires of the pausable jobs (`visual-feed`, `drawing-lookup`) are skipped. Each skip is recorded as a `job_runs` row with status `skipped` and counts `{"reason": "paused"}`. Auto-release does not run while paused. "Run now" still works while paused. Housekeeping and the phase 9 jobs are not paused.

### Run now and job_requests

1. Clerical presses **Run now** on the From VISUAL panel (or an Admin on the Feed tab). The app inserts a `job_requests` row and sends `NOTIFY ps_job_request` with `{"job": "<name>"}` in the same transaction.
2. The worker moves that job's next run to now.
3. The run claims every pending request for the job at its start. Ten clicks make one run. The run's trigger is `request` when it claimed any.
4. A request that arrives while the job is running is left pending, and the job runs again straight after.

### Advisory lock

Each run opens its own connection and takes `pg_try_advisory_lock(7310, hashtext(<job name>))`. When the lock is held, the run records status `skipped` with counts `{"reason": "already running"}` and stops. One run per job at a time, whether the trigger was the schedule, a request, a chain or the command line.

### job_runs

Every run writes one `job_runs` row:

- `trigger`: `schedule`, `request`, `chain` or `cli`.
- `requested_by`: the users whose requests the run claimed.
- `started_at`, `finished_at`.
- `status`: `running`, then `ok`, `degraded` (finished but part of it failed, for example the order read), `failed` (an exception), `skipped` or `abandoned`. On start the worker marks its own host's rows still `running` as `abandoned`.
- `counts`: a JSON object (below).
- `errors`: at most 20 strings of at most 500 characters each, redacted. Redaction strips anything that looks like `password=...`, `pwd=...` or an `Authorization:` header, and the VISUAL and M-Files passwords themselves when they are in the environment.
- `build_version`, `host`, and `visual_target` (`"<server> / <database>"`) on feed runs.

Feed run counts, all integers: `wos_window`, `wos_tracked`, `rows`, `materials`, `orders`, `labor_legs`, `new_drafts`, `drafts_updated`, `cos_opened`, `cos_reaimed`, `cos_withdrawn`, `retired`, `wo_gone_flagged`, `closed_observed`, `ignored_golive`, `projects_created`, `units_created`, `auto_released`, `unmapped_resources`, `skipped_removed`, `skipped_closed` and `suspect` (0 or 1). `counts["unmapped"]` holds `{resource: rows}` for the top 30 unmapped resources.

Drawing run counts: `checked`, `fetched`, `unchanged`, `attached`, `flagged` (`{kind: n}`), `lookup_failed`, `cos_opened`, `staged` and `budget_left`.

`packetstation job run <name>` runs one job once in the foreground with trigger `cli`, prints its `job_runs` row as JSON, and exits 1 when the run failed.

### Heartbeat

The worker writes the `worker_heartbeat` row every 30 seconds (time, build, host, pid and the jobs running now), and touches the beat file the container healthcheck reads. The From VISUAL panel shows "Worker not running" when the heartbeat is stale.

### Stack guard

At start the worker runs the same stack guard as the app: the test stack refuses a production VISUAL target, and the production stack refuses a VISUAL target that was not set explicitly. Before every feed run the guard runs again with fresh VisualKit settings. A failed guard fails the run. The run records the target it read.

## Jobs and cadence

| Job | Cadence | Pausable | What it does |
|---|---|---|---|
| `visual-feed` | `interval_shop_minutes` inside shop hours (default 5), `interval_off_minutes` outside (default 30) | yes | The five reads, the cache, the packet rules, auto-release. Chains `drawing-lookup`. |
| `drawing-lookup` | After every feed run, plus every `drawing_interval_minutes` (default 30) | yes | The drawing queue (see "Drawings"). |
| `housekeeping` | Daily at 03:30 site time | no | Prunes old rows. The plan keeps the caches 90 days and `job_runs` 180 days. |
| `ad-reconcile` | Daily at 01:30 site time, plus "Refresh from AD now" | no | Users and roles from the AD groups, inactive users signed out, employee ID flags (phase 9). |
| `session-refresh` | Every `roles_refresh_minutes` (default 15) | no | Every live session's groups and account state read again (phase 9). |
| `integrity` | Daily at 04:00 site time | no | Seals, asset hashes, missing and orphan files, the audit hash chain (phase 9). |
| `backup-check` | Daily at 07:15 site time | no | The newest dump is there, not empty, under 26 hours old, and not followed by a failure (phase 9). |
| `notify-send` | Every 60 seconds | no | Rule seed, the version check, the watch checks, immediate delivery and the digest queue (phase 9). |
| `notify-digest` | Every 5 minutes | no | Sends each digest that is due, one message per address (phase 9). |

`job_schedule` holds each job's next run, interval and pause state for the Admin tab. Migration 0004 seeds rows for `visual-feed`, `drawing-lookup` and `housekeeping`; 0010 adds the six phase 9 jobs.

## Phase 9 jobs

None of the six phase 9 jobs is pausable. Each is a module in `worker/jobs/` with its own `SPEC`. `load_jobs()` imports them lazily, so a module that fails to import is logged and left out, and the rest run. They use the same run now, advisory lock and `job_runs` rules as the feed. `packetstation job run <name>` runs any of them once.

### ad-reconcile (01:30 site time, plus "Refresh from AD now")

1. Reloads the departments (and with them the group map).
2. Reads AD, outside any transaction: every group in `LDAP_GROUP_BASE_DN` (name and DN), the members of every mapped group through `LDAP_MATCHING_RULE_IN_CHAIN` in pages of 500 (disabled accounts included, so they can be marked), and, by objectGUID in batches of 50, every active user who was not in any group.
3. **Refusal guard.** When no mapped group is in the OU (usually a wrong `LDAP_GROUP_BASE_DN`), or when every mapped group is empty while active users exist, the run raises `ReconcileRefused`, fails with that reason in its errors, and changes nothing (PLAN.md open question 44).
4. One transaction: users upserted on objectGUID and their roles rebuilt; a disabled user marked inactive with their sessions ended `ad_disabled`; a user AD no longer has marked inactive with `ad_gone`; a user in AD with no mapped group left without roles and their sessions ended `roles_lost`. `directory_flags` gets `emp_missing`, `emp_duplicate`, `gone` and `disabled`. Every ended session writes `auth.session_ended` with `by: ad-reconcile`, and the run writes an `ad.refresh` audit row with its counts.
5. Events, in their own transaction: `group_missing` (a condition per missing group, subject the group name) raised, and resolved for every mapped group present; `ad_reconcile_report` (one-shot, info) when anyone was added or deactivated, an extra group is in the OU, or an employee ID is missing or shared. A notify error is recorded on the run and does not fail it.

AD unreachable fails the run and changes nothing. A failed or refused run raises the `ad_reconcile_report` condition "AD reconcile did not run" (subject `ad-reconcile:failed`), so the rule's recipients get one mail until a run succeeds and clears it. The counts are the report the Admin page shows: `groups_checked`, `groups_missing` and `groups_extra` (names), `users_seen`, `created`, `updated`, `deactivated`, `sessions_ended`, `emp_missing` (accounts) and `emp_duplicate` (employee IDs), lists capped at 200.

"Refresh from AD now" inserts a `job_requests` row for `ad-reconcile` and answers 202 (open question 48). `packetstation ad-refresh` runs the same reconcile in the foreground.

In CI the Samba DC has an extra group and a duplicate employee ID on purpose, so the report is raised every night there.

### session-refresh (every `roles_refresh_minutes`, default 15)

For every user with a live session (not revoked, not expired): AD first, by objectGUID in batches, then each enabled user's groups. Then one transaction: a disabled or gone user is marked inactive and their sessions ended (`ad_disabled`, `ad_gone`); a user left with no role has their roles cleared and sessions ended (`roles_lost`); everyone else gets their user row, `user_roles`, each live session's `roles_snapshot` and `roles_checked_at` updated. Any AD failure makes the run `degraded` with `counts.ad = "unavailable"`, and nothing is written or ended. The interval comes from the Security tab setting, read at each run, so a changed value applies after the next run. Counts: `users`, `refreshed`, `roles_changed`, `deactivated`, `roles_lost`, `sessions_ended`.

### integrity (04:00 site time)

Each step in its own transaction:

- Seals: every unbroken seal of a filed packet is rebuilt and compared with the seal key. A seal made under another `key_id` is a finding. No seal key makes the run degraded and the seals unchecked.
- Assets: every asset row checked for its file (`asset_missing`). The sha256 is recomputed for assets uploaded in the last 7 days every night, and for every asset on Sundays (site time); `hash_checked_at` and `hash_ok` are set (`asset_hash`).
- Orphans: files under `ASSETS_DIR` that no asset names, older than one hour (uploads in flight are skipped), `lost+found` skipped (`blob_orphan`). Reported, never deleted.
- Audit: `audit.verify_chain` over the hash chain (`audit_chain`).

One `integrity_findings` row per finding, at most 500 per kind (the count in the run's counts). Any finding raises `integrity_failure` (condition, critical, subject `integrity`). A clean run that was not degraded resolves it. Counts: `full`, `seals`, `assets`, `hashed`, `files`, `audit_rows`, `findings` per kind, `total`.

### backup-check (07:15 site time)

Reads the stack's backup folder, mounted read-only at `/backups` (`BACKUP_DIR` on the host). Problems: the folder is not there, no `*.dump` at the top of it, the newest dump is empty or older than 26 hours (by modification time), or `last-failure` is newer than `last-success` (or there is a failure marker and no success marker). Any problem raises `backup_stale` (condition, subject the stack name); none resolves it. The run itself stays `ok`; the alert carries the news. Counts: `folder`, `dumps`, `newest`, `newest_at`, `age_hours`, `size`, `last_success`, `last_failure`, `problems`. The CI stack's backup folder is empty, so CI raises `backup_stale` in dry run.

### notify-send (every 60 seconds)

1. Seeds any missing rule (on, at once, digest 07:00, to PS-Admins). Never changes a rule that exists.
2. On its first run in each worker process: when `PS_BUILD` differs from the settings row `notify.last_build`, updates the row and raises `version_deployed` (subject `build:<build>`). Not at worker start, since the scheduler may start before PostgreSQL answers (open question 47). A restart on the same build raises nothing.
3. At most every 15 minutes per process, the watch checks: `drawing_failing` when drawing lookups have been failing for more than the rule's hours (default 24; subject `drawings`, the list of up to 50 in the body), resolved when none is; `co_waiting` per change order pending review more than the rule's days (default 3) on a live packet (subject `co:<id>`, the packet's department), resolved when it is no longer waiting.
4. Delivery: up to 200 undispatched events, oldest first, each in its own transaction with `FOR UPDATE SKIP LOCKED`. No rule: `no_rule`. Rule off: `rule_off`. Digest rule: an outbox row, `queued`. Immediate: recipients resolved now, one message, one `notifications_sent` row, `sent`, `dry_run`, `failed` or `no_recipients`. A failed SMTP send is recorded and not retried. AD unavailable while resolving a group: a `failed` row "directory unavailable", the event stays pending (`directory_retry:N`), and the third failure dispatches it `failed`.

A failed or retried send makes the run degraded.

### notify-digest (every 5 minutes)

For each rule whose digest time (site time) has passed today and that has outbox rows queued before it: recipients resolved per event, then grouped per address across every rule. One message per address lists every due event (`Daily digest: N notifications`), with one `notifications_sent` row per message (event type `digest`). An outbox row is marked sent only when every address got it. A failed address leaves the rows for the next run, which skips addresses that already got an event, and the job backs off 30 minutes (open question 43). Nothing queued: the run ends without touching AD.

### Events from the feed and the drawing job

After each `visual-feed` and `drawing-lookup` run, `worker/runs.py` raises or resolves the feed conditions:

| Run status | `feed_failed` (subject the job name, critical, the first ten errors in the body) | `feed_degraded` (visual-feed only) |
|---|---|---|
| failed | raised | unchanged |
| degraded | resolved | raised |
| ok | resolved | resolved |
| skipped | unchanged | unchanged |

`drawing_failing` comes from the notify-send watch check above, not from the run. Each event goes to its rule's recipients, PS-Admins by default. A broken run connection falls back to a fresh one and never fails the run.

### What each event means

| Event type | Raised by | Kind | Cleared by |
|---|---|---|---|
| `security_alert` | Account cooldown, PIN disabled, session ended after wrong PINs | one-shot, 15 min per subject | none |
| `signin_burst` | An address over its limit, or the burst threshold across accounts | one-shot, 30 min per subject | none |
| `integrity_failure` | integrity | condition | a clean integrity run |
| `feed_failed` | visual-feed or drawing-lookup failed | condition | an ok or degraded run |
| `feed_degraded` | visual-feed degraded | condition | an ok run |
| `drawing_failing` | notify-send watch | condition | no lookup failing that long |
| `ad_reconcile_report` | ad-reconcile | one-shot | none |
| `group_missing` | ad-reconcile, per group | condition | the group found |
| `backup_stale` | backup-check | condition | a good backup |
| `co_waiting` | notify-send watch, per change order | condition, department | decided or no longer waiting |
| `qc_reject` | A Q.C. reject on a packet, an ITP line reject | one-shot, department | none |
| `version_deployed` | notify-send, first run on a new build | one-shot | none |
| `assets_quota` | The storage check after every upload and in housekeeping | condition | usage back under the threshold |

A condition sends once when it opens and a "Cleared: <title>" message to the same rule when it closes. Raised again while open, it only counts (`raised_count`). The department of an ITP reject is the one department among the unit's live packets that works to the line's page; none or several gives none.

## Feed reads

Each feed run opens one VisualKit connection (`Visual.from_env()`), does five reads outside any database transaction, and closes it. Every value goes through one parser, so VisualKit and the fixture source produce the same data. The site is `SITE_ID` (default `PESCO`) in every call.

The window comes from today in site time: `since` is today minus `window_back_days` (default 14), `until` is today plus `window_ahead_days` (default 30).

1. **Released work orders in the window.** `v.work_orders.released(site, since, until, status=("F",))`, paged. Firmed only. `until` is always passed. This read creates and updates drafts.
2. **Tracked work orders by key.** `v.work_orders.get_many(keys)` for every work order key on a live packet that is not filed, plus every ignored work order Clerical pulled in. No status filter and no date filter, so a packet already in the shop keeps getting changes and its closed state. Keys VISUAL no longer returns are collected as missing.
3. **Operations, text, parts and materials** for the union of reads 1 and 2: `operations_for`, `operation_text_for` (the planner instructions, called partspecs), `parts.get_many` for descriptions. Materials: `work_orders.for_bases(bases, type="W", status=("F", "R"))`, `requirements_for`, `requirement_text_for`, `parts.get_many` and `parts.sites_for`. Materials are known per leg: the legs `for_bases` returned are the legs whose BOM the run knows (`mat_known`). A closed leg's materials are not read, and that does not look like a removed BOM.
4. **Order links** by work order base and by project, through `v.links.orders_for_bases`, `v.links.orders_for_projects`, `v.orders.get_many` and `v.orders.customers`. When this read fails, the run is degraded and the apply keeps using the last cached orders.
5. **Labor tickets**, when `labor_tickets` is on and the last labor read is more than 30 minutes old: `v.labor.tickets_for_work_orders(keys, since=...)`, summed to hours and ticket count per leg and operation. Employee 9999 is left out (VisualKit's default). No cost column is ever read.

A work order that leaves the window is not a removal. Once a packet tracks it, read 2 keeps reading it whatever its date or status.

### Suspect batch

A batch is suspect when:

- the window read returned zero rows and the previous batch had some, or
- more than 5 tracked keys are missing and they are more than 10% of the tracked keys.

A suspect batch is still cached and applied, but without the "gone" marking in the cache and without the "work order gone" step. The run is degraded and the reason is in its counts.

## Cache

`feed/cache.py` writes one batch per run, in one transaction:

- **Batch.** A `visual_batches` row: run, source (server and database, or the fixture state), site, window, window rows, tracked keys, missing keys, suspect, counts.
- **Rows.** The rows are copied into a temporary table and merged into `visual_rows`, keyed on the typed key (base, lot, split, sub, seq). A new row gets `first_batch` and `last_batch`. A row whose hash changed is updated. A row seen again moves `last_batch`. `row_hash` is a sha256 of the canonical row (the same canonical form as the fingerprint).
- **Materials.** The same for `visual_materials`, keyed on the typed key plus piece, with `mat_hash`.
- **Change rows.** Each insert, change, return and disappearance writes a `visual_row_changes` or `visual_material_changes` row for the batch, with the row before and after. Kinds: `added`, `changed`, `gone`, `returned`. Admin reads these as "what VISUAL changed".
- **Gone marking.** A cached row that is in play (its key is tracked, or it is firmed with its release date inside the window) and was not in this batch gets `gone_batch` and a `gone` change row. Materials are in play when their base is. A row that comes back clears `gone_batch` and gets a `returned` change row. A suspect batch marks nothing gone.
- **Orders.** `visual_orders` is upserted and `visual_order_links` is replaced for the bases and projects read.
- **Labor.** `visual_labor` is upserted per leg and operation.

The apply then reads the rows back from the cache, so it works on one consistent snapshot.

## Packet rules

### One packet

One packet is one drawing, one lot and one operation (resource). Its match key, `packets.visual_key`, is `<unit>|<resource>|<drawing>`, with `(no drawing)` when the drawing is blank. Several work order operations (legs) can share one packet.

### Resource map

Each VISUAL resource maps to a shop department code, `fold` or `ignore` (`visual_resource_map`, seeded from the old build's `visual_map.json`, edited on the Admin "VISUAL resources" tab).

- A department code makes packets in that department.
- `fold` makes no packet. The row's operation is listed on the packets of the same work order and lot as a folded operation. A resource starting `MAT_` with no map entry is fold.
- `ignore` makes nothing.
- A resource with no entry makes nothing and is counted under `unmapped`, so IT can assign it.
- When `departments` is set in the feed settings, rows routed to other departments are skipped silently.

### Project and unit

- The project number is the work order base's prefix as the old build took it.
- The unit is the project number plus the lot zero-padded to 3 digits (`L900` and lot `1` make `L900001`).
- A lot that is not all digits (lot `7A`) makes no candidate. The old build dropped them the same way. Hand units may still use such a lot.
- A project or unit that does not exist is created (see the apply, step 1).

### Which rows take part

A row takes part when it routes to a department and either its key is tracked (any status, any date), or it is firmed with a release date inside the window. A row that is not tracked and was released before the go-live date is set aside (see "Go-live boundary").

### The candidate

Rows are grouped by match key. For each group:

- Legs are deduplicated on (key, seq).
- `rel` is the earliest release date, `want` the latest want date.
- `rev` is the highest numeric rev among the legs. A non-numeric rev only wins when there is no rev yet.
- `title` is the first leg's description.
- `folded` lists the fold rows of the same work order and lot. `partspecs` lists the distinct planner instructions of every row of the same work order and lot, in first-seen order.
- The BOM is built from the materials (the old `matBom`):
  - A line belongs to a leg when its drawing is blank or matches the leg's drawing, ignoring spaces and case.
  - Per piece the best status wins: R, then F, then U, then anything else.
  - Lines without a part are dropped, except as group headers: a piece that is a multiple of 10 heads the next 9 pieces.
  - The item is the piece number, with `-<last part of the work order>` added on a repeat.
  - A line is a sub-assembly and n/a when its commodity is WIP or its warehouse is ASY.
  - The location is the warehouse (unless MAIN) and the location, joined by a space.
  - When no leg of the candidate has known materials, the BOM is unknown and the packet's BOM is left alone.

### The apply

The apply runs in one transaction with a 5-second lock timeout. A packet row locked by someone else is skipped and picked up next run. Steps, in order:

1. **Projects and units.** Find each candidate's project through the alias table, so renames resolve. Create a missing project (source `visual`, no name, `needs_details` on, due = the latest want date) and its alias. Create a missing unit the same way. Fill the order-derived fields (see "Several open orders"). Candidates under a closed project or unit are skipped (`skipped_closed`).
2. **Match.** Look up every candidate key among packets, removed ones included. A key whose only packets are removed makes nothing, ever (`skipped_removed`). A key with a live packet is existing. Anything else is new.
3. **New drafts.** Apply the go-live boundary first. Then insert the draft: header, work orders per leg (qty, target hours, op, status, partspecs, drawing, rev), BOM lines (source `visual`), the visual state row, `drawing_state` `needed` and flag `drawing_needed` (or `none` and flag `no_drawing` when the drawing is blank), the fingerprint. Log "Packet created from VISUAL" by "VISUAL feed". Audit `packet.created` with role `system`.
4. **Drafts updated in place** when the fingerprint or the status facts changed (action `vis_update_draft`):
   - Work orders are replaced, keeping rows whose key and seq match.
   - The BOM is replaced unless Clerical uploaded one by hand.
   - When the drawing or the rev moved: `drawing_state` goes back to `needed`, every drawing page is taken off (the file stays in assets), a hand drawing is taken off, any approved rev is cleared, the drawing flags are cleared and `drawing_needed` is added.
   - The fingerprint, want and release dates, folded operations and partspecs are stored.
   - Log "Updated from VISUAL" with the lines that changed ("work order X added", "X target 1.5 h -> 2 h", "drawing rev 1 -> 2", want date, material lines).
5. **Past draft: change orders** (action `vis_open_co`). See "Change orders from the feed". Filed packets get nothing.
6. **Retire moved drafts** (action `vis_retire_draft`). A live draft made by the feed, with no drawing pages and no hand uploads, that is not among this run's candidates, and whose every leg now sits on another live candidate, is soft-removed with reason `moved`. `visual.moved` records where the legs went. Log "Retired: work orders moved to another packet". It is hidden from the boards and, being removed, is never recreated.
7. **Work order gone.** Skipped on a suspect batch. Each tracked leg VISUAL did not return gets `missing_runs + 1`. At 2 the leg gets `gone_at`, the packet gets flag `wo_gone` and `wo_gone_at`, and the log says "Work order <key> no longer in VISUAL". A leg that comes back resets to 0. When no leg is gone any more the flag goes and the log says "Work order back in VISUAL". The feed never removes a packet for this.
8. **Closed and cancelled.** Each leg's status code is stored, with ticket hours and the time observed. When every leg is C, `visual_closed_at` is set. Any leg X sets flag `cancelled` and `visual_cancelled_at`. The flag goes when no leg is X. A cancelled draft is not removed.
   Since change C5 (Gitea issue 31, built 2026-10-30) the feed also sets and clears the packet's close in VISUAL, `vclosed_at`, on a VISUAL packet that is not filed or removed. It is the one writer there; Clerical sets it only on hand packets.
   - When every leg has a status and each is C or X, the feed sets `vclosed_at` to now, with `vclosed_by` null (an existing value is kept). The log says "Closed in VISUAL (observed)" and the run counts `closed_observed`. The Clerical board then moves the packet from "Ready to close" to "Ready for data book" by itself.
   - When a leg shows open again (a status that is not C or X) on a packet with `vclosed_at` set, the feed clears `vclosed_at`, `vclosed_by` and `visual_closed_at`. The log says "Open again in VISUAL (observed)" with the open work orders as the note. No count.
   - The feed does this in any status before filed, so a packet can come back from Q.C. already "Ready for data book". A filed packet is never touched.
   - FILE on a VISUAL packet needs `vclosed_at` set and no leg open, on both routes (precondition `closed_in_visual`, see [POLICY.md](POLICY.md), "Filing").
   - A leg flagged `wo_gone` keeps its last status. If that status was open, the packet cannot file. Asked on Gitea issue 31 on 2026-10-30: count a leg gone for 2 runs as closed, or let an Admin close it with a reason.
9. **Auto-release** when `auto_release` is on and the feed is not paused: up to `auto_release_max` (default 25) live drafts made by the feed with the drawing attached, no flags at all, and the rev confirmed by the drawing job. A draft whose heat map is unanswered gets "yes" when it has a real material line (not a sub-assembly, not n/a, not consumable) and "no" otherwise. Then action `vis_auto_release` moves it to the warehouse with an automatic warehouse stage row and a transition row. The policy's preconditions (heat map answered, drawing attached) still apply.
10. **Summary.** One audit row `feed.run` with the counts, and one `feed` event to the screens after commit.

Every packet write in the apply writes a `packet_log` row (by "VISUAL feed", role `system`), an audit row (`packet.<action>`, role `system`) and a change event, through the same rules as a person's write (see [POLICY.md](POLICY.md), "Worker writes").

## Fingerprint

The fingerprint is a canonical hash over the fields a packet is built from. `feed/fingerprint.py` computes it per candidate. `packets.visual_fp_applied` stores the full hash, and `packets.visual.fp_parts` stores the hash of each part, so the next run can say which parts changed.

Canonical form: text in NFC, line ends as LF, trimmed; null and empty text alike; IDs (work order base, part, drawing, lot, split, sub) upper-cased; lot, split and sub kept as text; seq and piece as integers; numbers normalised (`2` and `2.0` are the same); dates in ISO form; lists sorted (legs by base, lot, split, sub, seq; BOM by item). JSON with sorted keys, then sha256.

| Field | Part | What a change does past draft |
|---|---|---|
| Legs (work order keys and operations on the packet) | major | Opens or re-aims a change order |
| Quantity per leg | major | Opens or re-aims a change order |
| Rev | major | Opens or re-aims a change order, waiting for the drawing at the new rev |
| Drawing number (upper-cased, spaces removed) | major | Opens or re-aims a change order, waiting for the new drawing |
| BOM (item, part, qty per line) | major | Opens or re-aims a change order |
| Target hours per leg | minor | Opens a minor change order: Clerical approval only, no department verifies |
| Want date | minor | Opens a minor change order |
| Release date | never | Nothing |
| Folded operations | never | Nothing |
| Partspecs (planner instructions) | never | Nothing |
| Title | never | Nothing |
| Work order status | never | Nothing (status is observed separately: closed, cancelled) |
| Issued quantity, warehouse, location | never | Nothing |
| Orders, labor, department, operation | never | Nothing |

On a draft any fingerprint change updates the draft in place, major or minor. A change order is only for packets past draft. Tests in `tests/server/test_fingerprint.py` show each row.

## Go-live boundary

With no migration from the M: install, the first run would make drafts for every firmed work order in the window, including jobs already running on paper or on M:. The `golive_date` setting keeps them out:

- A row that is not tracked and was released before the go-live date makes no candidate.
- A candidate none of whose legs is tracked and whose earliest release is before the go-live date is not created. Its legs go into `visual_ignored_wos` (`ignored_golive`).
- The From VISUAL panel lists the ignored work orders, newest release first. **Pull in** asks a reason, records who and when, and requests a feed run. A pulled-in work order is tracked from then on, so the next run makes its draft whatever its release date.

Jobs already open on M: finish there.

## Change orders from the feed

A packet past draft (warehouse to complete) is never changed silently. A fingerprint change opens a change order instead.

- **Open.** A changed fingerprint opens a change order with origin `visual`, review `pending`, ECN `VISUAL`, issued by nobody. It carries `vis` (from and to drawing and rev, the list of changes in words, the legs, the BOM, want and release dates, folded operations, partspecs and the fingerprint parts), `target_fp`, `minor`, and `waiting` (`{dwg, rev, since}`) when the drawing or rev changed and the new drawing is still needed.
- **One at a time.** At most one pending feed change order per packet (origin `visual` or `drawing`). A further change re-aims it: reason, `vis`, `waiting`, `target_fp` and `minor` are updated, the move is appended to `vis.moved`, and a staged drawing is kept when the target drawing and rev did not change.
- **Unchanged.** A pending change order with the same target does nothing. A rejected one with the same target does nothing either: Clerical rejected exactly this.
- **Withdraw.** When VISUAL goes back to what the packet has, the pending change order is withdrawn: review `rejected`, rejected by nobody, with the reason in the log.
- **Follow on.** An approved change order with the same target does nothing. A change after an approval opens a fresh change order.
- **Minor.** When only target hours or the want date changed, the change order is minor. It needs Clerical approval only. On approval no department is asked to verify.
- **Hold.** A pending change order holds nothing unless Clerical puts it on hold. While held it blocks the packet's held steps (see `v_open_cos` in [SCHEMA.md](SCHEMA.md)).
- **Approval.** Clerical approves with a signature. When the drawing changed, approval needs the new drawing staged first ("The new drawing has not been fetched from M-Files yet."). Approval applies `vis`: work orders from `vis.legs`, the BOM unless it was uploaded by hand, the header drawing and rev, the fingerprint, want and release dates. A staged drawing replaces the old drawing pages: the old pages are voided (never deleted) and the new pages added, with the page changes in `co_items`. A major change order then asks the departments it touches to verify, and holds the packet until they have. A minor one asks nobody.
- **Reject.** Clerical rejects with a reason and a signature.

A packet at `complete` still gets a change order. A filed packet gets nothing.

## Drawings

The `drawing-lookup` job finds each packet's drawing in M-Files through MFilesKit, downloads it, and attaches it to drafts.

### How MFilesKit finds a drawing

This is what `mfileskit/drawings.py` does. MFilesKit's own `docs/packet-station.md` is stale in two places, noted below.

- **Key.** Hyphens and spaces removed, upper-cased: `L073-76-2` becomes `L073762`.
- **Search by Name property, not folders.** The released PDF is searched in class "Drawing Production (PDF)" by the Name property starting with the key, and a hit counts only when its name is the key or the key plus `X`. There is no project-folder rule any more (the `P-521` to `H777` mapping in the MFilesKit docs is gone).
- **Released rev.** Only a document whose Controlled Status (or workflow state) is Effective counts. The rev is its Controlled Version Label, else the number after `R` in its title. The highest released rev wins.
- **Drafting rev.** Working copies are searched by Controlled Version Title. Copies in a done status (released, effective, retired, obsolete) or a done state (drawing released to shop, drawing already exists, cancel drawing) are skipped. The drafting rev is the highest label among the rest. Its workflow state maps to "still being drawn (unapproved)" or "in the drafting check cycle". (The MFilesKit docs still say "the higher of the Unapproved rev and the Check Cycle rev"; that is stale.)
- **In drafting.** A Drafting Assignment named for the key means the drawing is in drafting.
- **H sheets.** A drawing like `K978-01-6H13` has no object of its own. When nothing is found for it (no released PDF, no working copy, no assignment), the lookup tries the base drawing `K978-01-6` and reports `base`. The packet note then says "sheet of drawing K978-01-6".
- A search that hits the 500-result cap is a failed lookup, so a wrong rev is never reported.

### Queue and priorities

The queue covers live packets made by the feed, not filed, with a drawing number, in the departments in scope. It is distinct by drawing number, and the first item's priority wins:

1. The VISUAL rev changed since the last lookup of that drawing, or a pending change order waits for a drawing at a target rev.
2. Drafts with `drawing_state` `needed`.
3. Failed lookups whose backoff is over (15 minutes doubled per failure, at most 24 hours).
4. Rechecks due: last checked more than `drawing_recheck_hours` ago (default 4).

### Budget

At most `drawing_budget` lookups per run (default 60). The rest wait for the next run. The worker opens one M-Files vault per process and a new `DrawingLookup` per run, so its cache never outlives a run. When the vault cannot be opened, the run is skipped and degraded with one error.

### Download

The file is downloaded only when the object id and version or the file id and version differ from the last lookup, or nothing is stored yet. The download goes to a temporary file, is checked for the PDF magic bytes and the upload size limit, hashed with sha256 while streaming, then fsynced and renamed into the assets volume. A file with the same sha256 and size reuses the existing asset. Pages are counted with pypdfium2 (pypdf as a fallback). Each fetch writes an audit row `drawing.fetched`.

### Decision table

`drawings/decide.py` decides per packet sharing the drawing. Rows are checked top to bottom.

| Case | Decision | Flag | What happens |
|---|---|---|---|
| The lookup failed | `lookup_failed` | `lookup_failed` | What is attached stays. `drawing_state` becomes `lookup_failed` only if it was `needed`. Retried with backoff. |
| No released rev, but a drafting rev or a drafting assignment | `not_released` | `not_released` | Note "Not released in M-Files yet (sheet of <base>) - drafting status: rev N <where>". |
| No released rev and nothing in drafting | `not_found` | `not_found` | Note "Drawing not in M-Files - check the drawing number in VISUAL". |
| VISUAL's rev is not a whole number | `bad_rev` | `bad_rev` | Not attached. |
| Revs differ, and no approved rev matches M-Files, and M-Files holds the rev VISUAL calls as Effective or Retired | `attach_called` | none | VISUAL is the source of truth. On a draft the rev VISUAL calls is fetched into `drawing_revisions` and attached, with no flag and no Clerical approval: `drawing_state` `attached`, rev confirmed, `visual.vault.controlled` set (for example Retired), `visual.called` recorded. Log "Drawing attached from M-Files" with the note "Rev V is Retired in M-Files; VISUAL calls it (M-Files has rev N)". Already attached, a hand drawing, or past draft: unchanged, no flag and no change order. The same holds when M-Files has no Effective rev at all. |
| Revs differ, and no approved rev matches M-Files, and M-Files has no document at the rev VISUAL calls, or has it at another status (Released, Obsolete) | `rev_mismatch` | `rev_mismatch` | Note "Rev mismatch: VISUAL calls rev V, M-Files has rev N", plus the drafting status, plus "rev V is not in M-Files" or "rev V is <status> in M-Files - not attached". Only an Effective or Retired drawing is ever attached. The M-Files copy is fetched and staged for Clerical Supervisor approval. No change order: VISUAL did not change. |
| Revs match (or the approved rev matches), draft | `attach` | none | Drawing pages attached, `drawing_state` `attached`, drawing flags cleared, rev confirmed. Log "Drawing attached from M-Files". |
| Revs match, draft with a hand drawing at the same rev | `unchanged` | none | The hand drawing stays. |
| Revs match, past draft, same file (same sha256) | `unchanged` | none | Nothing. |
| Revs match, past draft, same rev but a new file | `open_co` | none | Drawing-origin change order: "M-Files re-released rev N of <dwg> (new file)". |
| Past draft, M-Files released a higher rev than the one attached | `open_co` | none | Drawing-origin change order: "M-Files released rev N of <dwg>; this packet has rev M". Never a silent swap. |
| A pending feed change order waits for this drawing at the rev M-Files now has | `stage_for_co` | none | The fetched file is staged on the change order and `waiting` is cleared. Log "Drawing for change order N ready for Clerical". |

A flag decision sets `drawing_state`, replaces the drawing flags with the decision's flag, stores the M-Files answer in `visual.appr`, and clears the rev confirmation. On a draft it is a `vis_update_draft` write. Past draft it is a `vis_open_co` write logged "Drawing status from M-Files", since that is the only system action allowed past draft.

"Drawing needed" blocks the warehouse step through the `drawing_attached` precondition.

### Rev mismatch staging and approval

When M-Files has a different released rev from VISUAL, the M-Files copy is fetched and its asset is kept on the `drawing_lookups` row. A Clerical Supervisor can approve it for one draft with a reason and a signature (action `rev_approve`). The pages are attached, the drawing flags are cleared, and `visual.rev_ok` records the rev, the VISUAL rev, who, when and why. The log says "Different rev approved by hand" with "rev N in place of VISUAL rev M". Without a staged copy the answer is "nothing staged".

### Hand drawing taken off on a new rev

A draft whose VISUAL drawing or rev moves loses every drawing page, a hand drawing included (step 4 of the apply). The drawing job then looks up the new rev. A hand drawing at the same rev is left alone.

### Drawing-origin change orders

A drawing change on a packet past draft opens a change order with origin `drawing`, ECN `M-FILES`, the new file already staged, and no target fingerprint. It counts as the packet's one pending feed change order. Approval voids the old drawing pages and adds the new ones, as for a feed change order.

## Flags

Flags live in `packets.flags`. A draft with any flag shows in Clerical's Flagged column. Auto-release never takes a packet with a flag. The find bar filters by flag kind.

| Flag | Text shown | What Clerical does |
|---|---|---|
| `drawing_needed` | Made from VISUAL - drawing not attached yet | Usually nothing: the drawing job attaches it. Run now to hurry it. Upload by hand only if it stays stuck. |
| `rev_mismatch` | The M-Files note, for example "Rev mismatch: VISUAL calls rev 2, M-Files has rev 3; rev 2 is not in M-Files" | M-Files has no usable copy of the rev VISUAL calls. Get the rev corrected in VISUAL, or a Clerical Supervisor approves the staged M-Files rev with a reason and a signature. (When M-Files holds the rev VISUAL calls as Effective or Retired, it is attached automatically and the packet is not flagged.) |
| `not_released` | The note: "Not released in M-Files yet ... drafting status: rev N <where>" | Wait for drafting to release it, or upload the drawing by hand with a reason and a signature. |
| `not_found` | Drawing not in M-Files - check the drawing number in VISUAL | Check the drawing number in VISUAL and have it fixed there. Or upload by hand. |
| `lookup_failed` | M-Files lookup failed: (the reason) | Nothing at first: the job retries with backoff. If it persists, tell IT. |
| `bad_rev` | VISUAL rev is not a number - drawing not attached | Have the rev fixed in VISUAL, or upload by hand. |
| `no_drawing` | (the draft has no drawing number) | Have the drawing number added in VISUAL, or upload by hand. |
| `wo_gone` | Work order no longer in VISUAL | Check VISUAL. If the job was deleted on purpose, remove the draft by hand (or handle the in-shop packet). The flag clears itself if the work order comes back. |
| `cancelled` | Work order cancelled in VISUAL | Decide: remove the draft by hand, or leave it if the cancel was a mistake. The feed never removes it. |
| `hand_drawing` | Drawing uploaded by hand - check it and send it on | Check the drawing and send the draft on by hand. |
| `hand_bom` | BOM uploaded by hand - check it and send it on | Check the BOM and send it on. "Use VISUAL's BOM again" puts the feed's BOM back. |

With both hand flags the text is "Drawing and BOM uploaded by hand - check it and send it on". Hand drafts keep their phase 2 text.

"Closed in VISUAL" is not a flag. It is an observed state, shown on the packet: `vclosed_at` when every leg is closed or cancelled, `visual_closed_at` when every leg is C. Filing a VISUAL packet needs `vclosed_at` and no open leg, on both routes, as the `closed_in_visual` precondition says (change C5).

### Hand uploads

On a flagged draft made by the feed (or one already uploaded by hand), Clerical can:

- **Upload a drawing** (PDF) with a reason and a signature. It replaces the drawing pages, sets `drawing_state` `hand`, clears the drawing flags and adds `hand_drawing`.
- **Upload a BOM** (XLSX or CSV) with a reason and a signature. The server parses it: the header row is found in the first 20 rows by the old build's column patterns (item, qty, part, description, length, material, drawing reference); qty and a description or part are needed; empty rows are skipped; items default to the row number. It replaces the BOM lines (source `hand`) and adds `hand_bom`. The feed then leaves the BOM alone.
- **Use VISUAL's BOM again**: rebuilds the BOM from the cache and removes `hand_bom`.

An unflagged draft refuses a hand upload ("not flagged").

## Several open orders

A project can have several customer orders in VISUAL. One function, `feed/orders.choose`, picks the order for a project or a unit. The first rule that finds one wins:

1. **Pin.** Clerical pinned an order (a customer order ID among the project's orders) on the project or unit, with a reason. Via `pin`.
2. **Base.** The order linked to the work order base itself. Via `base`.
3. **Unit top job.** The project order whose line's top job is the unit number (open orders first). Via `unit_top_job`.
4. **Project open.** The first open order of the project (status F, H or R), newest first. Via `project_open`.
5. **Project newest.** The first order of the project, open or not. Via `project_newest`.

When nothing matches there is no order.

### Provenance

The chosen order is stored as `order_ref` on the project and the unit: order ID, `via`, status, customer name, customer PO, order, desired ship and promise dates, and when it was chosen. The fields derived from it:

| Field | From the order |
|---|---|
| customer | customer name |
| po | customer PO reference |
| order date | order date |
| ship date | promise date, else desired ship date |

For each field the feed always writes `<field>_visual`. It writes the field itself, with `<field>_src = 'visual'`, only when `<field>_src` is empty or already `visual`. A value Clerical typed (`_src = 'hand'`) is never overwritten. Each change is audited as `project.updated` with `src: visual`.

### The pin

Clerical can pin an order on a project or a unit (`order_pin`, with a reason, who and when) from the project page, which lists every cached order of the project. The feed re-derives on its next run. Clearing the pin goes back to the rules.

## Settings

The feed settings are one `settings` row, key `feed`, a JSON object. An Admin edits them on the Admin page, Feed tab (password session only). Unknown keys are ignored and missing keys take the default. Every save is validated, bumps the row version, is audited as `feed.settings_changed` with each changed key's before and after, and notifies the worker.

| Setting | Default | Range | Meaning |
|---|---|---|---|
| `interval_shop_minutes` | 5 | 1 to 1440 | Feed interval inside shop hours |
| `interval_off_minutes` | 30 | 1 to 1440 | Feed interval outside shop hours |
| `shop_days` | Monday to Friday (`[0,1,2,3,4]`) | 0 (Monday) to 6 | Shop days, site time |
| `shop_start` | `06:00` | HH:MM | Start of shop hours, site time |
| `shop_end` | `18:00` | HH:MM | End of shop hours, site time |
| `window_back_days` | 14 | 0 to 120 | Window start, days before today |
| `window_ahead_days` | 30 | 0 to 365 | Window end, days after today |
| `golive_date` | none | a date | Work orders released before it are ignored |
| `departments` | all (`[]`) | shop codes | Departments to make packets for. Empty means every department the resource map routes to. |
| `drawing_recheck_hours` | 4 | 1 to 168 | Recheck each drawing after this long |
| `drawing_budget` | 60 | 1 to 500 | Drawing lookups per run |
| `drawing_interval_minutes` | 30 | not limited | The drawing job's own cadence |
| `labor_tickets` | on | on or off | Read labor tickets, at most every 30 minutes |
| `auto_release` | on | on or off | Release ready drafts to the warehouse |
| `auto_release_max` | 25 | not limited | Most drafts auto-released per run |
| `paused` | off | on or off | Stops scheduled feed and drawing runs and auto-release. Run now still works. |

The Feed tab also shows the next scheduled run and the last ten runs. The "VISUAL resources" tab edits the resource map: each resource with its rows in the last batch, target (a shop department, Fold, Ignore or not assigned) and a confirmed tick. Every map change is audited.

## Fixture sources

For CI and local runs the worker can read files instead of VISUAL and M-Files:

- `FEED_SOURCE=fixture` reads `FIXTURE_DIR/feed.<state>.json`, a file in the exact shape of VisualKit's `examples/packet_station_feed.py` output (`meta`, `rows`, `materials`, `orders_by_base`, `orders_by_project`) plus an optional `labor` list. The state is read from `FIXTURE_DIR/state` (default `1`). It returns the firmed rows released inside the window, plus every row whose key is tracked; tracked keys with no rows are missing.
- `DRAWING_SOURCE=fixture` reads `FIXTURE_DIR/drawings.<state>.json`, a map from drawing number to its rev, file name, drafting state and ids. A drawing not in the file is not found. A download copies `FIXTURE_DIR/<file>`.
- `FIXTURE_DIR` defaults to `/fixtures`.
- The production stack refuses a fixture source at start.

`tests/fixtures/generator.py` writes the files: `python -m tests.fixtures.generator --out DIR` makes `feed.1.json`, `feed.2.json`, `drawings.1.json`, `drawings.2.json`, the one-page PDFs they name, and `state` = `1`. It is deterministic. State 1 covers the packet rules (fold, ignore, an unmapped resource, a lot `7A`, two operations on one work order, no drawing, a non-numeric rev, the BOM rules, a work order before go-live, closed and cancelled legs, several open orders). State 2 changes a rev, a quantity and a target, removes one leg and moves another, so the change order, minor, gone and retire rules run. `packetstation fixtures use N --dir DIR` switches the state. The Playwright scenario writes the state file itself. See [DEPLOY.md](DEPLOY.md) for the CI step and [tests/fixtures/README.md](../tests/fixtures/README.md).

Real job data never enters git or CI. Recordings from VISUAL DEV stay local, in `tests/fixtures/local/` and `recordings/`.

## Ticket hours beside the station clock

The packet read model has a `labor` field from `visual_labor`. It joins the packet's work orders on (`wo_base`, `lot`, `split`, `sub`, `seq`) and sums per work order:

- `hours` and `tickets`: the totals over the packet's work orders.
- `per_wo`: one entry per work order leg, with `wo_base`, `lot`, `split`, `sub`, `seq`, `hours`, `tickets` and `last_ticket`.
- `observed_at`: when the worker last read the tickets.

`labor` is null for a hand packet, or when no `visual_labor` row matches. The packet page shows the sum beside the station clock, as `VISUAL tickets 3.2 h`.

The station timer stays and is shown as before. Since change C1 (Gitea issue 1, built 2026-10-30) the ticket hours decide "over target" on the Project Management reports: ticket hours above the work-order target. A packet with no ticket hours is not over target. The ticket hours never change the clock. The screen reads them from PostgreSQL, never from VISUAL.

## What the screens read

Screens read PostgreSQL only, never VISUAL or M-Files per request:

- Packets, flags, work order status, closed and cancelled state, ticket hours and the chosen order come from the packet tables and `packet_visual_state`. The ticket hours beside the station clock come from `visual_labor` (see "Ticket hours beside the station clock").
- The From VISUAL panel reads `/api/feed/overview`: pause state, next run, the heartbeat, the last ten feed and drawing runs, the ignored work orders, unmapped resources and projects without names.
- Admin reads the cache tables (`/api/feed/batches`, `/api/feed/changes`) for "what VISUAL changed".
- The project page reads the cached orders from `visual_order_links` and `visual_orders`.

Scheduled runs come from the worker's own clock, not from an open browser.