spencerm/PacketStation
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
39a4eb5213a06f01f99a391e2f5f2bbd71466186
PacketStation/docs/SCHEMA.md
T
spencermandClaude Fable 5.1 39a4eb5213
ci / frontend (push) Successful in 30s
Details
ci / server (push) Successful in 4m30s
Details
ci / image (push) Successful in 35s
Details
ci / e2e (push) Successful in 1m40s
Details
Phase 4: Warehouse screen, pull table, shorts, MTRs and the upload pipeline 
The Warehouse home with the numbers, the Open shorts table, the find bar and the five-column task board
(waiting, being pulled, on hold, pulled, released in the last 3 days) with buttons, hold reasons and drag.
Every role sees the warehouse stage on a packet as the WH pill and the line under it. The pull table on the
Materials tab: Pulled with auto-start, issued, location, heat with Enter to the next line, shorts with
receipts, N/A, MTR per line with a viewer. Send back to Clerical with a reason. Release to supervisor stays
blocked until every line is pulled or short and every pulled line has a heat.

Server: the heats_recorded precondition on WH_RELEASE, receive by custody, server-validated hold and short
reasons, the old packet history texts, GET /api/warehouse/board, cards with the warehouse stage and pull
counts. Uploads typed by content per kind (drawing, mtr, document, bom with XLSX and CSV), limits per kind,
optional ClamAV behind the clamav compose profile, a storage quota alert on the new notification_events
table (migration 0005) shown on the Admin Storage tab.

Tests: every warehouse action as every role, the release gate, the short-receive exception, the heat
pattern, upload sniffing and limits, the ClamAV path with a fake scanner, the quota alert; frontend board
and pull table tests; Playwright scenarios for the board and the pull flow. Verified on docker-1 in the CI
stack: 3557 server tests, 106 frontend tests, 17 Playwright scenarios. Docs updated; D5 closed.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-16 17:13:31 -06:00

199 lines
27 KiB
Markdown
Raw Blame History

# Packet Station schema
Status: phases 1 to 4 built (identity, organisation, projects, packets, change orders, documents, jobs, VISUAL and drawing caches, the warehouse columns in use, the `notification_events` placeholder). Phase 9 adds the notification rules and delivery. The DDL is in `server/packetstation/migrations/`; this file explains it. See [PLAN.md](PLAN.md), "Schema".
## Conventions
Identity bigints, except users, keyed on the AD objectGUID. Postgres enums for workflow states. Every timestamp is timestamptz in UTC. Soft delete uses removed_at, removed_by and removed_reason. Append-only tables give the app role insert and select only, plus a trigger that refuses updates and deletes.
## Database roles
A superuser for bootstrap, the migrate role `ps_migrate` (owns the database and the schema, runs migrations) and the app role `ps_app`. The worker shares `ps_app` and sets `application_name` to `ps-worker`, so the two are told apart in `pg_stat_activity`; a separate `ps_worker` role was not needed (decided in phase 3). `ps_app` holds CONNECT and TEMPORARY on the database (the feed COPYs each batch into temp tables) and no CREATE on the schema.
## Identity (phase 1, migration 0002)
Enums: `role_kind` (admin, clerical, clerical_supervisor, warehouse, warehouse_supervisor, qc, qc_supervisor, pm, shop_worker, shop_lead, shop_supervisor), `session_kind` (password, pin), `sig_factor` (password, pin), `dept_kind` (support, shop).
- `users`: `id uuid` is the AD `objectGUID`, stable across renames. Caches `sam_account` (unique, case-insensitive), `upn`, `display_name`, `emp_no` (text; indexed but not unique, because a duplicate must be storable and flagged), `email`, `welder_stamp`, `active`, `directory_flags` jsonb (disabled, missing or duplicate employee ID), `ad_synced_at`, `last_signin_at`. A user whose AD account is disabled or gone keeps the row, marked inactive.
- `user_roles`: (`user_id`, `role`, `dept_code`) rebuilt at every sign-in, at the 15-minute role refresh and by the AD refresh. Office roles carry their department code, admin carries null, shop roles carry the two-digit code. Only the highest shop level per department is stored.
- `sessions`: `token_hash` (HMAC-SHA256 of the cookie token with the session key), `kind`, `device_id` for PIN sessions, `expires_at` (absolute), `last_seen_at` (idle), `locked_since`, `revoked_at` and `revoked_reason`, `ip`, `user_agent`, `roles_snapshot` jsonb (`{"roles": [...], "groups": [...]}`), `roles_checked_at`, `stepup_failures`, `hidden_since` (the last hidden-tab report, cleared by the next request).
- `devices`: registered shop devices. `name`, `token_hash` of the device cookie, `registered_by`, `last_seen_at`, `revoked_at`, `revoked_by`.
- `user_pins`: one row per user with a PIN. Argon2id `hash`, `failed_count` (across sign-in, unlock and signature), `disabled_at`, `set_at`. Never readable by Admin in clear; Admin may delete the row.
- `signin_attempts`: append-only. `account` (`lower(login)`, `emp:<emp_no>` or `user:<uuid>`), `ip`, `kind` (password, pin, unlock, signature, pin_set), `ok`, `reason`, `user_id`, `session_id`. Both rate limits count the failed rows here.
- `signatures`: append-only. Who (`user_id`, `account` and `display_name` snapshots), the `session_id` and `session_kind`, the `factor`, `action`, `meaning`, `packet_id`, `subject_type`, `subject_id` and a `content_hash` (sha256 of the canonical JSON of what was signed). Later phases reference it from every sign-off row.
Append-only tables use the `ps_append_only` trigger from 0001 and revoke UPDATE, DELETE and TRUNCATE from `ps_app`.
## Organisation (phase 1 seed, phase 2 completes)
- `departments`: `code` (text PK, `CL`, `WH`, `QC`, `PM`, `01` to `14`), `slug` (the AD group short name, unique), `name`, `kind`, `sort`, `active`, `supervisor_group` (false only for PM). A trigger refuses changes to `code` and `slug`. Seeded with the 18 departments from the brief and the old `depts.js`. Migration 0003 adds `weld` (bool, default false), true for 03, 04, 05, 06, 07 and 10. A weld department gets the welding checklist, and submit there needs at least one weld and a filler.
- `stations`: (`dept_code`, `name`, `sort`, `active`), seeded with the old station lists (Stall 1 to 6, Station 1 to 4, Hydro Bay 1 and 2, Blast Booth, Paint Booth 1 and 2).
- `user_station_limits`: (`user_id`, `station_id`) primary key. A worker with rows here may only work on packets at those stations. No rows means any station in the department. The rows go when the user row goes.
The 50 AD group names are derived from `departments` plus `PS-Admins`: `PS-<slug>` and `PS-<slug>-Supervisor` for support departments, `PS-<code>-<slug>-<Worker|Lead|Supervisor>` for shop departments. `packetstation ad-groups --list` prints them.
## Enums (phase 2, migration 0003)
Declaration order matters for `packet_status`: it is the workflow order, so `status >= 'qc'` works in SQL.
| Enum | Values in order |
|---|---|
| `packet_status` | draft, warehouse, ready, assigned, in_progress, review, qc, complete, filed |
| `wh_stage` | incoming, pulling, hold, pulled |
| `co_review` | pending, approved, rejected |
| `co_origin` | visual, drawing, manual |
| `source_kind` | visual, hand |
| `project_status` | open, hold, closed |
| `route_kind` | databook, shipping |
| `drawing_state` | none, needed, attached, rev_mismatch, not_released, not_found, lookup_failed, hand |
| `itp_status` | open, signed, reopened |
Smaller closed sets are text with a CHECK: `projects.source` and every `*_src` (visual, hand), `project_aliases.kind` (renumber, unit_renumber, import), `packet_pages.type` (drawing, info, overlay), `bom_lines.src` (visual, hand), `qc_inspections.result` (accepted, rejected), `itp_lines.result` (approve, reject, na). Null means unset in each.
## Projects and units (phase 2)
- `projects`: `number` (text, unique among live rows on `upper(number)`), `name`, `customer`, `po`, `due`, `status` (project_status), `notes`, `source` (hand or visual), `needs_details` (the feed made it and Clerical has not filled it in), `version` for field CAS. Order-derived values carry provenance: `customer_src`, `po_src`, `order_date_src` and `ship_date_src` say visual or hand, and `customer_visual` and `po_visual` keep what VISUAL said. The feed never overwrites a hand value. Soft delete.
- `units`: `project_id`, `lot` (text), `name`, `due`, `status`, `notes`, the same `*_src` companions, `order_date`, `ship_date`, `version`. A CHECK keeps `lot` to 1 to 3 upper-case letters or digits, and a lot of digits only is exactly 3 long (`001`, `7A`). Unique among live rows on (`project_id`, `lot`). Soft delete.
- `project_aliases`: `alias` (text PK, upper-cased by CHECK), `project_id`, `unit_id` (null for a project number), `kind` (renumber, unit_renumber, import), `created_at`. Every current and old number of every project and unit is here, so a lookup by any number resolves, and a renumber cannot reuse a number that was ever taken.
- `project_renumbers`: append-only. `at`, `by`, `project_id`, `unit_id`, `old_number`, `new_number`, `reason`.
- View `v_units`: every `units` column plus `number` (`project.number || lot`, so `L999` and `001` make `L999001`), `project_number`, `project_name`, `project_status` and `due_effective` (the unit's due, else the project's).
## Packets (phase 2)
### Header
`packets` is the header. The columns that matter:
- Placement: `unit_id`, `dept_code`, `op`, `title`, `customer`, `drawing`, `rev`, `wps`.
- Workflow: `status` (packet_status), `status_at`, `station_id`, `welder_id` and `welder_stamp` (a snapshot taken at assign), `for_sup_id`, `wh_skipped` (true when Clerical sent the draft straight to a department or to Q.C.), `wh_sent_at`, `routed_to` (route_kind, set at Q.C. accept), `return_note` (the reason shown after a send-back), `vclosed_at` and `vclosed_by`, `databook_at`.
- Preconditions: `heat_map` is nullable on purpose. Null means not answered yet, and the warehouse step needs an answer.
- Source: `source_kind` (visual or hand), `source` jsonb, `visual_key` (the feed's match key), `visual_rel`, `visual_want`, `drawing_state`, `flags` text[], `visual_fp_applied` (the fingerprint the packet matches), `visual_closed_at`, `visual` jsonb (system-written sub-keys).
- Counters: `next_weld_no`, `next_heat_no`, `next_page_no` and `next_co_no`, all starting at 1. See "Versioning".
- `version`, `created_at`, `created_by`, `updated_at`, soft delete.
Indexes: `packets_visual_key` is unique on `visual_key` where the key is set and the row is live. A removed packet keeps its key, so two removed rows may share one, but only one live row may hold it. Also `(status)` for live rows, `(unit_id)` and `(dept_code, status)` for the boards.
The trigger `ps_packet_touch` sets `updated_at` on every UPDATE. It does not touch `version`.
### Children
- `packet_work_orders`: `wo_base`, `lot`, `split`, `sub` (all text; the last three default to `''` so the key has no nulls), `seq`, `qty`, `target_hrs`, `op`, `description`, `visual_status`, `missing_runs` and `gone_at` (the feed's "work order gone" check), `sort`, `row_version`. Phase 3 adds `partspecs`, `rev` and `dwg`. Unique on (`packet_id`, `wo_base`, `lot`, `split`, `sub`, `seq`) with nulls not distinct, since migration 0004 (see "VISUAL and drawing caches").
- `packet_pages`: `n` (the packet page number, from `next_page_no`), `type`, `asset_id`, `pn` (page within the asset), `dwg`, `rev`, `co_id` (the change order that added it), `void_co_id`, `void_at`, `void_by`, `void_new_n` (a replaced page is voided, never deleted), `row_version`. Unique on (`packet_id`, `n`).
- `bom_lines`: `item`, `qty`, `part_id` (text), `description`, `dwg_ref`, `cl_item`, `length`, `material`, `heat`, `na`, `issued`, `loc`, `pulled`, `pulled_at`, `pulled_by`, `mtr_asset_id`, `src`, `vkey`, `whs`, `comm`, `cons`, `sub`, `grp`, `sort`, `row_version`. Unique on (`packet_id`, `item`).
- `bom_shorts`: `bom_line_id` (cascades with the line), `qty`, `qty_open`, `reason`, `ref`, `eta`, `note`, `opened_at`, `opened_by`, `closed_at`, `row_version`. At most one open short per line (unique on `bom_line_id` where `closed_at` is null).
- `short_receipts`: append-only. `short_id`, `packet_id`, `qty`, `heat`, `note`, `at`, `by`.
What the warehouse columns mean in practice (phase 4):
- `bom_lines.pulled`: the line is off the shelf. The first tick sets `pulled_at` and `pulled_by`, and sets `issued` to `qty` when it was empty. Unticking clears all three of `pulled`, `pulled_at` and `pulled_by`. The first tick while the warehouse stage is incoming or on hold also starts pulling (a `packet_wh` row with `auto` true).
- `bom_lines.issued`: how many pieces went out. Less than `qty` means the rest should be on a short.
- `bom_lines.loc`: where the parts were taken from, as the warehouse types it.
- `bom_lines.heat`: the heat number of the pieces issued, checked against the heat pattern. A receipt adds its heat to the line, so a line with parts from two heats reads `H1 / H2` (heats joined with " / ", no repeats).
- `bom_lines.na`: the part has no traceability. An N/A line needs no heat, takes no heat, and does not count toward the release gate.
- `bom_lines.mtr_asset_id`: the mill test report for the line, an `assets` row that must be a PDF, PNG or JPEG. Null clears it.
- A line "needs a heat" when it is not N/A, not a sub-assembly (`sub` is `Y`), is pulled, has `issued` above 0 and has no heat. The release gate counts these lines (see [POLICY.md](POLICY.md)).
- `bom_shorts`: one open short per line at most. Opening a short sets the line's `issued` to `qty` minus the short quantity and ticks it pulled. `qty` is what was short in total, `qty_open` what is still missing. `reason` is one of the closed short reasons, `ref` the PO or requisition, `eta` the expected date. Editing an open short keeps what was already received in `qty`. "Not short after all" closes it (`closed_at`) and sets `issued` to the line's `qty`. While a short is open, the line's heat, issued and location change only through the short.
- `short_receipts`: one row per delivery, never changed. Each receipt lowers `qty_open`, adds its pieces to the line's `issued` and its heat to the line's `heat`. The receipt that brings `qty_open` to 0 closes the short.
- `packet_wh`: `reason` is one of the closed hold reasons when the stage is `hold`, and `note` is up to 200 characters.
- `packet_stages`: one row per visit to a shop department. `dept_code`, `station_id`, `welder_id`, `welder_stamp`, `checklist` jsonb (key to `{done, at, by, stamp}`, written per key with `jsonb_set`), `filler`, `filler_lot`, `preheat`, `notes`, the clock (`clock_elapsed_ms`, `clock_running_since`, `clock_started_at`), `qc_snapshot`, `opened_at`, `closed_at`, `row_version`. At most one open stage per packet.
- `welds`: `weld_no` (from `next_weld_no`), `page`, `fx` and `fy` (fractions of the page), `wo_base`, `piece`, `items` int[], `type`, `size`, `process`, `stamp`, `welder_id`, `dept_code`, `stage_id`, `carried_from`, `at`, `by`, `row_version`, soft delete. Unique on (`packet_id`, `weld_no`) among live rows.
- `heat_tags`: `tag_no` (from `next_heat_no`), `page`, `item`, `fx`, `fy`, `carried_from`, `at`, `by`, `row_version`, soft delete. Unique on (`packet_id`, `tag_no`) among live rows.
- `heat_table_placements`: `page`, `x`, `y`, `s`, `by`, `at`. Unique on (`packet_id`, `page`), written as an upsert.
- `packet_wh`: the warehouse stage history. `stage` (wh_stage), `by`, `at`, `reason`, `note`, `auto`, `closed_at`. The current row has `closed_at` null and is unique per packet. A stage change closes it and inserts the next.
- `packet_transitions`: append-only. Every status change: `at`, `by`, `action`, `from_status`, `to_status`, `from_dept`, `to_dept`, `reason`, `signature_id`, `it_reason`. Replaces the old `hist` and `sentBack` lists.
- `qc_inspections`: `stage_id`, `sent_at`, `sent_by`, `book`, `spool`, `location`, `sent_note`, `checks` jsonb, `weld_exam` jsonb (weld_no to `{result, note}`), `nde_method`, `nde_result`, `nde_report`, `hardness`, `nace`, `inspector`, `stamp`, `notes`, `result`, `result_at`, `result_by`, `result_note`, `signature_id`, `row_version`, `closed_at`. At most one open inspection per packet.
- `qc_history`: append-only. `at`, `result`, `inspector`, `note`, `dept_code`, `snapshot` jsonb.
- `packet_seals`: the HMAC-SHA256 `hash` written at filing, `key_id`, `sealed_at`, `sealed_by`, `signature_id`, and `broken_at`, `broken_by`, `broken_reason` when IT reopens. At most one unbroken seal per packet.
- `packet_log`: append-only. The user-visible history: `at`, `by`, `by_name` and `role` (snapshots), `action`, `note`, `it_reason`, `signature_id`, `transition_id`. Indexed on (`packet_id`, `id`).
- `packet_visual_state`: one row per packet, written by the worker. `legs` jsonb, `ticket_hours`, `order_ref`, `observed_at`.
Every who column references `users (id)`. Every `signature_id` references `signatures (id)`.
## Change orders (phases 2 and 3)
- `change_orders`: `co_no` (from `next_co_no`; unique per packet), `origin` (co_origin), `review` (co_review), `minor`, `hold`, `held_by`, `held_at`, `ecn`, `reason`, `issued_*`, `approved_*` and `rejected_*` (each with a `*_signature_id`), `vis`, `waiting` and `staged` jsonb (feed-shaped), `target_fp`, `row_version`. At most one pending VISUAL change order per packet (unique on `packet_id` where origin is visual and review is pending), so the feed re-aims it instead of stacking new ones.
- `co_items`: the page changes. `old_page`, `old_dwg`, `old_rev`, `new_page`, `new_dwg`, `new_rev`, `added`.
- `co_required_depts`: (`co_id`, `dept_code`) primary key. The departments that must verify.
- `co_verifications`: append-only, keyed on (`co_id`, `dept_code`). `by` and `signature_id` are required, plus `at`, `note`, `it_reason`. A primary key violation means the department already verified. A verification cannot be withdrawn.
- View `v_open_cos` ports the old `openCOs()`: `packet_id`, `co_id`, `co_no`, `review`, `hold`, `pending_depts` (text[], the required departments without a verification, sorted by code). A row appears when review is approved and `pending_depts` is not empty, or when review is pending and `hold` is true. A pending change order without hold holds nothing. A rejected one never appears. The policy's CO hold and the boards' open-CO count read this view.
## Documents (phases 2, 4 and 6)
- `assets`: `id` uuid, `sha256` (indexed, for dedupe), `size`, `mime`, `original_name`, `storage_key` (the path under ASSETS_DIR, unique), `source` (`upload` by default), `source_ref` jsonb, `pages`, `uploaded_by`, `uploaded_at`, `packet_id` (the packet it was uploaded for, without a foreign key; `packet_pages` and `packet_docs` are the real links).
Asset kinds (phase 4). An upload names its kind in the `purpose` field, and `source_ref` keeps `{"purpose": kind}`. No purpose means `document`. The type is read from the content, never from the name or the browser.
| Kind | Types | Limit (setting) |
|---|---|---|
| drawing | PDF | 25 MB (`upload_max_bytes`) |
| mtr | PDF, PNG, JPEG | 25 MB (`upload_max_bytes`) |
| document | PDF, PNG, JPEG | 25 MB (`upload_max_bytes`) |
| bom | XLSX, CSV | 5 MB (`upload_max_bytes_bom`) |
`mime` is one of `application/pdf`, `image/png`, `image/jpeg`, `application/vnd.openxmlformats-officedocument.spreadsheetml.sheet` and `text/csv`. An XLSX is a zip that holds `[Content_Types].xml` and `xl/workbook.xml`; a plain zip is refused. A CSV is UTF-8 text with no NUL and no control character other than tab, CR and LF. An upload the virus scanner refuses is never stored, so it has no row.
- `packet_docs`: `asset_id`, `kind`, `title`, `added_at`, `added_by`, soft delete.
- `qc_docs`: `project_id`, `unit_id`, `type`, `rev`, `asset_id`, `auto`, `superseded_by` (the newer row), `added_at`, `added_by`.
- `itps`: one per unit (`unit_id` unique). `template` jsonb, frozen when the ITP is made. `status` (itp_status), `signoff_*` (with the signature and the signed PDF asset), `reopen_*`, `version`.
- `itp_lines`: `key` (unique per ITP), `result`, `by`, `at`, `ncr`, `note`, `signature_id`, `row_version`.
- `itp_rejects`: append-only. `line_key`, `ncr` (required), `note`, `at`, `by`. Every reject keeps its NCR id, even after the line is approved later.
- `itp_blocks`: `block_key` (unique per ITP), `signed_at`, `signed_by`, `signature_id`.
## Append-only set
These tables have the `ps_append_only` row and TRUNCATE triggers, and `ps_app` holds only SELECT and INSERT on them:
`audit_events`, `signin_attempts`, `signatures`, `project_renumbers`, `short_receipts`, `packet_transitions`, `qc_history`, `packet_log`, `itp_rejects`, `co_verifications`.
The app role is refused by privilege first. The migrate role owns the tables, so the trigger refuses it. The views `v_units` and `v_open_cos` are SELECT only for `ps_app`.
## Audit, settings, jobs (phases 1 and 3)
- `audit_events` (migration 0001): append-only. `at`, `user_id`, `session_id`, `role_snapshot` (`system` for the worker), `action`, `subject_type`, `subject_id`, `packet_id`, `detail` jsonb, `it_reason`, `ip`. A hash chain: `row_hash` is sha256 over `prev_hash` and the canonical JSON of the row, and `prev_hash` is the row before it. BRIN index on `at`.
- `settings` (migration 0001): `key` (text PK), `value` jsonb, `version`, `updated_at`, `updated_by`. Phase 3 adds the row `feed`, the feed settings as one JSON object (see [FEED.md](FEED.md), "Settings"). A missing key takes its default. A save bumps `version` and sends `ps_notify('ps_settings', ...)` so the worker reschedules.
Phase 3, migration 0004:
- `job_requests`: one row per "Run now". `job`, `requested_at`, `requested_by`, `note`, `claimed_run_id` and `claimed_at` (set by the run that took it). Index on `job` where `claimed_at` is null. The app inserts the row and NOTIFYs `ps_job_request` in the same transaction.
- `job_runs`: one row per run. `job`, `trigger` (schedule, request, chain, cli), `requested_by` uuid[] (the requests claimed), `started_at`, `finished_at`, `status` (running, ok, degraded, failed, skipped, abandoned), `counts` jsonb, `errors` jsonb (a list of redacted strings), `build_version`, `visual_target` (`"<server> / <database>"`), `host`. Index on (`job`, `id DESC`).
- `job_schedule`: one row per job (`job` PK). `next_run_at`, `interval_seconds`, `paused`, `updated_at`. Written by the worker after each run for the Admin tab. Seeded for `visual-feed`, `drawing-lookup` and `housekeeping`.
- `worker_heartbeat`: a single row (`id` = 1 by CHECK). `at`, `build`, `host`, `pid`, `running` text[] (the jobs running now). Upserted every 30 seconds.
## Notifications (phases 4 and 9)
Phase 4, migration 0005:
- `notification_events`: one row per alert. `id` bigserial, `kind` (text, for example `assets_quota`), `severity` (`info`, `warning` or `critical` by CHECK, default `warning`), `subject` (what the alert is about, for example `assets`), `detail` jsonb (for the quota alert, the volume usage: `total`, `used`, `free`, `pct`, `warn`), `created_at`, `resolved_at` (set when the condition clears), `acknowledged_at` and `acknowledged_by` (an Admin saw it). The unique index `notification_events_open` on (`kind`, `coalesce(subject, '')`) where `resolved_at` is null keeps one open alert per kind and subject, so a check that runs again does not stack rows. The table is not append-only: the app sets `resolved_at` and the acknowledgement. 0005 also grants `ps_app` USAGE and SELECT on every sequence in the schema.
Phase 9 builds the delivery on this table: rules, recipients, exclusions, outbox, sent log, state.
## VISUAL and drawing caches (phase 3)
Migration 0004. Every ID part is text (`wo_base`, `lot`, `split`, `sub`, part IDs, order IDs), with `''` in place of null in a key. `seq` and `piece` are int. Closed sets are text with a CHECK. The worker fills these tables. Screens read them, never VISUAL. [FEED.md](FEED.md) says how each is written.
- `visual_batches`: one row per feed run. `run_id`, `started_at`, `finished_at`, `server`, `database`, `site`, `since` and `until` (the window), `window_rows`, `tracked_keys`, `missing_keys`, `suspect`, `counts` jsonb.
- `visual_rows`: one row per work order operation (TYPE W), unique on (`wo_base`, `lot`, `split`, `sub`, `seq`). The contract fields: `res`, `dwg`, `rev`, `ps` (partspecs), `descr`, `qty`, `tgt`, `rel`, `want`, `st` (the status code F, R, U, C, X or `''`), `part`, `barcode`. `row_hash` (sha256 of the canonical row), `first_batch`, `last_batch` and `gone_batch` (set when a row in play was not read; cleared when it returns). Indexes on (`wo_base`, `lot`), on `dwg`, and on `rel` for rows not gone.
- `visual_materials`: one row per material line, unique on (`wo_base`, `lot`, `split`, `sub`, `seq`, `piece`). `part` (null for a line with no part), `qty`, `issued`, `whs`, `loc`, `st`, `istat`, `dwg`, `descr`, `comm`, `cons`, `mat_hash`, `first_batch`, `last_batch`, `gone_batch`. Index on (`wo_base`, `lot`).
- `visual_orders`: one row per customer order (`cust_order_id` PK). `status`, `customer_id`, `customer_name`, `customer_po_ref`, `order_date`, `desired_ship_date`, `promise_date`, `last_batch`.
- `visual_order_links`: `kind` (base or project), `key` (the base or the project number), `cust_order_id`, `rank` (open first, newest first), `lines` int[], `top_jobs` text[], `last_batch`. Unique on (`kind`, `key`, `cust_order_id`). Replaced for every key read.
- `visual_labor`: ticket hours per leg, unique on (`wo_base`, `lot`, `split`, `sub`, `seq`). `hours`, `tickets`, `last_ticket`, `observed_at`. No cost or pay column.
- `visual_row_changes`: what VISUAL changed, per batch. `batch_id`, the key and `seq`, `kind` (added, changed, gone, returned), `before` and `after` jsonb (the row as text values), `at`. Index on `batch_id`. `visual_material_changes` is the same plus `piece`.
- `visual_ignored_wos`: work orders kept out by the go-live date, unique on (`wo_base`, `lot`, `split`, `sub`). `rel`, `project`, `unit_number`, `res`, `dwg`, `first_seen_batch`, `last_seen_at`, and `pulled_in_at`, `pulled_in_by`, `pulled_in_reason` when Clerical pulls one in.
- `visual_resource_map`: `resource` (PK), `target` (`fold`, `ignore` or a two-digit department code, by CHECK), `confirmed`, `updated_at`, `updated_by`. Seeded from the old build's `visual_map.json`, every entry unconfirmed. "Electical" is spelled that way in VISUAL.
- `drawing_lookups`: one row per drawing number as VISUAL writes it (`dwg` PK, trimmed). The last answer: `checked_at`, `ok`, `rev` (released), `name`, `obj_id`, `obj_version`, `file_id`, `file_version`, `file_size`, `base` (H sheet), `draft_rev`, `draft_where`, `in_drafting`, `note`. Retry state: `fail_count`, `failing_since`, `next_check_at`. The fetched file: `asset_id`, `sha256`. `visual_rev_seen` is the VISUAL rev at the last lookup, so a rev change moves the drawing to the front of the queue. A rev mismatch keeps the staged M-Files copy here for approval.
Column additions in 0004:
- `packets`: `visual_cancelled_at` and `wo_gone_at`, so boards can filter on cancelled and gone without reading `flags`.
- `packet_work_orders`: `partspecs`, `rev` and `dwg`, the leg's own planner instructions, rev and drawing as VISUAL has them. The header keeps the highest rev. The unique key now includes `seq`: 0004 drops the 0003 unique on (`packet_id`, `wo_base`, `lot`, `split`, `sub`) and adds `UNIQUE NULLS NOT DISTINCT (packet_id, wo_base, lot, split, sub, seq)`, because the feed puts two operations of one work order on one packet. Hand drafts keep `seq` null, so they still have one row per key.
- `projects`: `order_date_visual`, `ship_date_visual`, `order_ref` jsonb (the chosen order and how it was chosen), and the Clerical pin `order_pin` (a customer order ID), `order_pin_by`, `order_pin_at`, `order_pin_reason`.
- `units`: `customer`, `po`, `customer_visual`, `po_visual`, `order_date_visual`, `ship_date_visual`, `order_ref`, and the same pin columns.
- `change_orders`: nothing new. `vis`, `waiting`, `staged`, `target_fp` and `minor` came in 0003.
## Versioning
- `packets.version` goes up by one in every write transaction that touches the packet or any of its children. The write pipeline does it explicitly (`UPDATE packets SET version = version + 1 ... RETURNING version`) after the caller's SQL. No trigger bumps it, so a read never does. It is the packet ETag and the NOTIFY payload.
- `projects.version`, `units.version` and `itps.version` play the same part for those rows.
- Child rows that are edited in place carry `row_version`. A PATCH sends `If-Match: <row_version>`. A mismatch is 409 `row_conflict` with the current row. The write bumps `row_version`.
- Every write takes `SELECT ... FOR UPDATE` on the packet row first. That lock replaces the old 8-second lease and serialises the counters: `next_weld_no`, `next_heat_no`, `next_page_no` and `next_co_no` hand out the next number with `UPDATE packets SET next_weld_no = next_weld_no + 1 RETURNING next_weld_no - 1`, so two concurrent adds never get the same number.
- Transitions do not check `version`. They compare and set on `expect: {status, dept}`, so a weld edit never blocks a send.
Reference in New Issue View Git Blame Copy Permalink