spencerm/PacketStation
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 2: core domain, policy engine, packet API, Clerical screen

Browse Source 
Migration 0003: projects, aliases, renumbers, units, packets and every child table,
change orders, assets, documents, ITPs, station limits, department weld flag.
Policy package: Principal, PacketState, Action rules, custody, preconditions,
CO hold, order of checks, can/blocked/holder on reads. docs/POLICY.md with the
tightenings table for Seth. Write pipeline: FOR UPDATE, CAS on status and dept,
If-Match row versions, server-assigned numbers, field CAS, version bump, packet
log, audit, NOTIFY, signatures, IT reasons, filed lock and HMAC seal.
API: /api/packets (header, transitions, work orders, pages, docs, move, remove,
BOM, shorts, welds, heat tags, heat table, stage, checklist, clock, warehouse
stage, Q.C., change orders, log, seal), /api/projects, /api/units, /api/assets,
/_blob. Frontend: Clerical home, projects and units, packet page with pdf.js
and QR codes, hand draft, heat map, send to warehouse, department or Q.C.
Tests: policy matrix, API, concurrency, schema, projects; Playwright Clerical
draft scenario. Verified on docker-1 with the CI stack and the Samba DC.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
spencerm
2026-10-16 09:31:45 -06:00
co-authored by Claude Fable 5.1
parent 97b0973130
commit 37d9a0cb50
83 changed files with 16490 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/POLICY.md
+191 -11
View File
@@ -1,39 +1,219 @@
# Packet Station policy
Status: planned, phase 2. This is a stub. It lists what the approved plan says this document will contain. See [PLAN.md](PLAN.md), "Policy port and concurrency rules".
Who may do what to a packet, and when. This file is written from the code in `server/packetstation/policy/`. When they disagree, the code is the rule and this file is out of date.
## Purpose
Who may do what to a packet, and when. A Python port of the old `guard.js`, run only on the server. Pure functions, no I/O.
The policy is a Python port of the old `guard.js`. It runs only on the server. Every packet write calls `policy.check()` inside the write transaction, under the packet row lock. Every packet read returns `can`, `blocked` and `holder` from the same rules, so the screens only show what the server will accept.
The package is pure functions with no I/O:
- `actions.py`: the `Action` enum, the `Who` predicates, the `Rule` record and the `RULES` table.
- `principal.py`: `Principal`, `PacketState`, `principal_from_roles()`.
- `custody.py`: `holds()`, `works_on()`, `holder_name()` and the Who predicates.
- `preconditions.py`: the named preconditions and their messages.
- `engine.py`: `check()`, `evaluate()`, `can()` and `Decision`.
## Roles
Every role comes from an AD group: Admin; Clerical, Warehouse and Q.C. with a member and a Supervisor level; Project Management (view only); and 14 shop departments with Worker, Lead and Supervisor. A person's roles are the union of their groups.
Every role comes from an AD group (see docs/SECURITY.md). The role strings are `admin`, `clerical`, `clerical_supervisor`, `warehouse`, `warehouse_supervisor`, `qc`, `qc_supervisor`, `pm` and `shop:<code>:<worker|lead|supervisor>`. A person's roles are the union of their groups.
`principal_from_roles(user_id, roles, station_limits=(), session_kind="password")` builds the policy's `Principal`:
| Field | Meaning |
|---|---|
| `admin` | In PS-Admins. Admin is not membership. It is the bypass rule below. |
| `support` | The codes among CL, WH and Q.C. the person belongs to, at any level. A supervisor role implies membership. |
| `support_sup` | The codes among CL, WH and Q.C. where the person is Supervisor. |
| `shop` | Shop department code to level: 1 worker, 2 lead, 3 supervisor. The highest level wins. |
| `station_limits` | Station ids a worker is limited to. Empty means any station. |
| `pm` | Project Management. PM can read every packet and change nothing. |
| `session_kind` | `password` or `pin`. |
| `system` | `visual-feed` for the worker. Never set from a session. |
A role string that does not parse is ignored.
## Custody
Only whoever holds the packet may change it. Draft: Clerical. Warehouse: Warehouse. Ready and review: the department lead or above. Assigned and in progress: the lead or above, or the assigned worker. Q.C.: Q.C. Complete: Clerical. Filed: nobody.
Only whoever holds the packet may change it. Everyone else can open it and follow it.
## Actions and transitions
| Status | Holder |
|---|---|
| draft | Clerical |
| warehouse | Warehouse |
| ready, review | Lead or supervisor of the packet's department |
| assigned, in_progress | Lead or supervisor of the department, or a worker who works on it |
| qc | Q.C. |
| complete | Clerical (data book) |
| filed | Nobody |
An Action enum with one rule per action: who, from and to status, department change, change-order hold, preconditions, signature meaning, and whether a reason is required.
`works_on(principal, state)`: a member of the packet's department. Leads and supervisors always pass. A worker passes when they have no station limits, or the limits hold the packet's station. START and SUBMIT also need the assigned welder for a worker (see the tightenings).
`holder_name(state, dept_name, station_name)` gives the text for messages: "Clerical", "Warehouse", "Q.C.", "Clerical (data book)", "the data book (filed)", else the department name, plus " · <station>" while the packet is at a station.
Statuses in workflow order: draft, warehouse, ready, assigned, in_progress, review, qc, complete, filed. SHOP is ready, assigned, in_progress and review. ACTIVE is assigned and in_progress.
## Who
| Who | Passes when |
|---|---|
| CL, WH, QC | A member of that support department, any level |
| CL_SUP, WH_SUP, QC_SUP | Supervisor of that support department |
| CL_OR_QC | A member of Clerical or Q.C. |
| HOLDER | Holds the packet (custody above) |
| DEPT_LEAD | Lead or supervisor in the packet's department |
| DEPT_SUP | Supervisor in the packet's department |
| WORKS_ON | Lead or above in the department, or a worker in the department who is the assigned welder and whose station limits allow the station |
| CO_VERIFIER | Supervisor of the department named in the request: a shop Supervisor for a shop code, WH or Q.C. Supervisor for WH or QC. Clerical never verifies. |
| ADMIN | In PS-Admins |
| SYSTEM | The worker (`system` is set) |
## Rule table
`from` is the set of statuses the action starts from. "any" means any status, with the filed lock applied separately. `co hold` means an open change order holds the step.
| Action | Who | From | To | Dept | CO hold | Preconditions | Signature | Reason |
|---|---|---|---|---|---|---|---|---|
| send_to_warehouse | CL | draft | warehouse | no | | heat_map_answered, drawing_attached | | |
| release_to_shop | CL | draft | ready | allowed | | heat_map_answered, drawing_attached, target_dept_given | | |
| send_draft_to_qc | CL | draft | qc | no | | heat_map_answered, drawing_attached | | |
| wh_send_back | WH | warehouse | draft | no | | reason_given | | yes |
| return_to_clerical | DEPT_LEAD | ready, only when wh_skipped | draft | no | | reason_given | | yes |
| wh_release | WH | warehouse | ready | no | yes | bom_all_pulled_or_short | | |
| send_back_to_warehouse | DEPT_LEAD | ready | warehouse | no | | reason_given | | yes |
| assign | DEPT_LEAD | ready | assigned | no | | welder_set | | |
| unassign | DEPT_LEAD | assigned | ready | no | | | | |
| pull_back | DEPT_LEAD | in_progress | ready | no | | | | |
| start | WORKS_ON | assigned | in_progress | no | | | | |
| submit | WORKS_ON | assigned, in_progress | review | no | yes | checklist_complete, welds_recorded, filler_set | Submit work to the supervisor | |
| return_to_station | DEPT_SUP | review | assigned | no | | reason_given | | yes |
| send_to_qc | DEPT_SUP | review | qc | no | yes | qc_book_spool_location | Send to Q.C. | |
| route_from_review | DEPT_SUP | review | ready | required | yes | target_dept_given | Route to another department | |
| qc_reject | QC | qc | review | no | | reason_given | Q.C. reject | yes |
| qc_accept | QC | qc | complete | no | yes | qc_missing_empty, route_given | Q.C. accept | |
| qc_accept_and_route | QC | qc | ready | required | yes | qc_missing_empty, target_dept_given | Q.C. accept and route | |
| file | CL | complete | filed | no | yes | vclosed_if_databook | File in the data book | |
| admin_reopen | ADMIN | filed (filed_ok) | complete | no | | reason_given | Reopen a filed packet | yes |
| edit_header | CL | draft | | | | | | |
| edit_drawing_ref | CL | any | | | | | | |
| edit_dept | CL | draft | | | | | | |
| move_unit | CL | any | | | | | | |
| edit_wos | CL | draft | | | | | | |
| edit_bom | HOLDER | draft, warehouse, SHOP | | | | | | |
| wh_receive_short | WH | any | | | | short_open | | |
| edit_heats | HOLDER | warehouse, SHOP | | | | | | |
| edit_heat_table | HOLDER | warehouse, SHOP | | | | | | |
| edit_welds | HOLDER | SHOP | | | | | | |
| edit_checklist | HOLDER | SHOP | | | | | | |
| edit_stage_fields | HOLDER | SHOP | | | | | | |
| clock | HOLDER | SHOP | | | | | | |
| edit_qc | QC | qc | | | | | | |
| set_wh_stage | WH | warehouse | | | | | | |
| edit_document_set | CL | any | | | | | Attach a drawing by hand (attach only) | |
| doc_add | CL_OR_QC | any | | | | | | |
| doc_remove | CL_OR_QC | any | | | | | | |
| rev_approve | CL_SUP | draft | | | | reason_given | Approve the M-Files revision | yes |
| set_vclosed | CL | complete, only when routed_to is databook | | | | | | |
| set_databook_at | CL_OR_QC | any (filed_ok) | | | | | | |
| remove | CL | draft | | | | reason_given | | yes |
| co_issue | CL | any | | | | | Issue a change order | |
| co_approve | CL | any | | | | | Approve a change order | |
| co_reject | CL | any | | | | reason_given | Reject a change order | yes |
| co_hold_toggle | CL | any | | | | | | |
| co_verify | CO_VERIFIER | any | | | | | Verify a change order for the department | |
| itp_edit_line | QC | any | | | | | | |
| itp_block_sign | QC | any | | | | | Sign an ITP block | |
| itp_sign_off | QC | any | | | | | Sign off the ITP | |
| itp_reopen | QC_SUP (Admin through the bypass) | any | | | | reason_given | | yes |
| vis_update_draft | SYSTEM | draft | | | | | | |
| vis_retire_draft | SYSTEM | draft | | | | | | |
| vis_open_co | SYSTEM | any but draft and filed | | | | | | |
| vis_auto_release | SYSTEM | draft | warehouse | | | heat_map_answered, drawing_attached | | |
The `vis_*` actions are system only. People can never do them, and the worker can do nothing else.
## Preconditions
Named predicates, checked on the server and never bypassed, including by Admin.
Each precondition is a named predicate over the packet state and the request input. They are never bypassed, Admin included. The first one that fails gives the message.
| Name | Passes when | Message |
|---|---|---|
| heat_map_answered | heat_map is yes or no | Say whether this packet needs a heat map. |
| drawing_attached | a drawing is attached | Attach the drawing first. |
| bom_all_pulled_or_short | every line not marked n/a is pulled or has an open short | Pull every material line or mark it short first. |
| short_open | at least one BOM line has an open short (the old whCanReceive) | No part is short on this packet. |
| welder_set (input) | the request gives station_id and welder_id | Pick a station and a welder. |
| checklist_complete | every key of the department checklist is done | Finish the checklist first. |
| welds_recorded | at least one weld, in a weld department | Record at least one weld first. |
| filler_set | filler metal entered, in a weld department | Enter the filler metal first. |
| qc_book_spool_location (input) | book and spool confirmed and a location given | Confirm the book and the spool, and give the location. |
| qc_missing_empty | the Q.C. record has nothing missing | Q.C. still needs: (the list). |
| reason_given (input) | a reason of at least 8 characters after cleaning | Give a reason of at least 8 characters. |
| vclosed_if_databook | routed to Shipping, or closed in VISUAL | Close the job in VISUAL first. |
| co_staged_if_dwg_change (input) | no drawing change, or the new drawing is staged (phase 3) | Stage the new drawing for the change order first. |
| route_given (input) | routed_to is databook or shipping | Choose Data Book or Shipping. |
| target_dept_given (input) | a shop department different from the packet's; from a draft any shop department | Choose a department. / Choose a different department. |
"(input)" marks a precondition that reads request input, not packet state. `can()` skips those.
## Order of checks
Removed, wrong status, change-order hold, admin bypass of custody with an IT reason, filed lock, who and custody, then preconditions.
`check(principal, state, action, *, extra={}, it_reason=None) -> Decision` runs these in this order and stops at the first failure:
1. **Removed.** A removed packet refuses everything, Admin included. Code `removed`.
2. **Status.** The status must be in the rule's from set. RETURN_TO_CLERICAL also needs wh_skipped. SET_VCLOSED also needs routed_to databook. An unknown status fails here too. Code `wrong_status`.
3. **Change-order hold.** A held action refuses while an approved change order waits for verification, or a pending one is on hold. Admin included. Code `co_hold`, with the hold text.
4. **System.** A system-only action refuses every person (`system_only`). The system principal passes only system-only actions (`forbidden`).
5. **Admin bypass.** An admin who is not allowed by the rule's Who passes who and custody when an IT reason of at least 8 characters is given. Without one the code is `it_reason_required`. SET_DATABOOK_AT needs no reason. The decision is marked `admin_bypass`, and the write marks the log row as an IT edit.
6. **Filed lock.** A filed packet refuses everything except SET_DATABOOK_AT and ADMIN_REOPEN, Admin included. Code `filed`: "This packet is filed and locked. IT can reopen it if a correction is needed."
7. **Who and custody.** Code `forbidden`. For a step: "Your access doesn't allow this step." For an edit: "Your access doesn't allow changing (what) on this packet. It's with (holder) right now." PM fails here for everything.
8. **Preconditions,** in rule order, never bypassed. Code `precondition` with the message, or `reason_required` when the missing piece is the rule's reason, so the screen can ask for one.
`Decision` carries `ok`, `code`, `why`, `admin_bypass`, `signature` (the meaning text when the rule needs one) and, in read mode only, `reasons`.
The packet API answers 403 with the code for most failures, and 400 for `it_reason_required` and `reason_required`.
## Signatures
Which actions need a PIN or domain password signature, and what the signature records.
These actions need a signature: submit, send_to_qc, route_from_review, qc_reject, qc_accept, qc_accept_and_route, file, admin_reopen, edit_document_set (attach only), rev_approve, co_issue, co_approve, co_reject, co_verify, itp_block_sign and itp_sign_off. The rule's signature text is the meaning that is signed.
The request carries `sign: {factor, secret}`. The write pipeline calls the phase 1 helper `packetstation.auth.signatures.sign()` inside the same transaction, after the policy passes. It checks the PIN or domain password against the session's own user. A PIN works in any session. It logs the attempt and applies the five-strike PIN disable and the three-strike session end. It writes an append-only `signatures` row with the user, session, account, display name, session kind, factor, action, meaning, packet, subject and a sha256 over the canonical JSON of what was signed (the action, the packet id, the expected state and the request fields). The `packet_transitions` and `packet_log` rows point at it.
## Concurrency
Compare-and-set on workflow steps (409 "someone else already took this step"), row versions on child rows, server-assigned numbers, append-only tables.
- **Workflow steps** are compare-and-set on `expect: {status, dept}` under the packet row lock (`SELECT ... FOR UPDATE`). When the packet moved on, the answer is 409 `stale_step` "Someone else already took this step." with the current status, department and version. There is no packet version check on a step, so a weld edit never blocks a send. Side effects run in the same transaction.
- **Keyed child rows** (BOM lines, Q.C. inspections, stage fields, welds, heat tags) take `If-Match: <row_version>`. A mismatch is 409 `row_conflict` with the current row.
- **Numbers** for welds, heat tags, pages and change orders come from counters on the packet row under the row lock, so concurrent adds never collide.
- **Header scalars** use field-level compare-and-set: the body sends `set` and `base`. A key whose base differs from the current value is 409 `field_conflict` with the current value. A key with no base is written as is.
- **Work orders** are a draft-only full replace with If-Match on the packet version.
- **Shorts and change-order actions** are state compare-and-set. Checklist keys are written one key at a time. The heat table position is an idempotent upsert. A change-order verification is an insert, so a second one is "already verified".
- **Append-only tables** never conflict: packet_transitions, packet_log, qc_history, short_receipts, project_renumbers, itp_rejects and co_verifications. Triggers refuse UPDATE, DELETE and TRUNCATE, and the app role has no grant for them.
## Reads: can, blocked and holder
Every packet read returns:
- `can`: the actions that pass now. Input-only preconditions are skipped, because they are what the user gives with the step.
- `blocked`: the actions that pass who and custody but fail a state precondition or the change-order hold, with every reason. The screen shows the control disabled with the reason.
- `holder`: `holder_name()` for the packet.
`can(principal, state, assume_it_reason=True) -> (list[Action], dict[Action, list[str]])`. For an admin the IT reason is assumed, so the list shows what the admin may do once they give one. `evaluate(principal, state, action)` returns the read-mode decision with `admin_bypass` set, so the screen knows to ask for the reason. System-only actions are never listed. CO_VERIFY needs a department in the request, so it is not listed; the change order list says which departments the person can verify.
## Tightenings against the old build
The old policy is looser than the old UI in places. Each tightening is listed here so Seth can veto it. Known so far: Clerical lead actions move to Clerical Supervisor; packets routed to Shipping get the Clerical close and file path; SET_DATABOOK_AT is restricted to Clerical and Q.C. by default.
The old policy was looser than the old screens in places. The rebuild follows the screens where they were stricter, and the README where it states the intent. Each row is a change Seth can veto.
| What the old build allowed | What the rebuild does | Why | Can be vetoed |
|---|---|---|---|
| A department lead could write welds, BOM, heat tags, the checklist, stage fields and the clock through the policy whenever the packet was in a shop status. | These edits need the holder of the packet (the holder rule, applied as the old screens did). A lead of another department cannot. | The screens only offered these to the holder. The policy now matches. | Yes |
| Any worker in the department within their station limits could start or submit a packet (`worksOn` ignored who was assigned). | START and SUBMIT need the assigned welder for a worker. Leads and supervisors still can. | The README says only the assigned worker. | Yes |
| A status change could carry any other field in the same save. | A transition carries only its own fields (the step's inputs). Other changes go through their own edit actions and rules. | One rule per write keeps the check honest. | No |
| Clerical "lead" actions (approve an M-Files rev mismatch, move back from VISUAL review) were open to any Clerical member with a lead flag. | REV_APPROVE and moving back from VISUAL review need Clerical Supervisor (PS-Clerical-Supervisor). | There is no Clerical lead AD group. | Yes |
| A packet routed to Shipping had no path to filed. | Complete packets routed to Shipping get the same Clerical close and file path. Only the Data Book route needs the VISUAL close before filing. | Shipping packets were stuck at complete. | Yes |
| Anyone could set the data book date, even on a filed packet. | SET_DATABOOK_AT is Clerical or Q.C. only (open question 17). It stays allowed on a filed packet. | The data book date is a Clerical and Q.C. record. | Yes |
| Change-order verifications were stored in the packet document, with no rule against changing them later. | Verifications are insert-only and cannot be withdrawn (open question 18). | A signed verification is a record. | Yes |
| The old policy did not look at the BOM when the warehouse released a packet. | WH_RELEASE needs every BOM line not marked n/a to be pulled or to have an open short. | A packet should reach the shop with its material, or with the shorts on record. | Yes |
| A hand draft could leave Clerical with no drawing (the packet PDF upload stood in for it). | A hand draft needs a drawing attached before any send from Clerical. | There is no packet PDF upload in the rebuild. | Yes |
| Attaching drawing pages by hand needed no signature. | Attaching a drawing by hand needs a signature ("Attach a drawing by hand"). | It replaces what M-Files would supply, so a person stands behind it. | Yes |
| Admin could write anything on a filed packet with an IT reason. | Admin cannot write a filed packet except through ADMIN_REOPEN (with a reason and a signature), and SET_DATABOOK_AT. | A filed packet is sealed. A change breaks the seal on purpose, on record. | No |
| Admin passed every check after the change-order hold, so an admin could make a step from any status. | Admin goes through the same status check. The bypass covers who and custody only. Preconditions still apply. | Admin fixes data, not the workflow order. | Yes |
docs/SCHEMA.md
+86 -8
View File
@@ -1,6 +1,6 @@
# Packet Station schema
Status: phase 1 built (identity and organisation seed). Phases 2 and 3 add the rest. The DDL is in `server/packetstation/migrations/`; this file explains it. See [PLAN.md](PLAN.md), "Schema".
Status: phases 1 and 2 built (identity, organisation, projects, packets, change orders, documents). Phase 3 adds the job and VISUAL tables, phase 9 the notification tables. The DDL is in `server/packetstation/migrations/`; this file explains it. See [PLAN.md](PLAN.md), "Schema".
## Conventions
@@ -26,27 +26,101 @@ Append-only tables use the `ps_append_only` trigger from 0001 and revoke UPDATE,
## Organisation (phase 1 seed, phase 2 completes)
- `departments`: `code` (text PK, `CL`, `WH`, `QC`, `PM`, `01` to `14`), `slug` (the AD group short name, unique), `name`, `kind`, `sort`, `active`, `supervisor_group` (false only for PM). A trigger refuses changes to `code` and `slug`. Seeded with the 18 departments from the brief and the old `depts.js`.
- `departments`: `code` (text PK, `CL`, `WH`, `QC`, `PM`, `01` to `14`), `slug` (the AD group short name, unique), `name`, `kind`, `sort`, `active`, `supervisor_group` (false only for PM). A trigger refuses changes to `code` and `slug`. Seeded with the 18 departments from the brief and the old `depts.js`. Migration 0003 adds `weld` (bool, default false), true for 03, 04, 05, 06, 07 and 10. A weld department gets the welding checklist, and submit there needs at least one weld and a filler.
- `stations`: (`dept_code`, `name`, `sort`, `active`), seeded with the old station lists (Stall 1 to 6, Station 1 to 4, Hydro Bay 1 and 2, Blast Booth, Paint Booth 1 and 2).
- `user_station_limits`: phase 2.
- `user_station_limits`: (`user_id`, `station_id`) primary key. A worker with rows here may only work on packets at those stations. No rows means any station in the department. The rows go when the user row goes.
The 50 AD group names are derived from `departments` plus `PS-Admins`: `PS-<slug>` and `PS-<slug>-Supervisor` for support departments, `PS-<code>-<slug>-<Worker|Lead|Supervisor>` for shop departments. `packetstation ad-groups --list` prints them.
## Enums (phase 2, migration 0003)
Declaration order matters for `packet_status`: it is the workflow order, so `status >= 'qc'` works in SQL.
| Enum | Values in order |
|---|---|
| `packet_status` | draft, warehouse, ready, assigned, in_progress, review, qc, complete, filed |
| `wh_stage` | incoming, pulling, hold, pulled |
| `co_review` | pending, approved, rejected |
| `co_origin` | visual, drawing, manual |
| `source_kind` | visual, hand |
| `project_status` | open, hold, closed |
| `route_kind` | databook, shipping |
| `drawing_state` | none, needed, attached, rev_mismatch, not_released, not_found, lookup_failed, hand |
| `itp_status` | open, signed, reopened |
Smaller closed sets are text with a CHECK: `projects.source` and every `*_src` (visual, hand), `project_aliases.kind` (renumber, unit_renumber, import), `packet_pages.type` (drawing, info, overlay), `bom_lines.src` (visual, hand), `qc_inspections.result` (accepted, rejected), `itp_lines.result` (approve, reject, na). Null means unset in each.
## Projects and units (phase 2)
projects, project_aliases, project_renumbers, units.
- `projects`: `number` (text, unique among live rows on `upper(number)`), `name`, `customer`, `po`, `due`, `status` (project_status), `notes`, `source` (hand or visual), `needs_details` (the feed made it and Clerical has not filled it in), `version` for field CAS. Order-derived values carry provenance: `customer_src`, `po_src`, `order_date_src` and `ship_date_src` say visual or hand, and `customer_visual` and `po_visual` keep what VISUAL said. The feed never overwrites a hand value. Soft delete.
- `units`: `project_id`, `lot` (text), `name`, `due`, `status`, `notes`, the same `*_src` companions, `order_date`, `ship_date`, `version`. A CHECK keeps `lot` to 1 to 3 upper-case letters or digits, and a lot of digits only is exactly 3 long (`001`, `7A`). Unique among live rows on (`project_id`, `lot`). Soft delete.
- `project_aliases`: `alias` (text PK, upper-cased by CHECK), `project_id`, `unit_id` (null for a project number), `kind` (renumber, unit_renumber, import), `created_at`. Every current and old number of every project and unit is here, so a lookup by any number resolves, and a renumber cannot reuse a number that was ever taken.
- `project_renumbers`: append-only. `at`, `by`, `project_id`, `unit_id`, `old_number`, `new_number`, `reason`.
- View `v_units`: every `units` column plus `number` (`project.number || lot`, so `L999` and `001` make `L999001`), `project_number`, `project_name`, `project_status` and `due_effective` (the unit's due, else the project's).
## Packets (phase 2)
The packet header and its child tables: work orders, pages, BOM lines, shorts, welds, heat tags, stages, warehouse, transitions, Q.C., seals, log.
### Header
`packets` is the header. The columns that matter:
- Placement: `unit_id`, `dept_code`, `op`, `title`, `customer`, `drawing`, `rev`, `wps`.
- Workflow: `status` (packet_status), `status_at`, `station_id`, `welder_id` and `welder_stamp` (a snapshot taken at assign), `for_sup_id`, `wh_skipped` (true when Clerical sent the draft straight to a department or to Q.C.), `wh_sent_at`, `routed_to` (route_kind, set at Q.C. accept), `return_note` (the reason shown after a send-back), `vclosed_at` and `vclosed_by`, `databook_at`.
- Preconditions: `heat_map` is nullable on purpose. Null means not answered yet, and the warehouse step needs an answer.
- Source: `source_kind` (visual or hand), `source` jsonb, `visual_key` (the feed's match key), `visual_rel`, `visual_want`, `drawing_state`, `flags` text[], `visual_fp_applied` (the fingerprint the packet matches), `visual_closed_at`, `visual` jsonb (system-written sub-keys).
- Counters: `next_weld_no`, `next_heat_no`, `next_page_no` and `next_co_no`, all starting at 1. See "Versioning".
- `version`, `created_at`, `created_by`, `updated_at`, soft delete.
Indexes: `packets_visual_key` is unique on `visual_key` where the key is set and the row is live. A removed packet keeps its key, so two removed rows may share one, but only one live row may hold it. Also `(status)` for live rows, `(unit_id)` and `(dept_code, status)` for the boards.
The trigger `ps_packet_touch` sets `updated_at` on every UPDATE. It does not touch `version`.
### Children
- `packet_work_orders`: `wo_base`, `lot`, `split`, `sub` (all text; the last three default to `''` so the key has no nulls), `seq`, `qty`, `target_hrs`, `op`, `description`, `visual_status`, `missing_runs` and `gone_at` (the feed's "work order gone" check), `sort`, `row_version`. Unique on (`packet_id`, `wo_base`, `lot`, `split`, `sub`).
- `packet_pages`: `n` (the packet page number, from `next_page_no`), `type`, `asset_id`, `pn` (page within the asset), `dwg`, `rev`, `co_id` (the change order that added it), `void_co_id`, `void_at`, `void_by`, `void_new_n` (a replaced page is voided, never deleted), `row_version`. Unique on (`packet_id`, `n`).
- `bom_lines`: `item`, `qty`, `part_id` (text), `description`, `dwg_ref`, `cl_item`, `length`, `material`, `heat`, `na`, `issued`, `loc`, `pulled`, `pulled_at`, `pulled_by`, `mtr_asset_id`, `src`, `vkey`, `whs`, `comm`, `cons`, `sub`, `grp`, `sort`, `row_version`. Unique on (`packet_id`, `item`).
- `bom_shorts`: `bom_line_id` (cascades with the line), `qty`, `qty_open`, `reason`, `ref`, `eta`, `note`, `opened_at`, `opened_by`, `closed_at`, `row_version`. At most one open short per line (unique on `bom_line_id` where `closed_at` is null).
- `short_receipts`: append-only. `short_id`, `packet_id`, `qty`, `heat`, `note`, `at`, `by`.
- `packet_stages`: one row per visit to a shop department. `dept_code`, `station_id`, `welder_id`, `welder_stamp`, `checklist` jsonb (key to `{done, at, by, stamp}`, written per key with `jsonb_set`), `filler`, `filler_lot`, `preheat`, `notes`, the clock (`clock_elapsed_ms`, `clock_running_since`, `clock_started_at`), `qc_snapshot`, `opened_at`, `closed_at`, `row_version`. At most one open stage per packet.
- `welds`: `weld_no` (from `next_weld_no`), `page`, `fx` and `fy` (fractions of the page), `wo_base`, `piece`, `items` int[], `type`, `size`, `process`, `stamp`, `welder_id`, `dept_code`, `stage_id`, `carried_from`, `at`, `by`, `row_version`, soft delete. Unique on (`packet_id`, `weld_no`) among live rows.
- `heat_tags`: `tag_no` (from `next_heat_no`), `page`, `item`, `fx`, `fy`, `carried_from`, `at`, `by`, `row_version`, soft delete. Unique on (`packet_id`, `tag_no`) among live rows.
- `heat_table_placements`: `page`, `x`, `y`, `s`, `by`, `at`. Unique on (`packet_id`, `page`), written as an upsert.
- `packet_wh`: the warehouse stage history. `stage` (wh_stage), `by`, `at`, `reason`, `note`, `auto`, `closed_at`. The current row has `closed_at` null and is unique per packet. A stage change closes it and inserts the next.
- `packet_transitions`: append-only. Every status change: `at`, `by`, `action`, `from_status`, `to_status`, `from_dept`, `to_dept`, `reason`, `signature_id`, `it_reason`. Replaces the old `hist` and `sentBack` lists.
- `qc_inspections`: `stage_id`, `sent_at`, `sent_by`, `book`, `spool`, `location`, `sent_note`, `checks` jsonb, `weld_exam` jsonb (weld_no to `{result, note}`), `nde_method`, `nde_result`, `nde_report`, `hardness`, `nace`, `inspector`, `stamp`, `notes`, `result`, `result_at`, `result_by`, `result_note`, `signature_id`, `row_version`, `closed_at`. At most one open inspection per packet.
- `qc_history`: append-only. `at`, `result`, `inspector`, `note`, `dept_code`, `snapshot` jsonb.
- `packet_seals`: the HMAC-SHA256 `hash` written at filing, `key_id`, `sealed_at`, `sealed_by`, `signature_id`, and `broken_at`, `broken_by`, `broken_reason` when IT reopens. At most one unbroken seal per packet.
- `packet_log`: append-only. The user-visible history: `at`, `by`, `by_name` and `role` (snapshots), `action`, `note`, `it_reason`, `signature_id`, `transition_id`. Indexed on (`packet_id`, `id`).
- `packet_visual_state`: one row per packet, written by the worker. `legs` jsonb, `ticket_hours`, `order_ref`, `observed_at`.
Every who column references `users (id)`. Every `signature_id` references `signatures (id)`.
## Change orders (phases 2 and 3)
change_orders, co_items, co_required_depts, co_verifications.
- `change_orders`: `co_no` (from `next_co_no`; unique per packet), `origin` (co_origin), `review` (co_review), `minor`, `hold`, `held_by`, `held_at`, `ecn`, `reason`, `issued_*`, `approved_*` and `rejected_*` (each with a `*_signature_id`), `vis`, `waiting` and `staged` jsonb (feed-shaped), `target_fp`, `row_version`. At most one pending VISUAL change order per packet (unique on `packet_id` where origin is visual and review is pending), so the feed re-aims it instead of stacking new ones.
- `co_items`: the page changes. `old_page`, `old_dwg`, `old_rev`, `new_page`, `new_dwg`, `new_rev`, `added`.
- `co_required_depts`: (`co_id`, `dept_code`) primary key. The departments that must verify.
- `co_verifications`: append-only, keyed on (`co_id`, `dept_code`). `by` and `signature_id` are required, plus `at`, `note`, `it_reason`. A primary key violation means the department already verified. A verification cannot be withdrawn.
- View `v_open_cos` ports the old `openCOs()`: `packet_id`, `co_id`, `co_no`, `review`, `hold`, `pending_depts` (text[], the required departments without a verification, sorted by code). A row appears when review is approved and `pending_depts` is not empty, or when review is pending and `hold` is true. A pending change order without hold holds nothing. A rejected one never appears. The policy's CO hold and the boards' open-CO count read this view.
## Documents (phases 2, 4 and 6)
assets, packet_docs, qc_docs, ITPs.
- `assets`: `id` uuid, `sha256` (indexed, for dedupe), `size`, `mime`, `original_name`, `storage_key` (the path under ASSETS_DIR, unique), `source` (`upload` by default), `source_ref` jsonb, `pages`, `uploaded_by`, `uploaded_at`, `packet_id` (the packet it was uploaded for, without a foreign key; `packet_pages` and `packet_docs` are the real links).
- `packet_docs`: `asset_id`, `kind`, `title`, `added_at`, `added_by`, soft delete.
- `qc_docs`: `project_id`, `unit_id`, `type`, `rev`, `asset_id`, `auto`, `superseded_by` (the newer row), `added_at`, `added_by`.
- `itps`: one per unit (`unit_id` unique). `template` jsonb, frozen when the ITP is made. `status` (itp_status), `signoff_*` (with the signature and the signed PDF asset), `reopen_*`, `version`.
- `itp_lines`: `key` (unique per ITP), `result`, `by`, `at`, `ncr`, `note`, `signature_id`, `row_version`.
- `itp_rejects`: append-only. `line_key`, `ncr` (required), `note`, `at`, `by`. Every reject keeps its NCR id, even after the line is approved later.
- `itp_blocks`: `block_key` (unique per ITP), `signed_at`, `signed_by`, `signature_id`.
## Append-only set
These tables have the `ps_append_only` row and TRUNCATE triggers, and `ps_app` holds only SELECT and INSERT on them:
`audit_events`, `signin_attempts`, `signatures`, `project_renumbers`, `short_receipts`, `packet_transitions`, `qc_history`, `packet_log`, `itp_rejects`, `co_verifications`.
The app role is refused by privilege first. The migrate role owns the tables, so the trigger refuses it. The views `v_units` and `v_open_cos` are SELECT only for `ps_app`.
## Audit, settings, jobs (phases 1 and 3)
@@ -62,4 +136,8 @@ visual_batches, visual_rows, visual_materials, order links, labor, change rows,
## Versioning
packets.version goes up in every transaction that touches the packet or a child. It is the ETag and the NOTIFY payload.
- `packets.version` goes up by one in every write transaction that touches the packet or any of its children. The write pipeline does it explicitly (`UPDATE packets SET version = version + 1 ... RETURNING version`) after the caller's SQL. No trigger bumps it, so a read never does. It is the packet ETag and the NOTIFY payload.
- `projects.version`, `units.version` and `itps.version` play the same part for those rows.
- Child rows that are edited in place carry `row_version`. A PATCH sends `If-Match: <row_version>`. A mismatch is 409 `row_conflict` with the current row. The write bumps `row_version`.
- Every write takes `SELECT ... FOR UPDATE` on the packet row first. That lock replaces the old 8-second lease and serialises the counters: `next_weld_no`, `next_heat_no`, `next_page_no` and `next_co_no` hand out the next number with `UPDATE packets SET next_weld_no = next_weld_no + 1 RETURNING next_weld_no - 1`, so two concurrent adds never get the same number.
- Transitions do not check `version`. They compare and set on `expect: {status, dept}`, so a weld edit never blocks a send.
docs/SECURITY.md
+14 -8
View File
@@ -2,7 +2,7 @@
This document is rewritten for the server build. The old browser build's security document is in the old repo (`C:\Users\spencerm\Coding\PacketStation\docs\SECURITY.md`). The design comes from [BRIEF.md](BRIEF.md) and [PLAN.md](PLAN.md).
Status: phase 1. The identity controls (sessions, PIN, rate limits, audit, Origin check, derived group map) are built and tested against a Samba domain controller in CI. Every other control names the phase that builds it.
Status: phase 2. The identity controls (sessions, PIN, rate limits, audit, Origin check, derived group map) and the policy controls (server policy on every write, workflow steps as compare-and-set, server-side validation and limits, IT edit reasons, the filed lock, the upload pipeline for drawings and documents) are built and tested. Every other control names the phase that builds it.
Contents:
@@ -25,7 +25,7 @@ The M: drive build had 20 layers of protection, most of them enforced in the bro
| 7 | Record integrity check | A worker job (seals, asset hashes, audit hash chain) with an alert. | 9 |
| 8 | Self-hosted libraries | Kept: no CDN. The CSP allows nothing external. | 0 |
| 9 | Output escaping | React escapes output by default. The PDF export escapes on the server. | 1, 7 |
| 10 | Central permission policy | Server policy, checked on every write, including writes that do not come from the screens. | 2 |
| 10 | Central permission policy | Server policy, checked on every write, including writes that do not come from the screens. Built. | 2 |
| 11 | PIN sign-off (electronic signature) | Server-verified signature with the PIN or the domain password, stored with who, when, what, factor and a content hash. | 1 |
| 12 | IT edit reasons | Kept. Admin writes need a reason and are marked as IT edits in the packet history and the audit. | 2 |
| 13 | Screen lock | Server-enforced sessions. The browser reports idle and background state; the server records it and refuses an idle or locked session. Unlock is a step-up check on the server. | 1 |
@@ -71,9 +71,15 @@ The M: drive build had 20 layers of protection, most of them enforced in the bro
### Policy (phase 2)
- One policy in Python on the server. Every write goes: session, policy, transition, validation, version check, audit row, in one transaction.
- Every packet read returns `can: [...]` and `blocked: {...}` so the browser enables controls from the server's answer.
- A workflow step taken twice is refused with 409 and the current state.
- One policy in Python on the server (`server/packetstation/policy/`), pure functions with no I/O. Every write goes: session, policy, transition, validation, version check, packet log row, audit row, NOTIFY, in one transaction with the packet row locked. Nothing in the browser decides.
- Every packet read returns `can: [...]`, `blocked: {...}` and `holder`, so the browser enables controls from the server's answer. The server re-checks on the write.
- A workflow step taken twice is refused with 409 and the current state (compare-and-set on the expected status and department). Child rows carry a row version checked with If-Match. Weld, heat tag, page and change order numbers are assigned by the server under the lock.
- Preconditions (heat map answered, drawing attached, BOM pulled or short, checklist complete, welds recorded, Q.C. items complete, closed in VISUAL before filing) run on the server and are never bypassed, Admin included.
- Admin edits outside custody need an IT reason of at least 8 characters, recorded in the packet log and the audit row (`it_reason`). An Admin cannot write a filed packet except through the reopen action, which needs a reason and a signature and breaks the seal.
- Signatures (PIN or domain password, verified in the same transaction) on submit, send to Q.C., Q.C. accept and reject, route, file, reopen, change order issue, approve, reject and verify, the revision approval and hand drawing uploads.
- Input: every string is cleaned (control and invisible characters removed, NFC, trimmed) and cut to its field limit; heat numbers, stamps, project numbers and lots are checked against the old patterns; JSON bodies over 200 KB are refused.
- Uploads: typed by content (PDF, PNG, JPEG), 25 MB limit read in chunks, sha256, stored by id under the assets volume, served only with a session and with `Content-Disposition` and `X-Content-Type-Options: nosniff`.
- The filed lock: a filed packet accepts only the data book date and the Admin reopen. Filing writes an HMAC-SHA256 seal with the server seal key; a reopen keeps the old seal with who broke it and why.
- Details in [POLICY.md](POLICY.md).
### Audit (phase 1)
@@ -122,13 +128,13 @@ The IDs come from Part 3 of the old SECURITY.md. The status follows the approved
| I6 | Largely closed, phase 9 (groups plus reconcile); quarterly review IT's |
| I7 | Closed, phase 1 |
| I8 | App part closed, phase 1; kiosk lock policy IT's (Intune or GPO) |
| A1, A2, A3 | Closed, phase 2 |
| A1, A2, A3 | Closed, phase 2 (server policy on every call, workflow steps as compare-and-set, validation and limits on the server) |
| A4 | Closed, phase 0 |
| D1, D2, D4 | Deferred (envelope encryption, Key Vault) |
| D3 | IT's (host disk encryption on docker-1) |
| D5 | Closed, phase 4 (optional ClamAV) |
| D6, D7 | Deferred (retention period and customer copy need a decision from Seth and Q.C.) |
| M1 | Closed, phase 2 |
| M1 | Closed, phase 2 (append-only audit and packet log written only by the server; every packet write carries both) |
| M2 | Closed, phase 7 |
| M3 | Closed, phase 9 |
| M4, M5 | Deferred (logs to stdout now; SIEM shipping is IT's) |
@@ -136,7 +142,7 @@ The IDs come from Part 3 of the old SECURITY.md. The status follows the approved
| S2 | Closed, phase 1 |
| S3 | Closed, phase 9 (CI scanning) |
| S4 | Deferred to IT (pentest before go-live) |
| S5 | Closed, phase 2 |
| S5 | Closed, phase 2 (React escapes output; server messages are plain text; PDF export escaping comes with phase 7) |
| S6, S7, S8 | Closed, phase 7 |
| O1, O2 | Closed, phase 9 (pg_dump timer plus PBS; one rehearsed restore); quarterly restore IT's |
| O3, O4 | Closed, phase 10 |