Merge pull request 'Record Gitea answers: Seth's keep-as-built (6, 10, 14, 15, 16, 22, 26, 28, 29), archive (7), VMs (37)' (#51) from seth-keep-answers into main

Reviewed-on: #51

CHANGELOG.md

@@ -6,6 +6,10 @@ All notable changes to this project are recorded here. The format follows [Keep
 
 ### Decided
 
+- Gitea issues 10, 14, 15, 16, 22, 26, 28 and 29: SethD kept the built rules (cancelled work orders flagged only, data book date for Clerical and Q.C., signature undo rules, the server release gate, no "copy marks" button, block signature removal, the data book built at filing, every role may read the PDFs). Nothing changes in the build.
+- Gitea issue 6: every policy tightening stays, filing for packets routed to Shipping included.
+- Gitea issue 37: production and test each run natively on their own Linux VM, not in Docker. docker-1 keeps CI only. Built in phase 10; the docker-1 test stack runs until the test VM is ready.
+- Gitea issue 7: project files are also archived to an NFS share on the QNAP. Phase 10 adds an Admin setting for the archive folder that checks read and write. Nothing is deleted from Packet Station.
 - Gitea issue 2: a PIN is enough for every sign-off, Q.C. accept and filing included. Nothing changes in the build.
 - Gitea issue 3: shop devices keep the registered-device cookie. No device certificate is planned.
 - Gitea issue 19: the upload limits stay (MTR 25 MB, hand BOM 5 MB). ClamAV stays off in both stacks. The workstations run Microsoft Defender.

CLAUDE.md

@@ -1,6 +1,6 @@
 # Packet Station
 
-Digital job packets for the PESCO shop floor, rebuilt as a server application. One Docker Compose stack per environment on `docker-1`: a Python server (FastAPI) that enforces every rule, a worker for scheduled jobs, PostgreSQL 16 for data, and a React frontend as a pure client. VISUAL ERP is read through VisualKit and M-Files through MFilesKit, both PESCO's own read-only libraries. Sign-in is PESCO's Active Directory; every role is a `PS-*` AD group.
+Digital job packets for the PESCO shop floor, rebuilt as a server application. Production and test will each run natively on their own Linux VM, and `docker-1` keeps CI only (decided 2026-10-27, Gitea issue 37, built in phase 10). Until then the test stack is Docker Compose on `docker-1`. The parts: a Python server (FastAPI) that enforces every rule, a worker for scheduled jobs, PostgreSQL 16 for data, and a React frontend as a pure client. VISUAL ERP is read through VisualKit and M-Files through MFilesKit, both PESCO's own read-only libraries. Sign-in is PESCO's Active Directory; every role is a `PS-*` AD group.
 
 This is a rebuild of the browser-only install on the M: drive, not a port. Feature parity with that install is the acceptance bar. Read [docs/BRIEF.md](docs/BRIEF.md) for the brief, [docs/PLAN.md](docs/PLAN.md) for the approved phased plan, and [CHANGELOG.md](CHANGELOG.md) for what is done.
 

docs/DEPLOY.md

@@ -19,7 +19,7 @@ Both stacks run from the same image as separate compose projects.
 | Stack | Compose project | DNS name | Network alias on npm_default | VISUAL |
 |---|---|---|---|---|
 | Test | `packetstation-test` | `packetstation-test.pescoinc.biz` | `packetstation-test-app` | DEV: `vsql.pesco.com\DEV` / `PESCOINCDEV` |
-| Production | `packetstation` | `packetstation.pescoinc.biz` | `packetstation-app` | `vsql` / `PESCOINC`, set explicitly (planned, phase 10) |
+| Production | `packetstation` | `packetstation.pescoinc.biz` | `packetstation-app` | `vsql` / `PESCOINC`, set explicitly (planned, phase 10; moves to its own VM, see below) |
 
 - The app publishes no port. It joins `npm_default` as an external network with an explicit alias per stack (`APP_ALIAS`). Plain service names would collide, since both stacks share that network.
 - PostgreSQL stays on each stack's own internal `backend` network, never on `npm_default`.
@@ -59,7 +59,7 @@ The PESCO CA chain is mounted into the containers once and referenced by `LDAP_C
 
 ## Env files
 
-One folder per stack. The test stack: `/home/spencerm/packetstation/test/`, like the other compose stacks on docker-1 (decided 2026-10-18, Gitea issue 38). It holds `stack.env`, `secrets/`, `ca/pesco-ca.pem` and `backups/`, and its `stack.env` sets `ENV_FILE`, `SECRETS_DIR`, `CA_BUNDLE_FILE` and `BACKUP_DIR` to those paths. The compose files run from the checkout in `/home/spencerm/packetstation/repo/deploy`. The production stack: `/etc/packetstation/prod/` until issue 38 says otherwise. Each folder holds `stack.env` (settings only, no secrets) and `secrets/`. The env file sets `PROJECT_NAME`, `SECRETS_DIR` and `APP_ALIAS`, which the compose files require. `.env.example` in the repo lists every setting.
+One folder per stack. The test stack: `/home/spencerm/packetstation/test/`, like the other compose stacks on docker-1 (decided 2026-10-18, Gitea issue 38). It holds `stack.env`, `secrets/`, `ca/pesco-ca.pem` and `backups/`, and its `stack.env` sets `ENV_FILE`, `SECRETS_DIR`, `CA_BUNDLE_FILE` and `BACKUP_DIR` to those paths. The compose files run from the checkout in `/home/spencerm/packetstation/repo/deploy`. On 2026-10-27 Spencer decided that production and test each run natively on their own Linux VM, without Docker, and that docker-1 keeps CI only (Gitea issue 37). The test stack stays here until the test VM is ready. The production lines in this file describe the old compose plan and are rewritten for the VMs in phase 10. Each folder holds `stack.env` (settings only, no secrets) and `secrets/`. The env file sets `PROJECT_NAME`, `SECRETS_DIR` and `APP_ALIAS`, which the compose files require. `.env.example` in the repo lists every setting.
 
 Key settings:
 
@@ -175,7 +175,7 @@ The test stack on docker-1 lives under /home/spencerm/packetstation/test (stack.
 
 ## Known issue: Docker subnet pool on docker-1 (found 2026-10-17)
 
-Docker's default address pool on docker-1 holds fifteen /16 bridge networks (172.17.0.0 to 172.31.0.0). Each compose stack takes one or two (Packet Station takes two: backend and egress), the Gitea runner takes one per job, and the CI e2e stack needs two more. With the test stack up the pool ran out and every e2e job failed within seconds with "all predefined address pools have been fully subnetted". Idle networks of stopped stacks were removed to make room. The lasting fix is a larger pool in /etc/docker/daemon.json, for example `{"default-address-pools":[{"base":"10.200.0.0/16","size":24}]}`, which needs sudo and a Docker restart (a short outage for every stack on the host), or trimming unused compose projects. When the production stack joins, count the networks first: `docker network ls --filter driver=bridge | wc -l`.
+Docker's default address pool on docker-1 holds fifteen /16 bridge networks (172.17.0.0 to 172.31.0.0). Each compose stack takes one or two (Packet Station takes two: backend and egress), the Gitea runner takes one per job, and the CI e2e stack needs two more. With the test stack up the pool ran out and every e2e job failed within seconds with "all predefined address pools have been fully subnetted". Idle networks of stopped stacks were removed to make room. The lasting fix is a larger pool in /etc/docker/daemon.json, for example `{"default-address-pools":[{"base":"10.200.0.0/16","size":24}]}`, which needs sudo and a Docker restart (a short outage for every stack on the host), or trimming unused compose projects. Production and test move to their own VMs (Gitea issue 37), so the pool will only have to fit the runner and the CI e2e stack. Count the networks before adding any other stack: `docker network ls --filter driver=bridge | wc -l`.
 
 ## Known issue: vsql serves a weak self-signed certificate (found 2026-10-18)
 

docs/PLAN.md

@@ -120,7 +120,8 @@ Each phase ends with passing tests, docs updated (README, CHANGELOG, SECURITY.md
 ### Phase 10: Regression bench, hardening, go-live (medium)
 - Rewrite the 38 scenarios against the new UI with synthetic fixtures; run in Gitea Actions against compose.ci.yml. Policy matrix, merge rules and packet-rule tests already run from phase 2 and 3.
 - CSP final pass, dependency scan, build-version banner and reload, DEPLOY.md with the NPM proxy host settings, SECURITY.md rewritten with the ID table, GO-LIVE.md (date, boundary, groups, tablets, URL, M: retirement as read-only archive), PAPER-FALLBACK.md, incident notes.
-- Production compose with explicit VISUAL_SERVER and VISUAL_DATABASE; first start is Spencer's approval.
+- Production and test each on their own Linux VM, natively, with no Docker (Spencer, 2026-10-27, Gitea issue 37; see "Host layout"). The test VM is built first from the same steps, so the production install, backups, restore and doctor are rehearsed before go-live. The app and the worker run as systemd services from a venv built from the same `server/requirements.txt` and `server/requirements-pesco.txt` pins as the image, with PostgreSQL 16, ODBC Driver 18 and the PESCO CA installed on the VM. Settings from an env file, secrets as root-owned files read through the same `*_PASSWORD_FILE` settings, VISUAL_SERVER and VISUAL_DATABASE set explicitly; first start is Spencer's approval. NPM on docker-1 proxies `packetstation.pescoinc.biz` and `packetstation-test.pescoinc.biz` to their VMs over the LAN. Backups, the doctor and the restore are rewritten for the VMs (no `docker exec`). DEPLOY.md gets a VM section; the compose files stay for CI. The docker-1 test stack keeps running until the test VM is ready, then it is stopped.
+- Archive location (Gitea issue 7, Spencer 2026-10-27). Spencer creates an NFS share on the QNAP and mounts it on docker-1 (root, steps in DEPLOY.md), with write access for uid 10001. Compose bind-mounts it into the app and the worker at a fixed root, `ARCHIVE_ROOT` (unset turns the feature off). The Admin page gets a File locations section with one setting for now, the archive folder, as a path under `ARCHIVE_ROOT`. The server refuses a path that leaves the root (`..`, absolute paths elsewhere, symlinks out). "Check" writes a probe file, reads it back, removes it and shows the result, and saving runs the same check first. The setting change and every check are audited. `doctor` runs the check too. What is written to the archive and when (for example the sealed data book, its documents and a manifest at filing) is decided on issue 7 before it is built; until then the setting and the check exist and nothing is copied.
 - Closes: O3, O4; S4 is IT's pentest, deferred.
 
 ## SECURITY.md Part 3: which phase closes each ID
@@ -222,25 +223,25 @@ Need an answer before the named phase. Defaults apply if none arrives in time.
 4. **PIN defaults** (confirm): 6 to 12 digits, five attempts, 10-hour shop session, device cookie. Was open (Seth and Q.C., before phase 6): whether Q.C. accept and filing require the domain password even with a PIN set. Default: PIN is enough. Confirmed by SethD on 2026-10-26, Gitea issue 2: the PIN is enough for every sign-off, Q.C. accept and filing included. IT: device certificate later. The device cookie stays and no device certificate is planned: confirmed by Spencer (IT) on 2026-10-18, Gitea issue 3.
 5. **Several open orders on a project** (before phase 3): which order a packet shows. Proposed rule: the open order whose line's top job is the packet's work order base, else the newest open order, else the newest closed one; the packet shows "via base" or "via project" and Clerical can override. Needs Seth's view. Phase 3 (2026-10-16) built the default in one function (`feed/orders.py`): a Clerical pin, else the base's own linked order, else the project order whose top job is the unit, else the project's first open order, else its newest; the source (`via`) is stored on the project and unit and shown on the project page, which lists every open order. Seth may change the rule later in that one place.
 6. **VISUAL_ENCRYPT** (IT, before production): yes once vsql has a trusted certificate; `no` until then.
-7. **Policy tightenings** listed under "Challenges" 1 to 3 (Seth, before phase 2).
+7. **Policy tightenings** listed under "Challenges" 1 to 3 (Seth, before phase 2). Confirmed by SethD on 2026-10-26, Gitea issue 6: "good as shown". Every row is kept, filing for packets routed to Shipping included (Spencer read the answer that way on 2026-10-27).
 8. **Repo location**: PacketStation2 as proposed, or another name.
-9. **Retention period** (D6) and customer-facing data book copy (D7): deferred, but a decision from Seth and Q.C. lets a later phase close them.
+9. **Retention period** (D6) and customer-facing data book copy (D7): deferred, but a decision from Seth and Q.C. lets a later phase close them. Gitea issue 7: SethD expects all data to be kept (2026-10-26). Spencer decided on 2026-10-27 that project files are also archived to an NFS share he sets up on the QNAP, with an Admin setting for the folder path that checks read and write (phase 10, "Archive location"). Nothing is deleted from Packet Station; the archive is a copy. Still open on issue 7: a minimum retention period, if any, and what goes into the archive and when.
 
 Smaller questions from the design, with the default the build takes unless told otherwise:
 
 10. Same-rev drawing re-release (new file, same rev) on an in-shop packet: default a drawing-origin change order, not a silent swap.
-11. Cancelled (X) work orders: default flag only, like gone; drafts are not removed.
+11. Cancelled (X) work orders: default flag only, like gone; drafts are not removed. SethD on 2026-10-26, Gitea issue 10: the default is fine, left to Spencer. Kept as built.
 12. Partspecs, title or folded-operation target changes on an in-shop packet: default no change order, as the old build.
 13. Work-order-gone confirmation after 2 consecutive runs and the 10% suspect-batch guard: defaults as stated. SethD left both numbers to Spencer on 2026-10-26, Gitea issue 12. The defaults stay unless Spencer changes them.
 14. Pause scope: default pauses the feed, the drawing job and auto-release together.
 15. Gitea runner: whether the Samba DC container can run there (overlayfs xattrs, possibly cap SYS_ADMIN). Checked in phase 1; if not, CI uses a runner with a Docker socket on docker-1. Phase 1 finding (2026-10-16): the container needs `cap_add: [SYS_ADMIN]` (Samba 4.17 sets sysvol NT ACLs through `security.*` xattrs) and runs that way on docker-1; the Gitea runner itself is still to be tried. Phase 2 (2026-10-16): an act_runner with the host Docker socket is registered on docker-1 as `docker-1` (deploy/runner/); resolved: the runner passes `cap_add` through, and the e2e job (stack with the DC plus the Playwright scenarios) passed on it on 2026-10-16.
 16. NPM: confirm `Block common exploits` does not interfere with uploads; checked when the test proxy host is made in phase 0.
-17. SET_DATABOOK_AT is writable by anyone in the old build, even when filed: default restrict to Clerical and Q.C.
+17. SET_DATABOOK_AT is writable by anyone in the old build, even when filed: default restrict to Clerical and Q.C. Confirmed by SethD on 2026-10-26, Gitea issue 14.
-18. CO verifications cannot be withdrawn; ITP line signatures can be undone by the same inspector or an Admin before the block is signed.
+18. CO verifications cannot be withdrawn; ITP line signatures can be undone by the same inspector or an Admin before the block is signed. Confirmed by SethD on 2026-10-26, Gitea issue 15.
 
 From phase 4 (2026-10-16), for Seth to veto:
 
-19. The release gate has two server rules: every line not N/A is pulled or on an open short, and every pulled line that issued pieces has a heat unless it is N/A or a sub-assembly. The old build enforced both on the screen only. Default: both on the server.
+19. The release gate has two server rules: every line not N/A is pulled or on an open short, and every pulled line that issued pieces has a heat unless it is N/A or a sub-assembly. The old build enforced both on the screen only. Default: both on the server. Confirmed by SethD on 2026-10-26, Gitea issue 16: both stay, so the warehouse fills the heat map for the parts it sends.
 20. Hold and short reasons: the old build accepted any text. Default: closed sets on the server; "Other" on a hold needs a note.
 21. Receiving a short on a packet the warehouse no longer holds: default it stays a warehouse right (WH_RECEIVE_SHORT, any status but filed), and the holder of the packet (department lead or supervisor, Q.C., Clerical) can receive too, through edit_bom.
 22. Upload kinds and limits: default an MTR is PDF, PNG or JPG up to 25 MB, a hand BOM is XLSX or CSV up to 5 MB. ClamAV is optional and off unless CLAMAV_HOST is set; when set, an unreachable scanner refuses uploads (fail closed). Confirmed by Spencer (IT) on 2026-10-18, Gitea issue 19: the limits stay, and ClamAV stays off in both stacks because every workstation that uses Packet Station runs Microsoft Defender. The `clamav` profile stays in the build, unused.
@@ -249,7 +250,7 @@ From phase 4 (2026-10-16), for Seth to veto:
 From phase 5 (2026-10-16), for Seth to veto:
 
 24. Carried marks: when a packet is routed to another department, the welds, heat tags and heat table stay on the packet. The read model computes a `carried` flag. Nothing is copied. Default: computed, not copied.
-25. The old build's manual "copy marks to the new sheet" button on a change order is not ported. On a page replacement only the heat table placement is copied to the new page. Welds and heat tags stay on the VOID page. Default: not ported in phase 5; a follow-up.
+25. The old build's manual "copy marks to the new sheet" button on a change order is not ported. On a page replacement only the heat table placement is copied to the new page. Welds and heat tags stay on the VOID page. Default: not ported in phase 5; a follow-up. Confirmed by SethD on 2026-10-26, Gitea issue 22: not ported. After a change order the department re-verifies the work, since a change can alter what was marked.
 26. Checklist templates live in the `checklist_templates` settings row, with the old lists as defaults. Default: no screen edits them in phase 5; Admin editing comes in phase 9.
 
 From phase 6 (2026-10-17), for Seth and Q.C. to veto:
@@ -257,14 +258,14 @@ From phase 6 (2026-10-17), for Seth and Q.C. to veto:
 27. Q.C. reject: the old build took any non-empty reason. Default: a reason of at least 8 characters, or a well-formed NCR number (`NCR-YYYY-NNNN`) in its place, or both. The NCR is stored on the inspection and linked to pesco-ncr. Applied in phase 6.
 28. A filed packet in a unit and the unit's ITP: the ITP belongs to the unit, not to a packet, so ITP writes never look at a packet's status. Default: a filed packet does not lock the ITP. This resolves the phase 5 follow-up. Applied in phase 6.
 29. ITP status: the 0003 enum had `open`, `signed` and `reopened`. Default: `itp_status` gains `complete` for a signed-off ITP; a reopened ITP is `open` again with the reopen recorded. `signed` and `reopened` stay in the enum unused, since PostgreSQL cannot drop an enum value. Applied in phase 6.
-30. Removing an ITP block signature: the old build let any Q.C. editor undo it. Default: the signer or an admin, while the ITP is open. The block's QN, remarks and customer fields stay. Applied in phase 6.
+30. Removing an ITP block signature: the old build let any Q.C. editor undo it. Default: the signer or an admin, while the ITP is open. The block's QN, remarks and customer fields stay. Applied in phase 6. Confirmed by SethD on 2026-10-26, Gitea issue 26: a fix to customer fields or an earlier step's remarks goes back to that step or to Clerical.
 31. The signed ITP PDF on sign-off: default a hook (`pdf.itp.file_signed_itp_pdf`) that phase 7 builds. Today it files nothing, the sign-off log says `PDF filing reserved for phase 7`, and the automatic `qc_docs` ITP row is written only when the hook returns a file. Applied in phase 6. Phase 7 (2026-10-17) built the hook: the sign-off files the signed ITP PDF and the automatic ITP document in the same transaction, and a PDF that cannot be built fails the sign-off.
 
 From phase 7 (2026-10-17), for Seth and Clerical to veto:
 
 32. Prints: the old build pushed page images into the browser's print dialog and offered the same PDF when the dialog was blocked. Default: a print opens as a stamped PDF in a new browser tab, and the person prints it from the PDF viewer (Ctrl+P). One path is simpler and prints the same stamped pages. Phase 7 (2026-10-17) applied the default.
-33. When the data book is built: default on the server when the packet is filed, inside the filing transaction, stored as an asset on the seal. A data book that cannot be built fails the filing, so nothing is filed half way. Before filing, the data book can be viewed or downloaded at any time from status complete on; that copy is stamped "not yet filed". Phase 7 (2026-10-17) applied the default.
+33. When the data book is built: default on the server when the packet is filed, inside the filing transaction, stored as an asset on the seal. A data book that cannot be built fails the filing, so nothing is filed half way. Before filing, the data book can be viewed or downloaded at any time from status complete on; that copy is stamped "not yet filed". Phase 7 (2026-10-17) applied the default. Confirmed by SethD on 2026-10-26, Gitea issue 28.
-34. Who may read the PDFs and the CSV: default every signed-in role that can open a packet, PM included. Every print and every export is logged with who, when, what and how many pages, in the append-only `print_log` and in the audit. Phase 7 (2026-10-17) applied the default.
+34. Who may read the PDFs and the CSV: default every signed-in role that can open a packet, PM included. Every print and every export is logged with who, when, what and how many pages, in the append-only `print_log` and in the audit. Phase 7 (2026-10-17) applied the default. Confirmed by SethD on 2026-10-26, Gitea issue 29: everyone may view the whole packet; only the holder edits, and missing work goes back to the department that owes it.
 35. The old "Original packet PDF" print kind: the rebuild has no VISUAL packet PDF. The drawing pages come from M-Files or a hand upload, and "Drawing (no markup)" prints them as attached. Default: the kind is gone. Phase 7 (2026-10-17) applied the default.
 36. Filing while the feed still shows a work order open in VISUAL: default a second confirmation (`confirm_open_wos`), a warning and not a block, on both routes. The work orders are kept on the seal and in the log note. A hand packet has no feed state and no warning. Phase 7 (2026-10-17) applied the default.
 37. The controlled-copy stamp carries a short signature made with the seal key, so a printed or exported copy can be checked against the server later. Default: phase 7 ships the signature, the check function and the print log; phase 9 adds the Admin screen that checks a copy. Phase 7 (2026-10-17) applied the default.
@@ -302,3 +303,5 @@ From phase 9 (2026-10-21), for Seth or Spencer to veto:
 ## Host layout on docker-1
 
 OPS2 (Gitea issue 38): the test stack's files stay under `/home/spencerm/packetstation/test`, like the other compose stacks on docker-1 (gitea, npm, pesco-ncr and the rest live under `/home/spencerm`). Decided by Spencer (IT) on 2026-10-18. The move to `/etc/packetstation/test` is dropped. Whether the production stack follows the same layout is asked on issue 38. Until it is answered, DEPLOY.md keeps `/etc/packetstation/prod`.
+
+Production host (Gitea issue 37): decided by Spencer on 2026-10-27. Production runs on its own Linux VM as native services, not in Docker and not on docker-1. The test stack moves to its own VM too, built the same way, so test rehearses production; docker-1 keeps CI only (the Gitea runner and the CI e2e stack). The docker-1 test stack runs until the test VM is ready. This replaces OPS2 for both stacks once the VMs exist: the production folder question on issue 38 no longer applies, and the VM file layout is set in phase 10. The docker-1 address pool (OPS1) then only has to fit the runner and the CI e2e stack. Still open on issue 37: each VM's name and address, the OS (Debian 12 matches the image), who builds them, their size, and how NPM reaches them (HTTP on the LAN or TLS to the VM).

docs/POLICY.md

@@ -995,6 +995,8 @@ On a draft made by the feed that is flagged (or already has a hand upload), Cler
 
 The old policy was looser than the old screens in places. The rebuild follows the screens where they were stricter, and the README where it states the intent. Each row is a change Seth can veto.
 
+SethD answered on 2026-10-26: every row is kept as shown (Gitea issue 6), and he confirmed the data book date (issue 14), the withdrawn verification and ITP initials rows (issue 15), both release rows (issue 16) and the block signature row (issue 26).
+
 | What the old build allowed | What the rebuild does | Why | Can be vetoed |
 |---|---|---|---|
 | A department lead could write welds, BOM, heat tags, the checklist, stage fields and the clock through the policy whenever the packet was in a shop status. | These edits need the holder of the packet (the holder rule, applied as the old screens did). A lead of another department cannot. | The screens only offered these to the holder. The policy now matches. | Yes |