PacketStation/tests/e2e/BENCH.md

Regression bench: the old 38 scenarios on the new UI

The old build's bench is tests/ps/*.js in the old repo (read-only, fixtures local only). Its scripts logged values for a person to read; a script "passed" when it printed no page error. Here every scenario is a Playwright test with real assertions, on the synthetic fixtures (tests/fixtures/generator.py) and the throwaway Samba DC, run by the e2e job in .gitea/workflows/ci.yml against deploy/compose.ci.yaml.

Every spec imports test from tests/helpers.ts. Its CSP guard fails a test on any securitypolicyviolation event in any frame, or a Content Security Policy error on the console, in every browser context the test opens.

Spec and test names below are file › test. "Server" points at the pytest file that holds a rule the old script checked in the browser. "Dropped" gives the reason: the BRIEF.md "Drop" list, or a mechanism the server replaced.

#	Old script	What it checked	New coverage
1	t8s	Data book PDF of a filed, sealed packet with a Q.C. reject in its history	close › "R1: the data book and the traceability CSV answer as files"; close › "R1: filed in the data book with the PIN; the seal holds and the filed data book downloads". Server: test_pdf_databook.py
2	t9s	Departments end to end: assign a stall and welder, checklist, weld, submit; the lead cannot review; route to 09 Hydro; Q.C. accepts; Clerical closes and exports	shop › "tsec2: assign, work at the stall, submit, send to Q.C."; shop › "tcust: ... the lead's read-only review ..."; shop › "skip Q.C.: sup.wes routes a review packet to 09 Hydro"; qc › "accept: ..."; close › R1 tests. First-run admin setup and the Users sheet: dropped (roles are AD groups, Drop list)
3	tauto	Clean VISUAL drafts go to the warehouse on their own; flagged ones stay; sub-assembly borrows the heat table; warehouse grouping by sent date	Server: test_feed_apply.py test_auto_release, test_paused_stops_auto_release; test_shop_api.py test_heat_src_borrowing. visual-feed › "Run now ..." accepts an auto-released packet. Warehouse grouping: warehouse › "warehouse board"
4	tcl	Only supervisors of the departments a change order names get the banner and Verify	shop › "tcosup: a change order requiring 03 on the Supervisor screen, verified with a signature" (sup.wes gets it; multi.mia, 06, does not)
5	tclose	Close in VISUAL, file; filing blocked before the close; move back undoes it	close › "R1: confirmed closed in VISUAL ..."; close › "R2: filing is blocked until the job is closed in VISUAL; move back undoes the close"
6	tclsync4	The feed makes packets, projects and units; a second pull updates drafts; the vault attaches the drawing; Sync now	visual-feed › "Run now makes project L900, its drafts and attaches the WELD-PART drawing". Server: test_feed_apply.py, test_feed_api.py. The browser-side apply, rows.json, found.json, want.json and the helper check-in: dropped (PowerShell sync in the Drop list; the worker runs the feed)
7	tcoappr	VISUAL changes on in-shop packets open change orders for Clerical; staged drawing; approve, hold, reject; old sheets VOID; supervisors verify	visual-feed › "VISUAL moves the drawing to rev 2: a change order, staged, approved"; shop › "tcosup ..."; shop › "tsync: two sessions at once" (two verifies). Server: test_co_apply.py (hold, reject, no second change order for the same rows), test_fingerprint.py
8	tcosup	Change-order banner, stall tag and rows on the Supervisor screen	shop › "tcosup: a change order requiring 03 on the Supervisor screen, verified with a signature"
9	tcust	Custody: view only, no writes; send back with a reason; short-line exception; change-order verify exception	shop › "tcust: custody, view only, the lead's read-only review, multi.mia cannot assign 03"; clerical-draft › "wh.wade sees a Clerical draft as view only, and no Send button"; warehouse › "pull flow" (shorts). Server: test_policy_matrix.py, test_policy_shop.py, test_policy_warehouse.py
10	tdemo	Demo data walkthrough: banner text, thumbnails, welder notice	Dropped: demo mode and the claude.ai build target (Drop list). The banner and the review flow are in shop › "tcosup ..."
11	tflag	Flagged column before Ready to release, flag reasons on cards; one sync progress bar	visual-feed › "tflag and tflagsearch: the Flagged column, the flag texts and the Flag filter". The helper's progress bar: dropped (PowerShell sync and its heartbeat, Drop list; the From VISUAL panel shows the worker's runs)
12	tflagsearch	Search and filter flagged drafts by reason; list rows show the flag	visual-feed › "tflag and tflagsearch: ..."; findbar › "tflt and tlist: ..."
13	tflt	Find bar: search keeps focus, Esc, status and station, shorts chip, list view, sort, saved filters	findbar › "tflt: search, Esc, status and station, clear, the shorts chip"; findbar › "tflt and tlist: the list view, sort, Enter opens, saved filters, only the screen's packets"
14	tfs	Full-screen drawing: grows, survives a re-render, Esc and Exit leave it, leaving the tab ends it; MTR viewer	viewer › "tfs: full screen from the viewer; Exit, Esc and leaving the tab end it"; viewer › "tmtr: ...". Marking in full screen: shop › "tsec2 ..." marks a weld on the Weld map
15	thand	Clerical uploads the drawing or the BOM by hand on a flagged draft only, with a reason; not the warehouse; the feed keeps a hand BOM	visual-feed › "thand: Clerical uploads the drawing and the BOM by hand on a flagged draft; the warehouse cannot". Server: test_feed_api.py test_hand_drawing, test_hand_bom; test_handbom.py
16	tht	Heat table on the drawing: place, drag, size, live heat, VOID page, Q.C. read only	shop › "tht: the heat table on the drawing"
17	titp	Digital ITP: template, initials, N/A, customer witness, block signatures, hold callout, sign-off PDF, reopen	qc › "titp: the digital ITP, the hold callout, every block signed, sign-off, reopen"; close › "ITP: signed off, its PDF is filed in the unit folder ..."; close › "C3: an open ITP on the unit blocks filing ..."
18	tlay2	KPI tap filters and KPIs stay; list view keeps the header; Filters opens and closes; search keeps focus	findbar › "tlay2: a KPI tap filters the board and the KPIs stay; Filters opens and closes"
19	tlist	List view shows the same packets as that role's board	findbar › "tflt and tlist: ..."
20	tmat	Job pull materials land on VISUAL packets: other drawings' lines left off, sub-assemblies kept, no change on a second pull, not overwritten in the shop	visual-feed › "tmat: the work order's materials from VISUAL are the WELD-PART packet's BOM". Server: test_feed_apply.py, test_candidates.py. materials.json and the browser apply: dropped (Drop list)
21	tmotion	Motion switch, reduced motion, a card moved by someone else glides	findbar › "tmotion: the switch, reduced motion, and a card moved by someone else glides"
22	tmtr	MTR upload (PDF and PNG) per line, view in the viewer, open in a new tab	viewer › "tmtr: a PDF and a PNG MTR, each shown in the MTR viewer and served for a new tab"; warehouse › "pull flow"
23	tnav	Back goes one level back to the same scroll spot	findbar › "tnav: Back from a packet lands on the home at the same scroll spot"
24	tpan2	Touch drags pan a zoomed drawing; after re-renders a drag still pans once	viewer › "tpan2: a mouse drag pans by exactly the drag after re-renders"; viewer › "tpan2: on a touch device each finger drag moves the view"
25	tpdocs	Packet documents for the data book: Q.C. adds, a non-PDF is refused, read-only for others, taken out with a reason	qc › "tpdocs: a packet document added, refused, seen read-only, taken out"
26	tpm	PM screen: overview, drill down, tabs, scope, exports, view only	pm › "tpm: overview, projects, drill down, tabs, scope, exports, view only"; pm › "tpm: the period changes the history figures"; pm › "C10: ..."
27	tpop	Sheets keep their margins on desktop and phone widths	findbar › "tpop: the New packet sheet keeps its margins on desktop and phone widths"
28	tpr	Print menu kinds, traveler, data book, print logged to the audit	close › "R1: the print menu lists the kinds; a traveler print is logged and shows on the History tab"; close › "ITP: ..." (ITP print); close › "api: print guards and a signed-out read"; admin-stampverify (the printed stamp). Server: test_prints.py, test_pdf_traveler.py
29	tqcf	Q.C. project and unit folders, a new revision supersedes, non-PDF refused, no Add for the warehouse	qc › "tqcf: project and unit folders, a new revision, a refused file, no Q.C. screen for wh.wade"
30	trel	Clerical's Released section: tiles per place with overdue and 7+ day counts, list, open, Back	Not covered: the new Clerical home has no Released section (old released.js). See the note below
31	trevchk2	M-Files rev vs VISUAL rev: flags with drafting status, mismatch callout, a Clerical lead approves with a reason, the flag returns on a new VISUAL rev	visual-feed › "tflag and tflagsearch: ..."; visual-feed › "trevchk2: a Clerical supervisor approves the M-Files rev with a reason and a signature". Server: test_drawing_decide.py (test_approved_other_rev_still_mismatches and the rest), test_feed_api.py test_rev_approve
32	tsec2	Signatures end to end, wrong PIN, cancel, seal, admin reason, idle lock, formula guard, integrity, access log	shop › "tsec2 ..."; signin › "8. three idle minutes lock the screen and the PIN unlocks it"; close › "R1: filed ... with the PIN ..." and "R1: admin.ann reopens with a reason and her password ..."; admin-security; admin-accesslog › "CSV export: a cell that starts with = is guarded". Second-admin approval: dropped (Drop list)
33	tsend	Heat map answer first; send straight to a department or to Q.C.; send back to Clerical with a reason	send › "tsend: the heat map answer and the drawing come first; straight to a department, straight to Q.C."; send › "tsend: the department's supervisor sees it and sends it back to Clerical with a reason"; clerical-draft › "clerk.clara makes a hand draft and sends it to the warehouse"
34	tsync	Two screens at once: merges, renumbered weld ids, two verifies kept, a stale step refused, filed packet blocked, long history	shop › "tsync: two sessions at once". Server: test_pipeline_concurrency.py, test_shop_concurrency.py
35	tsyncsafe	The feed never overwrites shop packets or records and never makes duplicates	Server: test_feed_apply.py (test_removed_packet_key_is_never_recreated, test_hand_customer_is_never_overwritten, change orders for in-shop packets), test_candidates.py. The browser-side apply it raced: dropped (Drop list)
36	tunits	Projects and units: many units at once, duplicate refused, move a packet, renumber with an alias, closed unit	projects › "tunits: a new project with three units at once; a duplicate unit is refused"; projects › "tunits: a packet moves to another unit through the two-step picker; the warehouse cannot move one"; projects › "tunits: a Clerical supervisor renumbers a unit; its packet follows and the old number is an alias"; projects › "tunits: a closed unit takes no new packet". Server: test_projects_api.py
37	tvis	Excel Job Release import in the browser; op-to-department map	Dropped: Excel upload of Job Release Information (Drop list); the feed replaces it (row 6). The resource map: server test_feed_api.py
38	twh	Warehouse board: stages, hold, drag, drop on Released refused, auto-start on a tick, stage shown to others	warehouse › "warehouse board"; warehouse › "sup.wes sees the warehouse pill and line but no warehouse buttons"; warehouse › "pull flow"

Phase 10 additions beyond the old bench

• archive › "Admin saves the archive folder under File locations; the check passes" and "a filed packet is copied to the archive folder" (Gitea issue 7).
• version › "the app page carries the final Content Security Policy and the other headers" and "a new build on the server shows the banner; it reloads only on Reload" (the update lock in the Drop list is replaced by the banner).
• The CSP guard in helpers.ts on every test, and version › "the CSP guard sees a refused inline style and a refused inline script" to prove the guard catches a violation.

Note on trel

The old Clerical home had a "Released" section (src/js/released.js in the old repo): where every released packet is, by place, with overdue and 7+ day counts. It is not in the Drop list and the new UI does not have it. The find bar (status and department filters), the Supervisor and Warehouse boards and the PM screen's Aging tab show the same packets another way. Whether that is enough is a question for Seth; until then this row stays uncovered.

Running it

python -m tests.fixtures.generator --out tests/fixtures/generated
docker compose -p packetstation-ci --env-file deploy/ci/ci.env -f deploy/compose.yaml -f deploy/compose.ci.yaml up -d --build --wait
docker compose -p packetstation-ci --env-file deploy/ci/ci.env -f deploy/compose.yaml -f deploy/compose.ci.yaml --profile e2e run --rm playwright

The specs share one database and run in file order on one worker. visual-feed writes the fixture state and edits drawings.1.json before its first run, so the stack must start from freshly generated fixtures.