Phase 9: Admin completion, reconcile, notifications, integrity, backups #49

Merged

spencerm merged 3 commits from phase-9 into main 2026-10-27 20:54:47 +00:00

Conversation 0 Commits 3 Files Changed 116

+15125 -297

spencerm commented 2026-10-22 00:27:42 +00:00

Owner

Copy Link

Copy Source

Phase 9 as docs/PLAN.md describes it, plus the items earlier phases left for it (checklist editing, PLAN question 26; the stamp check, question 37; the quota alert to PS-Admins, Gitea issue 20).

What is built

Admin. All of these tabs are for Admins in a password session, and every change is audited:

• Departments: edit the display name and stations, never the code or slug. Adding a department shows the three PS-<code>-<Slug>-Worker/Lead/Supervisor groups IT must create.
• Profiles: welder stamp, station limits per department, active. A CSV import keyed by employee ID shows a preview with every error and applies only when the preview is clean.
• Checklists: the weld and plain lists, saved with compare and set.
• Security:
  • session, lockout and rate-limit settings, within bounds;
  • alerts from the last 24 hours: lockouts, repeated wrong PINs, denied actions, wrong PINs at sign-off or unlock, admin changes, IT edits, integrity failures and sign-in bursts;
  • the latest integrity result.
• Access log: filters and a CSV export with the formula guard.
• Who is in: who is signed in now.
• Stamp check: paste a stamp line or upload a PDF.
• Notifications.
• Flags for AD accounts with a missing or shared employee ID.

"Refresh from AD now" now asks the worker for an ad-reconcile run (202) and shows its report.

Worker jobs.

• ad-reconcile, nightly at 01:30:
  • finds groups missing from or extra in the OU;
  • pages the members of every mapped group with IN_CHAIN and upserts them on objectGUID;
  • marks users disabled or gone and ends their sessions, and ends the sessions of users left with no role;
  • flags missing or duplicate employeeIDs;
  • refuses to change anything when the OU looks wrong.
• session-refresh, every 15 minutes.
• integrity, at 04:00: seals, asset sha256 (7 days nightly, all on Sundays), missing files, orphan files (reported, never deleted) and the audit chain.
• backup-check, at 07:15.
• notify-send and notify-digest.

Notifications.

• 13 event types, each with enabled, immediate or daily digest, and recipients: an AD group (searched under LDAP_BASE_DN and resolved at send time), a user, a mailbox, or the department's Supervisor group.
• Per-event exclusions, a test send, and a sent log.
• Seeded on first start: every event enabled, immediate, to PS-Admins.
• Mail goes over STARTTLS verified against SMTP_CA_BUNDLE, with no plain-text fallback, under the [Packet Station TEST] or [Packet Station] prefix.
• NOTIFY_DRY_RUN records without sending, except a test send.
• Account settings shows what reaches me and why. People can opt out of anything but the three security events.
• The quota check, feed and drawing failures, Q.C. and ITP rejects and change orders waiting all raise their events through this pipeline.

Backups.

• scripts/backup/ps-backup.sh: pg_dump custom format plus globals (no role passwords), checked with pg_restore --list, 7 kept, last-success and last-failure markers.
• ps-restore.sh.
• packetstation-backup.service and .timer (OnCalendar=*-*-* 02:00:00 America/Denver, Persistent=true).

CI (S3). pip-audit on server/requirements.txt and npm audit --audit-level=high. pip-audit found two known vulnerabilities, fixed by patch bumps: pypdf 6.21.1 and certifi 2026.10.19.

Migration 0010. Docs updated: README, CHANGELOG, SCHEMA, POLICY, SECURITY (I6 app part, M3, O1, O2, S3 closed), DEPLOY, FEED and PLAN (new open questions 43 to 52 with their defaults, for Seth and Spencer to veto).

How it was verified

• Locally:
  • ruff, ruff format and mypy are clean;
  • 5492 server tests pass without a database, and the agents ran their new database tests through the throwaway servers;
  • eslint, tsc, 337 vitest tests and the build pass.
• On docker-1, in the CI compose stack with the Samba DC:
  • 6065 server tests pass in the container, including the reconcile against the DC;
  • 337 frontend tests pass;
  • doctor passes at 0010;
  • 83 of 83 Playwright scenarios pass, including the new Departments, Notifications, Security, access log, stamp check and Who is in specs.
• Restore rehearsal on the test stack on 2026-10-21, as spencerm without sudo:
  • the backup took 0.76 s and wrote a 2.2 MB dump;
  • the restore into ps_restore_rehearsal took 2 s, and row counts matched for all 64 tables;
  • all 189 asset files (125 MB) matched assets.sha256;
  • the throwaway database and copies were removed, and the test stack was not touched.
    DEPLOY.md has the write-up.

Root-only steps for Spencer (docker-1, after merge)

cd ~/packetstation/repo   # synced to the merged main
sudo install -o root -g root -m 0755 scripts/backup/ps-backup.sh  /usr/local/sbin/ps-backup.sh
sudo install -o root -g root -m 0755 scripts/backup/ps-restore.sh /usr/local/sbin/ps-restore.sh
sudo install -d -o root -g root -m 0755 /etc/packetstation
sudo install -o root -g root -m 0644 scripts/backup/backup.conf.example /etc/packetstation/backup.conf
sudo install -o root -g root -m 0644 scripts/backup/packetstation-backup.service /etc/systemd/system/
sudo install -o root -g root -m 0644 scripts/backup/packetstation-backup.timer   /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemd-analyze verify /etc/systemd/system/packetstation-backup.service
sudo systemctl enable --now packetstation-backup.timer
sudo systemctl start packetstation-backup.service   # one run now; check ~/packetstation/test/backups

The production folder (/var/backups/packetstation/prod) and the prod line in backup.conf wait for the production stack in phase 10 (DEPLOY.md "Backup (phase 9)").

After merge

Roll the test stack to the main image CI publishes. That image carries the pypdf and certifi bumps. The test stack keeps NOTIFY_DRY_RUN=true.

🤖 Generated with Claude Code

Phase 9 as docs/PLAN.md describes it, plus the items earlier phases left for it (checklist editing, PLAN question 26; the stamp check, question 37; the quota alert to PS-Admins, Gitea issue 20). ## What is built **Admin.** All of these tabs are for Admins in a password session, and every change is audited: - Departments: edit the display name and stations, never the code or slug. Adding a department shows the three `PS-<code>-<Slug>-Worker/Lead/Supervisor` groups IT must create. - Profiles: welder stamp, station limits per department, active. A CSV import keyed by employee ID shows a preview with every error and applies only when the preview is clean. - Checklists: the weld and plain lists, saved with compare and set. - Security: - session, lockout and rate-limit settings, within bounds; - alerts from the last 24 hours: lockouts, repeated wrong PINs, denied actions, wrong PINs at sign-off or unlock, admin changes, IT edits, integrity failures and sign-in bursts; - the latest integrity result. - Access log: filters and a CSV export with the formula guard. - Who is in: who is signed in now. - Stamp check: paste a stamp line or upload a PDF. - Notifications. - Flags for AD accounts with a missing or shared employee ID. "Refresh from AD now" now asks the worker for an ad-reconcile run (202) and shows its report. **Worker jobs.** - ad-reconcile, nightly at 01:30: - finds groups missing from or extra in the OU; - pages the members of every mapped group with IN_CHAIN and upserts them on objectGUID; - marks users disabled or gone and ends their sessions, and ends the sessions of users left with no role; - flags missing or duplicate employeeIDs; - refuses to change anything when the OU looks wrong. - session-refresh, every 15 minutes. - integrity, at 04:00: seals, asset sha256 (7 days nightly, all on Sundays), missing files, orphan files (reported, never deleted) and the audit chain. - backup-check, at 07:15. - notify-send and notify-digest. **Notifications.** - 13 event types, each with enabled, immediate or daily digest, and recipients: an AD group (searched under LDAP_BASE_DN and resolved at send time), a user, a mailbox, or the department's Supervisor group. - Per-event exclusions, a test send, and a sent log. - Seeded on first start: every event enabled, immediate, to PS-Admins. - Mail goes over STARTTLS verified against SMTP_CA_BUNDLE, with no plain-text fallback, under the `[Packet Station TEST]` or `[Packet Station]` prefix. - NOTIFY_DRY_RUN records without sending, except a test send. - Account settings shows what reaches me and why. People can opt out of anything but the three security events. - The quota check, feed and drawing failures, Q.C. and ITP rejects and change orders waiting all raise their events through this pipeline. **Backups.** - `scripts/backup/ps-backup.sh`: pg_dump custom format plus globals (no role passwords), checked with `pg_restore --list`, 7 kept, last-success and last-failure markers. - `ps-restore.sh`. - `packetstation-backup.service` and `.timer` (`OnCalendar=*-*-* 02:00:00 America/Denver`, `Persistent=true`). **CI (S3).** pip-audit on server/requirements.txt and `npm audit --audit-level=high`. pip-audit found two known vulnerabilities, fixed by patch bumps: pypdf 6.21.1 and certifi 2026.10.19. Migration 0010. Docs updated: README, CHANGELOG, SCHEMA, POLICY, SECURITY (I6 app part, M3, O1, O2, S3 closed), DEPLOY, FEED and PLAN (new open questions 43 to 52 with their defaults, for Seth and Spencer to veto). ## How it was verified - Locally: - ruff, ruff format and mypy are clean; - 5492 server tests pass without a database, and the agents ran their new database tests through the throwaway servers; - eslint, tsc, 337 vitest tests and the build pass. - On docker-1, in the CI compose stack with the Samba DC: - 6065 server tests pass in the container, including the reconcile against the DC; - 337 frontend tests pass; - doctor passes at 0010; - 83 of 83 Playwright scenarios pass, including the new Departments, Notifications, Security, access log, stamp check and Who is in specs. - Restore rehearsal on the test stack on 2026-10-21, as spencerm without sudo: - the backup took 0.76 s and wrote a 2.2 MB dump; - the restore into `ps_restore_rehearsal` took 2 s, and row counts matched for all 64 tables; - all 189 asset files (125 MB) matched `assets.sha256`; - the throwaway database and copies were removed, and the test stack was not touched. DEPLOY.md has the write-up. ## Root-only steps for Spencer (docker-1, after merge) ```bash cd ~/packetstation/repo # synced to the merged main sudo install -o root -g root -m 0755 scripts/backup/ps-backup.sh /usr/local/sbin/ps-backup.sh sudo install -o root -g root -m 0755 scripts/backup/ps-restore.sh /usr/local/sbin/ps-restore.sh sudo install -d -o root -g root -m 0755 /etc/packetstation sudo install -o root -g root -m 0644 scripts/backup/backup.conf.example /etc/packetstation/backup.conf sudo install -o root -g root -m 0644 scripts/backup/packetstation-backup.service /etc/systemd/system/ sudo install -o root -g root -m 0644 scripts/backup/packetstation-backup.timer /etc/systemd/system/ sudo systemctl daemon-reload sudo systemd-analyze verify /etc/systemd/system/packetstation-backup.service sudo systemctl enable --now packetstation-backup.timer sudo systemctl start packetstation-backup.service # one run now; check ~/packetstation/test/backups ``` The production folder (`/var/backups/packetstation/prod`) and the prod line in backup.conf wait for the production stack in phase 10 (DEPLOY.md "Backup (phase 9)"). ## After merge Roll the test stack to the main image CI publishes. That image carries the pypdf and certifi bumps. The test stack keeps `NOTIFY_DRY_RUN=true`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

spencerm added 1 commit 2026-10-22 00:27:42 +00:00

Phase 9: Admin completion, reconcile, notifications, integrity, backups ... a82e056b6a

Admin: Departments (name and stations; add shows the three groups IT must
create), Profiles (welder stamp, station limits, active) with a CSV import
that previews and applies only when clean, Checklists (weld and plain),
Security (session, lockout and rate-limit settings with bounds, alerts for
the last 24 hours, the integrity result), Access log with filters and a
guarded CSV, Who is in, Stamp check, Notifications, and the employeeID
flags. Every Admin change is audited.

Worker: ad-reconcile (nightly and on "refresh from AD now"), session-refresh
(15 minutes), integrity (seals, asset hashes, missing and orphan files, the
audit chain), backup-check, notify-send and notify-digest. Notifications
route per event type to AD groups, users, mailboxes or the department
Supervisor group, immediately or in a daily digest, over STARTTLS verified
against the PESCO CA, with dry run on the test stack.

Backups: scripts/backup/ps-backup.sh and ps-restore.sh with a systemd timer
at 02:00 America/Denver. Restore rehearsed on the test stack 2026-10-21.
CI: pip-audit and npm audit. pypdf 6.21.1 and certifi 2026.10.19.

Migration 0010. Closes the app part of I6, M3, O1, O2 and S3.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

spencerm added 1 commit 2026-10-27 20:05:22 +00:00

deps: pypdf 6.21.1 to 6.21.2 ... 7a43236630

The CI dependency scan flags GHSA-6c7f-9265-g8fw, GHSA-jr96-gvgh-j3r4
and GHSA-gmh7-3g9r-mjh5 against 6.21.1, all fixed in 6.21.2.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

spencerm added 1 commit 2026-10-27 20:18:55 +00:00

ci: re-run after two e2e jobs collided on the CI stack name ... 8528e38448

Co-Authored-By: Claude Opus 5.5 <[email protected]>

spencerm merged commit 288804d5f5 into main 2026-10-27 20:54:47 +00:00

spencerm deleted branch phase-9 2026-10-27 20:54:47 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

decision: Clerical

decision: IT

decision: Q.C.

decision: Seth

phase 10

phase 3

phase 4

phase 5

phase 6

phase 7

phase 8

phase 9

production

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

SethD (Seth Dostaler) spencerm (Spencer McGuire)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: spencerm/PacketStation#49