spencerm/pesco-ncr
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: PESCO NCR system

Browse Source 
Complete Non-Conformance Report system replacing the PowerApps/SharePoint
prototype: FastAPI + SQLAlchemy 2 (async) + Alembic + MySQL 8 backend,
React 18 + Vite + TypeScript + MUI frontend, Entra ID auth (MSAL / JWKS,
group-gated), Microsoft Graph delegated Mail.Send notifications (OBO),
six-stage workflow state machine with server-side enforcement, atomic
NCR-YYYY-NNNN numbering, attachments with camera capture, immutable
field-level audit trail, admin reopen, reports + CSV export, WeasyPrint
PDF traveler, Power BI reporting views + read-only DB user, documented
VISUAL ERP job-lookup stub, pytest suite (26 tests), docker-compose
deployment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
ang3l12
2026-07-13 11:41:22 -06:00
commit dea316b113
111 changed files with 13817 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

308
README.md Normal file
View File

@@ -0,0 +1,308 @@
# PESCO NCR — Non-Conformance Report System
A web-based Non-Conformance Report (NCR / "QN") system for PESCO, replacing the
PowerApps/SharePoint prototype. Shop-floor and office users log nonconformance
issues on jobs, route them through disposition review, operations rework, QC
inspection, and costing, then close them — with full audit history, email
notifications, reporting, and Power BI access.
## Contents
- [Architecture](#architecture)
- [Quick start (local demo, no Entra required)](#quick-start-local-demo-no-entra-required)
- [Production setup](#production-setup)
- [Entra ID app registration](#entra-id-app-registration)
- [Email notifications (delegated Graph send)](#email-notifications-delegated-graph-send)
- [Workflow & roles](#workflow--roles)
- [Migrations & seed data](#migrations--seed-data)
- [Power BI](#power-bi)
- [Future VISUAL ERP integration](#future-visual-erp-integration)
- [Backend tests](#backend-tests)
- [Development outside Docker](#development-outside-docker)
- [Troubleshooting](#troubleshooting)
## Architecture
```
┌────────────┐ HTTPS ┌─────────────────────┐ ┌──────────────┐
│ Browser │ ─────────▶ │ frontend (nginx) │ │ Entra ID │
│ React SPA │ │ - serves built SPA │ │ (OIDC/JWKS) │
│ MSAL │ │ - proxies /api ────┼──┐ └──────▲───────┘
└────────────┘ └─────────────────────┘ │ │ token
▼ │ validation,
┌─────────────────────────────┐ │ OBO exchange
│ api (FastAPI, SQLAlchemy 2) │ ──────┘
│ - state machine, RBAC │ ──▶ Microsoft Graph
│ - audit trail, numbering │ (delegated
│ - WeasyPrint PDF, reports │ Mail.Send)
└───────┬──────────────┬───────┘
│ │
┌─────────▼───────┐ ┌───▼──────────────┐
│ MySQL 8 (volume)│ │ attachments │
│ + reporting │ │ (named volume) │
│ views for BI │ └──────────────────┘
└─────────────────┘
```
| Piece | Tech |
|---|---|
| Backend | Python 3.12, FastAPI, SQLAlchemy 2 (async, aiomysql), Alembic, Pydantic v2 |
| Frontend | React 18 + Vite + TypeScript, MUI, React Router, TanStack Query, TipTap, Recharts |
| Auth | Entra ID (OIDC) — `@azure/msal-react` in the SPA, `python-jose`/JWKS validation in the API |
| Email | Microsoft Graph **delegated** `Mail.Send` via the On-Behalf-Of flow |
| PDF | WeasyPrint (HTML → PDF) |
| Database | MySQL 8 (utf8mb4), named volume; SQLite used only by the test suite |
Key backend modules:
- `backend/app/domain.py` — roles, stages, allowed transitions
- `backend/app/services/workflow.py` — the state machine (all transitions validated server-side)
- `backend/app/services/numbering.py` — atomic `NCR-YYYY-NNNN` allocation (per-year row lock)
- `backend/app/services/notifications.py` — fault-tolerant Graph notifications
- `backend/app/services/job_lookup.py` — `JobLookupService` seam for the future VISUAL integration
- `backend/app/routers/ncrs.py` — NCR endpoints (create, queues, stage actions, attachments, audit, CSV, PDF)
API docs (OpenAPI/Swagger) are served at **`/api/docs`**.
## Quick start (local demo, no Entra required)
Runs the full stack with **dev auth** (a user switcher instead of Entra —
never use outside a lab):
```bash
cp .env.example .env
# In .env set:
# AUTH_MODE=dev
# MYSQL_PASSWORD / MYSQL_ROOT_PASSWORD / POWERBI_RO_PASSWORD → anything
docker compose up -d --build
# seed departments/categories, demo users, and sample NCRs (SEED_DEMO_DATA=true)
docker compose exec api python -m app.seed
```
Open **http://localhost:8080**. The switcher in the top bar signs you in as any
demo user (`admin@`, `dispo@`, `second@`, `ops@`, `qc@`, `cost@`,
`req@pescoinc.biz`) so you can walk an NCR through the whole workflow.
## Production setup
1. Complete the [Entra ID app registration](#entra-id-app-registration) below.
2. `cp .env.example .env` and fill in everything marked `__LIKE_THIS__`:
- `AUTH_MODE=entra`
- `ENTRA_TENANT_ID`, `ENTRA_CLIENT_ID`, `ENTRA_CLIENT_SECRET`
- `ENTRA_ALLOWED_GROUP_ID` — object ID of the `NCR-Users` security group
- `INITIAL_ADMIN_EMAILS` — who gets the Admin role on first sign-in
- `APP_BASE_URL` — the URL users browse to (used in email links)
- strong MySQL + Power BI passwords
3. `docker compose up -d --build` — migrations run automatically on API start.
4. `docker compose exec api python -m app.seed` — seeds departments/categories
(set `SEED_DEMO_DATA=false` first to skip demo users/NCRs).
5. Sign in with an `INITIAL_ADMIN_EMAILS` account, open **Admin → Users & Roles**,
and assign Disposition Authority / Operations / QC Inspector / Costing roles.
Put TLS in front of the `frontend` service (reverse proxy or load balancer) and
update the Entra redirect URI + `APP_BASE_URL` to the HTTPS URL.
## Entra ID app registration
One app registration serves both the SPA and the API.
1. **Create the registration** — Azure portal → Entra ID → App registrations →
*New registration*. Name: `PESCO NCR`. Supported account types: *single
tenant*.
2. **SPA redirect URIs** — Authentication → *Add a platform* →
**Single-page application** → add:
- `http://localhost:8080` (or your `APP_BASE_URL`)
- your production URL, e.g. `https://ncr.pescoinc.biz`
3. **Expose the API** — Expose an API → *Set* the Application ID URI to the
default `api://<client-id>` → *Add a scope*:
- Scope name: `access_as_user`
- Who can consent: Admins and users
- Display name/description: "Access the PESCO NCR API as the signed-in user"
4. **API permissions** — *Add a permission* → Microsoft Graph → **Delegated**:
- `User.Read` (usually present already)
- `Mail.Send` — required for stage-transition emails
- `GroupMember.Read.All` — optional; only needed for the group-overage
fallback (users in >200 groups)
Then click **Grant admin consent**.
5. **Client secret** — Certificates & secrets → *New client secret* → put the
value in `ENTRA_CLIENT_SECRET`. (Used by the API for the OBO exchange and
overage checks; the SPA never sees it.)
6. **Groups claim** — Token configuration → *Add groups claim* → select
**Security groups** (for both ID and access tokens). If your users belong to
many groups, prefer **Groups assigned to the application** and assign
`NCR-Users` to the app (Enterprise application → Users and groups) to avoid
claim overage.
7. **Access-token version** — the API accepts both v1 and v2 issuers. For
clean v2 tokens set `"accessTokenAcceptedVersion": 2` in the app manifest.
8. **Front-door group** — create (or reuse) a security group such as
`NCR-Users`, add everyone who may use the app, and put its **object ID** in
`ENTRA_ALLOWED_GROUP_ID`. Users outside the group get "Access denied" even
with a valid token.
Users are auto-provisioned in the local DB on first sign-in (OID, name, email,
employee ID when present in the token) with the default **Requester** role.
## Email notifications (delegated Graph send)
Notifications are sent **from the mailbox of the user who performed the
action**, using the **On-Behalf-Of (OBO) flow** — chosen over passing
frontend-acquired Graph tokens because it keeps Graph scopes and token plumbing
entirely server-side: the SPA only ever requests the API scope, and the API
exchanges the incoming access token for a delegated Graph token when it needs
to send mail (`backend/app/services/graph.py`).
| Event | Recipients | Sent from |
|---|---|---|
| New request submitted | selected Disposition Authority | requester |
| Secondary review assigned | "Notify These People" | initial reviewer |
| Released to Operations | all Operations users | releasing reviewer |
| Operations complete | all QC Inspectors | operations user |
| QC closed | all Costing users | QC inspector |
| NCR closed | original requester | costing user |
| Admin reopen | owners of the target stage + requester | admin |
Fault tolerance: a Graph failure **never blocks a workflow transition** — the
transition is already committed; the failure is logged and returned in the
response `warnings` array, which the UI shows as a toast. The Admin → Settings
screen has a global on/off toggle (handy during testing).
## Workflow & roles
Stages (enforced server-side; invalid transitions are rejected with HTTP 409):
```
New Request ──(initial disposition)──┬── needs secondary review ──▶ Secondary Disposition ─┐
└───────────── no ──────────────────▶ Operations ◀────┘
Operations ─▶ QC Inspection ─▶ Costing ─▶ Closed (locked; Admin-only reopen with reason)
```
Notes:
- "Initial Disposition" is the review a **Disposition Authority** performs on
an NCR sitting in the *New Request* queue (QC authority, work order,
rich-text disposition notes, secondary-review decision).
- **Secondary Disposition** is visible only to the assigned "Notify These
People" users (their personal queue) and Admins; they may update disposition
fields and release to Operations.
- **QC Inspection** can be saved repeatedly until *QC Closed* advances it.
- Saving **Costing** (Labor/Material/Service/Other) closes the NCR. Closed
NCRs are fully read-only — including attachments — until an Admin reopens
them (required reason, recorded in the audit trail).
- Every stage change writes a `stage_transitions` row (timestamp + acting
user) — the basis for the aging and cycle-time reports — and every field
change writes an immutable `audit_log` row (before/after values). The app
exposes no way to edit or delete audit rows.
Roles (assigned in **Admin → Users & Roles**; a user may hold several):
Requester (default), Disposition Authority, Secondary Disposition Authority,
Operations, QC Inspector, Costing, Admin.
## Migrations & seed data
```bash
# run migrations manually (they also run on every api container start)
docker compose exec api alembic upgrade head
# seed lookups (+ demo data when SEED_DEMO_DATA=true)
docker compose exec api python -m app.seed
# create a new migration after model changes
docker compose exec api alembic revision --autogenerate -m "describe change"
```
## Power BI
The schema ships three **read-only flattened views** for external reporting:
| View | Grain |
|---|---|
| `vw_ncr_full` | one row per NCR — all stage data, costs, computed `total_cost`, `days_in_stage`, ERP enrichment |
| `vw_ncr_stage_history` | one row per stage transition (for cycle-time analysis) |
| `vw_ncr_costs` | one row per costed NCR (cost of nonconformance) |
A dedicated MySQL account **`powerbi_ro`** is created on first startup
(`db/init/01-powerbi-user.sh`) with `SELECT` on exactly those views and nothing
else. If your MySQL volume was initialized before you set
`POWERBI_RO_PASSWORD`, run `scripts/powerbi_grants.sql` (instructions inside).
Pointing the gateway at it:
1. MySQL is published on `MYSQL_PUBLISHED_PORT` (default 3306). Firewall it so
only the Power BI gateway host can reach it.
2. On the gateway machine install the MySQL .NET connector
(Connector/NET 8.x — required for `caching_sha2_password`).
3. In Power BI Desktop: *Get data → MySQL database* → server
`<docker-host>:3306`, database `pesco_ncr`, user `powerbi_ro`.
4. Import (or DirectQuery) the three `vw_*` views; schedule refresh through the
gateway.
## Future VISUAL ERP integration
Job Number is free text today. The integration seam is already in place:
- `backend/app/services/job_lookup.py` defines the `JobLookupService`
protocol. The default **`NullJobLookupService`** returns no enrichment.
**`VisualJobLookupService`** is a documented stub containing the verified
VISUAL 10 query plan (WORK_ORDER composite key `TYPE/BASE_ID/LOT_ID/
SPLIT_ID/SUB_ID`, PART join, and the DEMAND_SUPPLY_LINK →
CUST_ORDER_LINE → CUSTOMER_ORDER → CUSTOMER customer linkage).
- The schema stores the job number as entered plus a nullable `job_info` row
(part, description, customer, WO status) any provider can populate.
- The SPA's job-number field already calls `GET /api/jobs/{job}/lookup` while
typing and displays whatever enrichment returns — validation/autocomplete
light up without a redesign.
To enable later: implement the stub (read-only SQL Server access — never write
to VISUAL tables), set `JOB_LOOKUP_PROVIDER=visual` plus the `VISUAL_DB_*`
variables, and restart the API.
## Backend tests
Covers the state machine (happy paths, invalid transitions, closure locking,
reopen), per-stage permission enforcement, NCR numbering (format, year
rollover, 12-way concurrent submission), attachments, and rich-text
sanitization.
```bash
cd backend
python -m venv .venv && .venv/bin/pip install -r requirements-dev.txt
.venv/bin/pytest
```
Tests run against SQLite by default (same models/state machine/numbering code
paths); to exercise real MySQL locking:
```bash
DATABASE_URL="mysql+aiomysql://user:pass@host/pesco_ncr_test?charset=utf8mb4" .venv/bin/pytest
```
## Development outside Docker
```bash
# API (SQLite works fine for dev; export the vars or put them in backend/.env)
cd backend
export DATABASE_URL="sqlite+aiosqlite:///dev.db" AUTH_MODE=dev ATTACHMENTS_DIR=./attachments
.venv/bin/python -m app.dev_init # create tables (models → SQLite)
SEED_DEMO_DATA=true .venv/bin/python -m app.seed # demo users + sample NCRs
.venv/bin/uvicorn app.main:app --reload --port 8000
# SPA (proxies /api to :8000; public/config.js defaults to dev auth)
cd frontend
npm install
npm run dev # http://localhost:5173
```
Note: generating PDFs locally requires WeasyPrint's system libraries (Pango/
Cairo — `brew install pango` on macOS). The Docker image includes them.
## Troubleshooting
| Symptom | Likely cause / fix |
|---|---|
| "Access token has no groups claim" | Add the groups claim in Token configuration (step 6 above). |
| "Could not verify group membership (group overage)" | Grant delegated `GroupMember.Read.All` + admin consent, or scope the group claim to "Groups assigned to the application". |
| Notification warning "OBO token exchange failed" | Check `ENTRA_CLIENT_SECRET`, and that `Mail.Send` has admin consent. |
| `api` container restarts at boot | MySQL still initializing; the entrypoint retries migrations 12×. Check `docker compose logs mysql`. |
| Power BI can't authenticate | Update Connector/NET (needs `caching_sha2_password`), verify the `powerbi_ro` grants (`scripts/powerbi_grants.sql`). |
| Uploads fail at ~25 MB | Raise `MAX_UPLOAD_MB` (API) — nginx `client_max_body_size` is 50 MB in `frontend/nginx.conf`. |