pesco-ncr/README.md

# PESCO NCR — Non-Conformance Report System

A web-based Non-Conformance Report (NCR / "QN") system for PESCO, replacing the
PowerApps/SharePoint prototype. Shop-floor and office users log nonconformance
issues on jobs, route them through disposition review, operations rework, QC
inspection, and costing, then close them — with full audit history, email
notifications, reporting, and Power BI access.

## Contents

- [Architecture](#architecture)
- [Quick start (local demo, no Entra required)](#quick-start-local-demo-no-entra-required)
- [Production setup](#production-setup)
- [Entra ID app registration](#entra-id-app-registration)
- [Email notifications (delegated Graph send)](#email-notifications-delegated-graph-send)
- [Workflow & roles](#workflow--roles)
- [Migrations & seed data](#migrations--seed-data)
- [Power BI](#power-bi)
- [Future VISUAL ERP integration](#future-visual-erp-integration)
- [Backend tests](#backend-tests)
- [Development outside Docker](#development-outside-docker)
- [Troubleshooting](#troubleshooting)

## Architecture

```
┌────────────┐   HTTPS    ┌─────────────────────┐        ┌──────────────┐
│  Browser   │ ─────────▶ │ frontend (nginx)     │        │  Entra ID    │
│ React SPA  │            │  - serves built SPA  │        │  (OIDC/JWKS) │
│ MSAL       │            │  - proxies /api ────┼──┐     └──────▲───────┘
└────────────┘            └─────────────────────┘  │            │ token
                                                   ▼            │ validation,
                          ┌─────────────────────────────┐       │ OBO exchange
                          │ api (FastAPI, SQLAlchemy 2)  │ ──────┘
                          │  - state machine, RBAC       │ ──▶ Microsoft Graph
                          │  - audit trail, numbering    │     (delegated
                          │  - WeasyPrint PDF, reports   │      Mail.Send)
                          └───────┬──────────────┬───────┘
                                  │              │
                        ┌─────────▼───────┐  ┌───▼──────────────┐
                        │ MySQL 8 (volume)│  │ attachments      │
                        │ + reporting     │  │ (named volume)   │
                        │   views for BI  │  └──────────────────┘
                        └─────────────────┘
```

| Piece | Tech |
|---|---|
| Backend | Python 3.12, FastAPI, SQLAlchemy 2 (async, aiomysql), Alembic, Pydantic v2 |
| Frontend | React 18 + Vite + TypeScript, MUI, React Router, TanStack Query, TipTap, Recharts |
| Auth | Entra ID (OIDC) — `@azure/msal-react` in the SPA, `python-jose`/JWKS validation in the API |
| Email | Microsoft Graph **delegated** `Mail.Send` via the On-Behalf-Of flow |
| PDF | WeasyPrint (HTML → PDF) |
| Database | MySQL 8 (utf8mb4), named volume; SQLite used only by the test suite |

Key backend modules:

- `backend/app/domain.py` — roles, stages, allowed transitions
- `backend/app/services/workflow.py` — the state machine (all transitions validated server-side)
- `backend/app/services/numbering.py` — atomic `NCR-YYYY-NNNN` allocation (per-year row lock)
- `backend/app/services/notifications.py` — fault-tolerant Graph notifications
- `backend/app/services/job_lookup.py` — `JobLookupService` seam for the future VISUAL integration
- `backend/app/routers/ncrs.py` — NCR endpoints (create, queues, stage actions, attachments, audit, CSV, PDF)

API docs (OpenAPI/Swagger) are served at **`/api/docs`**.

## Quick start (local demo, no Entra required)

Runs the full stack with **dev auth** (a user switcher instead of Entra —
never use outside a lab):

```bash
cp .env.example .env
# In .env set:
#   AUTH_MODE=dev
#   MYSQL_PASSWORD / MYSQL_ROOT_PASSWORD / POWERBI_RO_PASSWORD → anything
docker compose up -d --build

# seed departments/categories, demo users, and sample NCRs (SEED_DEMO_DATA=true)
docker compose exec api python -m app.seed
```

Open **http://localhost:8080**. The switcher in the top bar signs you in as any
demo user (`admin@`, `dispo@`, `second@`, `ops@`, `qc@`, `cost@`,
`req@pescoinc.biz`) so you can walk an NCR through the whole workflow.

## Production setup

1. Complete the [Entra ID app registration](#entra-id-app-registration) below.
2. `cp .env.example .env` and fill in everything marked `__LIKE_THIS__`:
   - `AUTH_MODE=entra`
   - `ENTRA_TENANT_ID`, `ENTRA_CLIENT_ID`, `ENTRA_CLIENT_SECRET`
   - `ENTRA_ALLOWED_GROUP_ID` — object ID of the `NCR-Users` security group
   - `INITIAL_ADMIN_EMAILS` — who gets the Admin role on first sign-in
   - `APP_BASE_URL` — the URL users browse to (used in email links)
   - strong MySQL + Power BI passwords
3. `docker compose up -d --build` — migrations run automatically on API start.
4. `docker compose exec api python -m app.seed` — seeds departments/categories
   (set `SEED_DEMO_DATA=false` first to skip demo users/NCRs).
5. Sign in with an `INITIAL_ADMIN_EMAILS` account, open **Admin → Users & Roles**,
   and assign Disposition Authority / Operations / QC Inspector / Costing roles.

Put TLS in front of the `frontend` service (reverse proxy or load balancer) and
update the Entra redirect URI + `APP_BASE_URL` to the HTTPS URL.

## Entra ID app registration

One app registration serves both the SPA and the API.

1. **Create the registration** — Azure portal → Entra ID → App registrations →
   *New registration*. Name: `PESCO NCR`. Supported account types: *single
   tenant*.
2. **SPA redirect URIs** — Authentication → *Add a platform* →
   **Single-page application** → add:
   - `http://localhost:8080` (or your `APP_BASE_URL`)
   - your production URL, e.g. `https://ncr.pescoinc.biz`
3. **Expose the API** — Expose an API → *Set* the Application ID URI to the
   default `api://<client-id>` → *Add a scope*:
   - Scope name: `access_as_user`
   - Who can consent: Admins and users
   - Display name/description: "Access the PESCO NCR API as the signed-in user"
4. **API permissions** — *Add a permission* → Microsoft Graph → **Delegated**:
   - `User.Read` (usually present already)
   - `Mail.Send` — required for stage-transition emails
   - `GroupMember.Read.All` — optional; only needed for the group-overage
     fallback (users in >200 groups)
   Then click **Grant admin consent**.
5. **Client secret** — Certificates & secrets → *New client secret* → put the
   value in `ENTRA_CLIENT_SECRET`. (Used by the API for the OBO exchange and
   overage checks; the SPA never sees it.)
6. **Groups claim** — Token configuration → *Add groups claim* → select
   **Security groups** (for both ID and access tokens). If your users belong to
   many groups, prefer **Groups assigned to the application** and assign
   `NCR-Users` to the app (Enterprise application → Users and groups) to avoid
   claim overage.
7. **Access-token version** — the API accepts both v1 and v2 issuers. For
   clean v2 tokens set `"accessTokenAcceptedVersion": 2` in the app manifest.
8. **Front-door group** — create (or reuse) a security group such as
   `NCR-Users`, add everyone who may use the app, and put its **object ID** in
   `ENTRA_ALLOWED_GROUP_ID`. Users outside the group get "Access denied" even
   with a valid token.

Users are auto-provisioned in the local DB on first sign-in (OID, name, email,
employee ID when present in the token) with the default **Requester** role.

## Email notifications (delegated Graph send)

Notifications are sent **from the mailbox of the user who performed the
action**, using the **On-Behalf-Of (OBO) flow** — chosen over passing
frontend-acquired Graph tokens because it keeps Graph scopes and token plumbing
entirely server-side: the SPA only ever requests the API scope, and the API
exchanges the incoming access token for a delegated Graph token when it needs
to send mail (`backend/app/services/graph.py`).

| Event | Recipients | Sent from |
|---|---|---|
| New request submitted | selected Disposition Authority | requester |
| Secondary review assigned | "Notify These People" | initial reviewer |
| Released to Operations | all Operations users | releasing reviewer |
| Operations complete | all QC Inspectors | operations user |
| QC closed | all Costing users | QC inspector |
| NCR closed | original requester | costing user |
| Corrective action assigned | the CA owner | user who assigned it |
| Admin reopen | owners of the target stage + requester | admin |

Fault tolerance: a Graph failure **never blocks a workflow transition** — the
transition is already committed; the failure is logged and returned in the
response `warnings` array, which the UI shows as a toast. The Admin → Settings
screen has a global on/off toggle (handy during testing).

## Workflow & roles

Stages (enforced server-side; invalid transitions are rejected with HTTP 409):

```
New Request ──(initial disposition)──┬── needs secondary review ──▶ Secondary Disposition ─┐
                                     └───────────── no ──────────────────▶ Operations ◀────┘
Operations ─▶ QC Inspection ─▶ Costing ─▶ Closed (locked; Admin-only reopen with reason)
```

Notes:

- "Initial Disposition" is the review a **Disposition Authority** performs on
  an NCR sitting in the *New Request* queue (QC authority, work order,
  rich-text disposition notes, secondary-review decision).
- **Secondary Disposition** is visible only to the assigned "Notify These
  People" users (their personal queue) and Admins; they may update disposition
  fields and release to Operations.
- **QC Inspection** can be saved repeatedly until *QC Closed* advances it.
- The **CAPA section** (API Q1 §5.9.1.2 / §6.4.2) sits outside the stage
  sequence: QC Inspectors, Disposition Authorities, and Admins can edit it at
  any point before closure (`POST /ncrs/{ref}/capa`). It captures the root
  cause (+ 6M root cause category), the *Corrective Action Required?* Yes/No
  gate with justification, the action plan (owner + due date, owner is
  notified), effectiveness verification (server-stamped verifier/date), and
  the recurring-issue flag with links to prior NCRs.
- Saving **Costing** (Labor/Material/Service/Other) closes the NCR — but only
  once the CAPA gate passes: the CA question must be answered, and when the
  answer is Yes the plan must be complete and verified *effective* (HTTP 409
  otherwise). Closed NCRs are fully read-only — including attachments — until
  an Admin reopens them (required reason, recorded in the audit trail).
- Every stage change writes a `stage_transitions` row (timestamp + acting
  user) — the basis for the aging and cycle-time reports — and every field
  change writes an immutable `audit_log` row (before/after values). The app
  exposes no way to edit or delete audit rows.

Roles (assigned in **Admin → Users & Roles**; a user may hold several):
Requester (default), Disposition Authority, Secondary Disposition Authority,
Operations, QC Inspector, Costing, Admin.

## Migrations & seed data

```bash
# run migrations manually (they also run on every api container start)
docker compose exec api alembic upgrade head

# seed lookups (+ demo data when SEED_DEMO_DATA=true)
docker compose exec api python -m app.seed

# create a new migration after model changes
docker compose exec api alembic revision --autogenerate -m "describe change"
```

## Power BI

The schema ships three **read-only flattened views** for external reporting:

| View | Grain |
|---|---|
| `vw_ncr_full` | one row per NCR — all stage data, costs, computed `total_cost`, `days_in_stage`, ERP enrichment |
| `vw_ncr_stage_history` | one row per stage transition (for cycle-time analysis) |
| `vw_ncr_costs` | one row per costed NCR (cost of nonconformance) |

A dedicated MySQL account **`powerbi_ro`** is created on first startup
(`db/init/01-powerbi-user.sh`) with `SELECT` on exactly those views and nothing
else. If your MySQL volume was initialized before you set
`POWERBI_RO_PASSWORD`, run `scripts/powerbi_grants.sql` (instructions inside).

Pointing the gateway at it:

1. MySQL is published on `MYSQL_PUBLISHED_PORT` (default 3306). Firewall it so
   only the Power BI gateway host can reach it.
2. On the gateway machine install the MySQL .NET connector
   (Connector/NET 8.x — required for `caching_sha2_password`).
3. In Power BI Desktop: *Get data → MySQL database* → server
   `<docker-host>:3306`, database `pesco_ncr`, user `powerbi_ro`.
4. Import (or DirectQuery) the three `vw_*` views; schedule refresh through the
   gateway.

## Future VISUAL ERP integration

Job Number is free text today. The integration seam is already in place:

- `backend/app/services/job_lookup.py` defines the `JobLookupService`
  protocol. The default **`NullJobLookupService`** returns no enrichment.
  **`VisualJobLookupService`** is a documented stub containing the verified
  VISUAL 10 query plan (WORK_ORDER composite key `TYPE/BASE_ID/LOT_ID/
  SPLIT_ID/SUB_ID`, PART join, and the DEMAND_SUPPLY_LINK →
  CUST_ORDER_LINE → CUSTOMER_ORDER → CUSTOMER customer linkage).
- The schema stores the job number as entered plus a nullable `job_info` row
  (part, description, customer, WO status) any provider can populate.
- The SPA's job-number field already calls `GET /api/jobs/{job}/lookup` while
  typing and displays whatever enrichment returns — validation/autocomplete
  light up without a redesign.

To enable later: implement the stub (read-only SQL Server access — never write
to VISUAL tables), set `JOB_LOOKUP_PROVIDER=visual` plus the `VISUAL_DB_*`
variables, and restart the API.

## Backend tests

Covers the state machine (happy paths, invalid transitions, closure locking,
reopen), per-stage permission enforcement, NCR numbering (format, year
rollover, 12-way concurrent submission), attachments, and rich-text
sanitization.

```bash
cd backend
python -m venv .venv && .venv/bin/pip install -r requirements-dev.txt
.venv/bin/pytest
```

Tests run against SQLite by default (same models/state machine/numbering code
paths); to exercise real MySQL locking:

```bash
DATABASE_URL="mysql+aiomysql://user:pass@host/pesco_ncr_test?charset=utf8mb4" .venv/bin/pytest
```

## Development outside Docker

```bash
# API (SQLite works fine for dev; export the vars or put them in backend/.env)
cd backend
export DATABASE_URL="sqlite+aiosqlite:///dev.db" AUTH_MODE=dev ATTACHMENTS_DIR=./attachments
.venv/bin/python -m app.dev_init                      # create tables (models → SQLite)
SEED_DEMO_DATA=true .venv/bin/python -m app.seed      # demo users + sample NCRs
.venv/bin/uvicorn app.main:app --reload --port 8000

# SPA (proxies /api to :8000; public/config.js defaults to dev auth)
cd frontend
npm install
npm run dev            # http://localhost:5173
```

Note: generating PDFs locally requires WeasyPrint's system libraries (Pango/
Cairo — `brew install pango` on macOS). The Docker image includes them.

## Troubleshooting

| Symptom | Likely cause / fix |
|---|---|
| "Access token has no groups claim" | Add the groups claim in Token configuration (step 6 above). |
| "Could not verify group membership (group overage)" | Grant delegated `GroupMember.Read.All` + admin consent, or scope the group claim to "Groups assigned to the application". |
| Notification warning "OBO token exchange failed" | Check `ENTRA_CLIENT_SECRET`, and that `Mail.Send` has admin consent. |
| `api` container restarts at boot | MySQL still initializing; the entrypoint retries migrations 12×. Check `docker compose logs mysql`. |
| mysql dies instantly: "Fatal glibc error: CPU does not support x86-64-v2" | You overrode `MYSQL_IMAGE` to an Oracle Linux build (e.g. `mysql:8.4`) on a host without x86-64-v2. Remove the override — the default `mysql:8.0-debian` runs on any x86-64 CPU. |
| Power BI can't authenticate | Update Connector/NET (needs `caching_sha2_password`), verify the `powerbi_ro` grants (`scripts/powerbi_grants.sql`). |
| Uploads fail at ~25 MB | Raise `MAX_UPLOAD_MB` (API) — nginx `client_max_body_size` is 50 MB in `frontend/nginx.conf`. |