Harden enterprise SAML & SCIM GA path (rebased re-port of #92)

[Bobcatsfan33]

Summary

This is a clean re-port of #92 (codex/enterprise-idp-ga) onto the refactored main. It supersedes #92, which was stale against main (routes were decomposed out of api.py into api_routers/*, and main independently added SCIM filter/patch + role-mapping and a hardened preflight). The original #92 branch no longer applies cleanly; this branch carries its security behavior forward on top of main's current architecture.

Do not merge without human review — this is security-critical auth code (SAML SSO + SCIM provisioning).

What was ported from #92

• SAML (modules/auth/saml.py) — durable AuthnRequest state + relay-state tables, RelayState host allow-list validation (open-redirect defense), single-use InResponseTo consumption, assertion replay protection (saml_assertion_replay table), Destination/Recipient checks against the ACS URL, IdP-initiated disabled by default, and in_response_to/assertion_id exposed on the parsed assertion.
• SAML ACS route — now lives in api_routers/enterprise.py (not api.py). Parses RelayState, passes it to parse_assertion, emits AUTH_SUCCESS/AUTH_FAILURE audit events, and returns in_response_to.
• SCIM (modules/auth/scim.py) — replaced the in-memory dict store with durable storage (scim_users/scim_groups via modules.storage.pg_connection), tenant-scoped uniqueness, ETag-style weak versions, and audit events for create/update/patch/delete.
• migrations.py / alembic baseline — register modules.auth.saml + modules.auth.scim init_db, add a 202605230001_enterprise_idp_ga migration (kept alongside main's api_key_roles migration).
• middleware.py — accept a tenant API key sent as Authorization: Bearer <key> (Okta/Entra/OneLogin SCIM clients) before falling through to JWT.
• requirements.txt — added defusedxml==0.7.1 and python3-saml==1.16.0 (kept main's newer pins: python-jose 3.5.0, cryptography 48.0.1, etc. — no downgrades).
• Dockerfile — xmlsec/libxml system build deps for python3-saml.
• preflight_prod.py — SAML gates (HTTPS SP URLs, IdP SSO URL + signing cert, RelayState allow-list, python3-saml + defusedxml runtime deps present, IdP-initiated off by default).
• New scripts/idp_ga_validation.py, tests/test_saml_scim.py additions, tests/test_preflight_prod.py additions, new tests/test_tenant_middleware.py, and docs.

How main's architecture was preserved

• SAML/SCIM edits applied to api_routers/enterprise.py (routes moved out of api.py).
• scim_filter.py and scim_patch.py are untouched and still drive filtering + PatchOp — durable storage was integrated under them.
• main's SCIM group→role mapping (TOKENDNA_SCIM_GROUP_ROLE_MAP_JSON) was re-implemented on durable storage: a new manual_roles_json column persists manually-assigned roles, and _sync_roles() recomputes effective roles = manual ∪ group-derived on every user/group write. main's test_scim_group_membership_syncs_roles passes unchanged.
• Kept main's newer dependency pins and hardened preflight (OIDC tenant-claim, compliance-profile, DoD/FedRAMP gates).

Test results (Python 3.10 sandbox)

• ruff check . — All checks passed.
• pytest tests/test_saml_scim.py tests/test_preflight_prod.py tests/test_tenant_middleware.py -q — all pass (28 + 14 = 42 passed).
• Broader run of auth/tenant/scim/saml/storage/migration/rbac tests — 254 passed. The only failures in the broader sweep are 7 pre-existing tests/test_v1_endpoints.py cases that fail identically on clean origin/main under the same -k filter (test-ordering/env-leakage, unrelated to this PR; they pass when run in isolation).

Note: the sandbox is Python 3.10, so requirements.txt (which pins numpy==2.4.3, etc. requiring 3.11+) could not be installed verbatim; compatible versions were used for the local run. CI on the target Python version should install the pinned set.

CodeQL hardening

The exact alert from #92 could not be fetched, so the new/changed code was proactively audited for the common CodeQL Python queries:

• XXE / XML-bomb (py/xxe, py/xml-bomb) — SAML response XML is parsed only with defusedxml. The original Harden enterprise SAML and SCIM GA path #92 fell back to the stdlib xml.etree parser when defusedxml was missing (an unsafe sink on attacker-controlled XML); that fallback was removed — if defusedxml is unavailable the code refuses to parse and logs an error. defusedxml is now a hard runtime dep and a preflight check.
• Clear-text logging of sensitive data (py/clear-text-logging-sensitive-data) — verified the raw SAMLResponse/assertion body and bearer credentials are never logged. Audit events record only identifiers (name_id, issuer, session_index, assertion_id) as is standard for auth audit trails.
• SQL injection — all SAML/SCIM SQL uses parameterized ? placeholders through AdaptedCursor; no f-string/%-format SQL was introduced.
• SSRF / open redirect — RelayState is constrained to relative paths or an explicit HTTPS host allow-list; the SP-initiated redirect only targets the configured IdP SSO URL.

⚠️ NEEDS HUMAN REVIEW

• SCIM durable-storage + filter/patch + role-mapping integration (highest priority). The in-memory role sync was re-implemented on SQL. Please review _sync_roles(), the new manual_roles_json column, and how patch_user feeds replace_user. Known behavioral quirk (preserved from main, not introduced here): because scim_patch.apply_patch echoes the full resource, a PATCH that doesn't touch roles still rebases the user's current effective roles into their manual roles. This exactly reproduces main's prior in-memory behavior (verified by running main's module directly), but a reviewer may want to decide whether GA should instead keep manual/derived strictly separate across PATCH.
• _sync_roles performance/consistency. It recomputes all users in a tenant on each user/group write and does not bump version/updated_at for role-only changes (matches main). Fine for provisioning volumes; confirm acceptable at scale and under Postgres (the sandbox verified SQLite only — Postgres path is code-review-only here).
• Middleware bearer key-lookup. I wrapped store.lookup_by_key(bearer.credentials) in try/except so a genuine JWT bearer falls through cleanly instead of 500-ing when the key probe errors (this was required to keep test_tenant_auth_rbac.py green). Confirm swallowing lookup errors here is acceptable.
• manual_roles_json migration. scim_users is a brand-new durable table (was in-memory), so there is no legacy data to migrate; init_db creates the full schema fresh. Confirm no environment already created a scim_users table without this column.
• defusedxml-only parsing. _extract_response_state returns {} if defusedxml is missing, which would break replay/request bookkeeping. This is intended (fail-closed) and guarded by a preflight check, but confirm the production image always ships defusedxml.
• SAML not_before/not_after are taken from the unauthenticated Conditions extraction for bookkeeping after python3-saml signature validation — confirm this matches your trust model (crypto trust stays with python3-saml).
• Docs (docs/ops/saml-scim.md) were rewritten to GA wording and re-merged with main's OIDC/SCIM role-map sections; please sanity-check the runbook.
• Full end-to-end SAML with a real IdP + python3-saml (xmlsec) was not exercised in the sandbox (dependency needs system libs). scripts/idp_ga_validation.py is the intended live harness.

Minor: the Dockerfile # ── Stage 1 ── comment separator differs from the source branch by a single box-drawing dash (cosmetic comment only; no build/CI/security impact).