-
Notifications
You must be signed in to change notification settings - Fork 1
Design Decisions
The choices that define how SCM-Master uses AI and handles money — each made on evidence and documented in the repo's docs/. The throughline: prefer the legible, tested, auditable option, and write down why.
Decision: an AI may propose (confidence, recommendation, rationale) and is structurally forbidden from deciding. Supplier comes from the sourcing service, quantity from net-demand + MOQ, price from the contract, cost floor from tested commodity math, and the place/stage/escalate disposition from a deterministic gate keyed on spend caps, a confidence floor, an approved-source check, and storage headroom.
Why: an AI is good at reading a messy quote and bad with money. Money decisions must be tested, auditable, and defensible to a CFO. The boundary is regression-tested by the agent-safety harness.
Decision: agent/confidence.py computes the auto-place confidence from the buy's evidence (sole-source? full contract data? observed vs forecast demand? fits storage?), returning a factor-by-factor audit trail. The LLM's self-reported confidence is recorded but never gates.
Why: a model that hallucinates 0.99 must not be able to trigger a buy. The score is an auditable artifact (persisted on the DecisionLog), and garbage / unavailable LLM output can't force an auto-place — the €200k ceiling is the brake. Detail in docs/deterministic-confidence.md.
Decision: auto-place when confidence ≥ 0.90 AND order < €200k, else a human approves. Under the single-ceiling policy auto_place_spend_cap == escalate_spend_threshold == €200k.
Why: a rule a buyer can read and a CFO can sign off on. The €200k is the working figure, pending management sign-off — honestly flagged as such in the code comments.
Decision: adopt Nixtla statsforecast (Croston/SBA + conformal intervals) for intermittent/lumpy SKUs, flag-gated behind FORECAST_ENGINE (default builtin).
Evidence (docs/forecast-engine-decision.md), benchmarked vs the hand-rolled TSB before adoption:
| 6 SKUs (local) | 1000 SKUs × 180 days (scale) | |
|---|---|---|
| Backtest accuracy (MAE) | tie (proves our TSB is correct) | statsforecast ~24% lower error on the lumpy tail |
| Speed | — | built-in TSB ~125× faster (12 ms vs 1.52 s) |
| Prediction intervals | unusable (too little data) | 95% conformal coverage (90% target) |
Why: the 6-SKU tie was a small-sample artefact; at scale the accuracy gap opens (~21% less mis-ordering per cycle on illustrative economics) and conformal intervals become trustworthy. Adopted for accuracy at scale + probabilistic safety stock at zero LLM-token cost — not for speed (the built-in TSB wins there and is the instant rollback).
Subtlety: the separate internal forecast-method backtest (run_rate vs tsb) found run-rate beats TSB on representative demand, so run-rate stays the default forecast_method. The real lesson: lumpy demand isn't point-forecastable — it's absorbed by service-level safety stock, not forecasts.
Decision: build the rule-based learning loop + the ML seam now; defer the ML model itself.
| Approach | Training data? | Auditable? | Right for tabular procurement? | Verdict |
|---|---|---|---|---|
| Rule-based (today) | No — works day one | Yes — read the rule | Yes | Built & running |
| Gradient-boosted trees (LightGBM + SHAP) | Yes — from RequisitionFeedback
|
Yes — SHAP per-feature attribution | Yes — beats nets on this shape |
Built in shadow mode (calibration_ml.py, flag off, declines when undertrained) |
| Deep learning (neural confidence) | Yes — thousands–millions of rows | No — "the MLP said 0.87" fails the CFO test | No — overkill, weaker on tabular | Rejected |
Why not now: (1) no outcome data — the DecisionLog is days old and dry-run-by-default, so a model trained on it would be worse than the rule; (2) deep learning is the wrong tool for tabular, low-frequency data (the forecast benchmark showed the same — classical stats beat LSTMs); (3) auditability is the whole thesis. confidence.py's factor breakdown is the feature vector a model would consume, and calibrate() is a pure function an ML calibrator replaces at the same signature — so ML is a drop-in, not a rewrite. Full evaluation in docs/autonomy-and-learning.md.
Decision: every LLM call goes through agent/client.py::call_claude (model claude-sonnet-4-6, env-overridable via ANTHROPIC_MODEL). With no ANTHROPIC_API_KEY the agent runs deterministically (templated narration), boots fast, costs zero tokens — used for seed-on-boot and CI. call_claude never raises — failures return an [agent-error] ... string. The long system prompt is marked cacheable (Anthropic ephemeral prompt caching) to bill repeat calls at a fraction.
Why: forecasting and every spend decision use zero LLM tokens (pure CPU math); the only token cost is per-line narration, which collapses to a template when the key is absent. The bill scales with explanation, not with operations.
Decision: agent/grounding.py forces any decision-critical number the model emits (quantity, shortfall) onto the code-computed value before anything reads it; mismatches are logged so the override rate is visible. Only named critical fields are grounded, so qualitative figures ("ETA 22d") don't false-trip.
Why: prefer templating (callers fill critical slots from the computed object); this is the strict backstop closing a pre-existing hole where the model's qty was rendered verbatim.
Found by an evidence-first review (each finding verified against source before any fix), all fixes additive and backward-compatible (333 tests stay green):
| Fix | What it does |
|---|---|
| Row-locked write guards | Receiving + every lifecycle transition take SELECT … FOR UPDATE on the order line / asset row, so two concurrent receipts can't both pass the over-receipt guard. Real work on Postgres, no-op on SQLite |
| Pooled, health-checked DB |
pool_pre_ping (replace a stale Railway connection) + explicit pool_size / max_overflow headroom |
| One query, not N | Spend analytics + inbound-pipeline views aggregate in one grouped query (killed the N+1) — identical numbers, milliseconds vs seconds at 100k assets |
| Indexed hot paths | The four analytics/capacity/provenance filter columns indexed via an additive migration |
Product (spec) is kept separate from ProductSupplier (one source). "Replacing a supplier" — critical under spiky demand and long chip lead times — is then a one-field repoint, without losing the product's identity or purchase history. See Data-Model.
The portfolio rollup exposes two labelled ratios: total_cost_pct (ΣTCO ÷ baseline, includes acquisition) and tscmc_pct (Σ(TCO − acquisition) ÷ baseline). The SCOR/APQC Total Supply-Chain Management Cost deliberately excludes acquisition (the COGS analog) — so the metric is named and computed to the standard, not invented. See Workflows.
Honest framing. The repo is a working MVP / capstone deliverable, not a production-graded SaaS. The €200k cap is a working figure; SAP inbound and Coupa write-back are designed but unbuilt; ML is a tested seam, not a live model. Each of these is flagged as such in the code and docs rather than overclaimed.
SCM-Master — Hardware procurement & asset-lifecycle SCM with an AI decision layer · Repository · Built by Eugen Müller
Overview
Deep dives