driftguard

Lightweight, self-hosted drift detection for production ML models.

driftguard wraps any model with a .predict(X) method, snapshots a baseline distribution at training time, and continuously scores feature and prediction drift in production. When PSI, KS, JS, or prediction shift breach configurable thresholds, it raises throttled alerts to a webhook or SMTP server and records every event in a local SQLite store you can inspect from a live terminal dashboard or the CLI.

Why driftguard

If you ship models and you do not have the budget for an Arize, WhyLabs, or Evidently account but you still need to know when your credit-scoring model quietly stops behaving like the data it was trained on, this library is for you. No SaaS account, no cloud bucket, no Kafka topic. One Python dependency (numpy), one SQLite file, one CLI.

• Pure NumPy detectors. No SciPy, no pandas required at runtime.
• Single-file SQLite store. Works on a laptop, in CI, or on a small VM.
• Built-in webhook + SMTP alerting with per-signal throttling.
• Live terminal dashboard powered by rich.
• A small DSL (psi[income]>0.2) for declaring thresholds.

Install

pip install driftguard

For pandas DataFrame baselines:

pip install "driftguard[pandas]"

Quickstart

import numpy as np
from sklearn.linear_model import LinearRegression

from driftguard import Baseline, SQLiteStore, WebhookAlerter, guard

# 1. Train a model on historical data.
X_train = np.load("train_X.npy")          # shape (n, 3)
y_train = np.load("train_y.npy")
model = LinearRegression().fit(X_train, y_train)

# 2. Capture the reference distribution at training time.
baseline = Baseline.from_array(
    X_train,
    feature_names=["income", "age", "debt_ratio"],
    predictions=model.predict(X_train),
)
baseline.save("baseline.json")

# 3. Wrap the model. Drift checks run every 1000 predictions.
store = SQLiteStore("driftguard.db")
alerter = WebhookAlerter("https://hooks.slack.com/services/XXX")

guarded = guard(
    model,
    baseline,
    thresholds=[
        "psi[income]>0.2",
        "psi[age]>0.2",
        "ks[debt_ratio]>0.15",
        "shift>2.0",
    ],
    alerters=[alerter],
    store=store,
    check_every=1000,
    window_size=5000,
)

# 4. Use it like the original model.
predictions = guarded(X_production)

When the income distribution starts to look unlike training data, you get a webhook ping like:

{"metric":"psi","feature":"income","value":0.31,"threshold":0.2,"breached":true,"timestamp":1736000000.0}

Drift metrics

Metric	What it measures	Sensible threshold
psi	Population Stability Index using quantile bins from baseline	> 0.2 is meaningful, > 0.25 is significant
ks	Kolmogorov-Smirnov two-sample statistic	> 0.1 worth watching, > 0.2 significant
js	Jensen-Shannon divergence between histograms (base 2, in [0,1])	> 0.1 worth watching
shift	abs(current_pred_mean - baseline_pred_mean) / baseline_pred_std	> 1.0 worth watching, > 2.0 significant

All four are NaN-safe and operate on raw NumPy arrays — no scipy needed.

Threshold DSL

Thresholds are short strings parsed by parse_threshold:

psi[income]>0.2          # PSI for the `income` feature must stay <= 0.2
ks[debt_ratio]>=0.15     # KS statistic for `debt_ratio`
js>0.1                   # JS divergence on every feature in the baseline
shift>2.0                # absolute prediction shift in baseline-std units

You can pass any mix of strings or Threshold objects to guard(...).

Alerting

from driftguard import WebhookAlerter, EmailAlerter

webhook = WebhookAlerter(
    url="https://hooks.slack.com/services/XXX",
    headers={"Authorization": "Bearer ..."},
)

email = EmailAlerter(
    smtp_host="smtp.example.com",
    smtp_port=587,
    username="alerts@example.com",
    password="...",
    from_addr="alerts@example.com",
    to_addrs=["ml-oncall@example.com"],
)

Throttling is keyed on (channel, metric, feature) with a configurable cooldown (default: 5 minutes). A flapping psi[income] will not page you every 1000 predictions.

Dashboard

driftguard watch driftguard.db

+-- PSI per feature ------------------+  +-- Recent predictions -----+
| income     0.31 ████████░░░  0.20 X |  |       █                   |
| age        0.05 █░░░░░░░░░░  0.20   |  |     █ █ █                 |
| debt_ratio 0.08 ██░░░░░░░░░  0.20   |  |   █ █ █ █ █               |
+-------------------------------------+  +---------------------------+
+-- Breached signals (last 24h) -----------------------------------+
| 14:01:33  psi   income   0.31  0.20                              |
| 14:00:18  ks    income   0.22  0.15                              |
+------------------------------------------------------------------+

CLI reference

Command	What it does
driftguard watch <store.db>	Open the live dashboard.
driftguard report <store.db> --since 24h	Print a summary table for the window.
driftguard alerts <store.db> --tail 50	Show the most recent alert deliveries.
driftguard purge <store.db> --older-than 30d	Delete old rows.

Durations parse as <n>s, <n>m, <n>h, or <n>d.

How driftguard compares

	driftguard	Evidently	WhyLabs	Arize
Open source	yes	partial	no	no
Self-hosted by default	yes	yes	no	no
Single-file storage	yes (SQLite)	optional	no	no
Built-in alerting	yes	partial	yes	yes
Live terminal UI	yes	no	no	no
Free for production	yes	yes	tier	no
Hosted dashboards	no	no	yes	yes
Multi-tenant SaaS	no	no	yes	yes

If you need cross-team SaaS observability with hosted dashboards, pay for Arize or WhyLabs. If you have one model and a deadline, use driftguard.

FAQ

Is it production-ready? It runs in-process and uses SQLite, which is appropriate for single-instance services. For multi-replica deployments, point each instance at its own store and aggregate with report.

Does it ship with scipy? No. Detectors are pure NumPy.

Why a baseline file? Baselines belong in your model registry next to the weights. JSON is durable, diffable, and tiny.

Does it support categorical features? Encode them as integers before passing to Baseline.from_array. PSI and KS work on the encoded values.

Does it handle concept drift? No — only data and prediction drift. Concept drift requires labels, which arrive late in most production systems. driftguard is designed to alert before you have ground truth.

License

MIT. Copyright 2026 Louis Chifura.