One small ML system per day on Hopsworks.
The usual way to catch a phishing site is to wait: crawl the page, wait for a user report, wait for it to land on a blocklist. By then it has already taken credentials. This system asks whether you can flag the domain earlier, at the one moment every HTTPS site announces itself to the world: when its TLS certificate is logged to Certificate Transparency. From the hostname read off that certificate, before the site serves a byte, a measurable fraction of phishing is already visible.
Every certificate issued on the public web streams through CT in real time. This reads that firehose, scores each freshly issued hostname, and pushes the scored events onto a Kafka topic the app tails like a live log. It retrains itself on a schedule and redeploys the better model on its own.
A hostname classifier over 15k balanced hosts: phishing from public abuse blocklists, benign sampled across nine CT logs, one host per registrable domain so no campaign leaks across the split. Holdout, not just cross-validation.
| metric | value |
|---|---|
| ROC-AUC (5-fold CV) | 0.769 |
| ROC-AUC (holdout) | 0.781 |
| precision @ the blind baseline's recall | 0.88 |
The blind baseline is the rule a defender ships: cheap TLD, or a brand lookalike, or a credential keyword in the host. On the same at-issuance information the model is cleaner at the baseline's catch rate (0.88 precision against 0.75) and reaches recall the keyword rule does not.
v1 (1842 rows) scored CV AUC 0.83; its negatives were dominated by one operator's
bulk infrastructure certs and were easy to separate. The larger, diversified set
(15k rows, 154k phishing registrables, CT negatives across nine logs) scores 0.77
and is more representative. The guardrails and the full data record are in
docs/HONESTY.md.
Read these before quoting the number anywhere.
- The label is a proxy. Positive means the host later appeared on a public abuse blocklist. Blocklists are incomplete, so some real phishing is unlabelled. That depresses measured recall; it does not inflate it.
- At issuance only. The model sees the hostname and the certificate, never the page. That is the claim: the early signal is enough to be useful, ahead of any crawl or report.
- Own-certificate only. It sees phishing that gets its own certificate. Phish on shared hosting, under a platform's wildcard, never appears in CT as its own cert, so it is out of scope by construction.
- A lead, not a verdict. A flag is a ranked lead seconds after issuance, not a conviction. Benign hosts can look suspicious.
An FTI (feature, training, inference) system on Hopsworks, built as a real-time stream that improves itself. Certificate Transparency is the source, a Kafka topic is the spine, the online feature store serves and logs, and a scheduled retrain keeps the served model honest and current.
flowchart LR
ct([Certificate Transparency logs]):::ext
lab([abuse blocklists]):::ext
subgraph FE[Feature]
direction TB
fp[feature pipeline] --> fg[(online Feature Group<br/>cert_features)]:::hops
end
subgraph TR[Training, self-improving]
direction TB
rt[auto-retrain<br/>champion / challenger] --> reg[(Model Registry<br/>cert_phish)]:::hops
rt --> hist[(retrain_history)]:::hops
end
subgraph INF[Inference]
direction TB
app[live feed app] --> topic[[Kafka topic<br/>certs_scored]]:::hops
topic --> app
app --> scored[(cert_scored)]:::hops
ep[KServe endpoint]:::hops
end
ct --> fp
lab -.label.-> fp
ct --> app
fg --> rt
reg --> app
reg --> ep
classDef hops fill:#10b98122,stroke:#34d399,color:#e5e7eb;
classDef ext fill:none,stroke:#6b7280,color:#9ca3af,stroke-dasharray:4 3;
The file-by-file map:
features.py shared leakage-free extractor at issuance (no skew)
collect/stream_ct.py read CT logs directly, yield host + cert metadata
collect/fetch_labels.py abuse blocklists -> labelled hostnames
pipelines/feature_pipeline.py CT window + labels -> online Feature Group (job)
pipelines/train.py Feature View -> model -> registry (job)
pipelines/retrain.py refresh -> challenger -> promote-if-better (scheduled job)
serving/score.py follow CT -> score -> Kafka topic + cert_scored
serving/predictor.py KServe predictor: poke it with a hostname
serving/deploy.py deploy the champion, repoint it on promotion
app/app.py self-contained live feed (produce + consume + UI)
pipelines/retrain.py runs on a schedule. Each run pulls a fresh CT sample and the
current blocklists, appends the new hosts to the feature group, trains a challenger
on the same folds, and promotes it to the served cert_phish only if it beats the
reigning champion by a margin wider than the CV noise. Every run is logged to
retrain_history, promoted or not, so the whole trajectory is auditable. The
guardrails that keep this from fooling itself are in docs/HONESTY.md.
Clone into a Hopsworks project on the /hopsfs/... FUSE mount.
python features.py # self-check the extractor, no cluster needed
python collect/stream_ct.py # pull live certs from a CT log
python pipelines/feature_pipeline.py # CT + labels -> online Feature Group
python pipelines/train.py # model -> registry
python pipelines/retrain.py # one self-improvement step (schedule this)
python serving/deploy.py # KServe endpoint for the championNo input box. The app follows the CT firehose, scores each freshly issued hostname
with the champion model, and renders it as it lands, newest on top, suspicious ones
lit up. It is a tail -f of the world's TLS issuance, with a phishing score on
every line.
The live feed scores inline for throughput. The phishscorer KServe deployment is
the separate request/response surface: poke it with a hostname, get a phishing
probability. It is not in the feed's hot path, and the retrain keeps it pointed at
the current champion.
import hopsworks
dep = hopsworks.login().get_model_serving().get_deployment("phishscorer")
dep.predict(inputs=[{"host": "secure-paypa1-login.top"}, {"host": "www.google.com"}])
# -> [0.99, 0.10]curl -s "$ENDPOINT/v1/models/phishscorer:predict" \
-H "Authorization: ApiKey $HOPSWORKS_API_KEY" -H "Content-Type: application/json" \
-d '{"instances": [{"host": "secure-paypa1-login.top"}]}'