domain-safe

High-performance Node.js/TypeScript library and CLI for offline domain reputation checks using downloadable blacklist sources.

Features

• Zero network at runtime — the library never downloads data; only the CLI does
• Sub-microsecond lookups — flat binary suffix trie with optional Bloom filter
• Lock-free hot reload — atomic index swap when CLI updates the database
• Suffix matching — evil.com blocks sub.evil.com
• IDN / punycode — full normalization support
• ESM + CJS — tree-shakeable, TypeScript-first

Quick Start

pnpm add domain-safe @domain-safe/core
npx domain-safe init
npx domain-safe update
npx domain-safe check example.com

CLI

Command	Description
domain-safe init	Create .domain-safe/ and domain-safe.config.json
domain-safe update	Download sources, build binary index
domain-safe check <domain>	Check if domain is listed
domain-safe stats	Show index statistics
domain-safe doctor	Validate config, index, and sources

Scheduled Updates

Run updates via cron (no built-in daemon):

0 */6 * * * cd /path/to/project && domain-safe update

Runtime API

import { DomainSafe } from "@domain-safe/core";

const safe = await DomainSafe.load({
  dbPath: ".domain-safe/index.bin",
  verifyChecksum: true,           // default: validate CRC32 on boot
  verifyChecksumOnReload: false,  // default: skip CRC on hot reload (faster)
});

const result = safe.lookup("sub.example.com");
// { listed: boolean, domain, matched?, source?, reason? }

safe.on("reload", ({ success, stats, error }) => {
  console.log("Index reloaded", success);
});

await safe.reload();
safe.close();

Configuration

{
  "sources": ["fabriziosalmi", "urlhaus", "openphish"],
  "output": ".domain-safe",
  "format": "binary",
  "compression": true,
  "runtime": {
    "autoReload": true,
    "watch": true
  }
}

Available sources

ID	Description
fabriziosalmi	Aggregated daily domain blacklist (~3M domains)
urlhaus	abuse.ch URLhaus malware hostnames
openphish	OpenPhish community phishing feed (domains extracted from URLs)

Use any combination in sources. All configured feeds are merged and deduplicated at build time.

Architecture

CLI (network)  →  download + parse + normalize  →  index.bin
Runtime (offline)  →  load binary index  →  lookup()

See docs/architecture.md for details.

Packages

Package	Description
@domain-safe/core	Runtime library (zero deps)
@domain-safe/sources	Source adapters (CLI only)
domain-safe	CLI binary
@domain-safe/benchmark	Performance benchmarks
@domain-safe/examples	Express, Hono, Fastify, Worker examples

Benchmarks

Measured with @domain-safe/benchmark on Node.js 20, synthetic domains, Bloom filter enabled.

Lookup (100K domains, warm cache)

Operation	Latency	Notes
lookup (miss)	~892 ns/op	Bloom filter short-circuits most negatives
lookup (hit)	~8 µs/op	Suffix trie traversal + binary search on children
lookup (suffix hit)	~7 µs/op	e.g. sub.evil.com matches listed evil.com

Index build & load

Metric	100K domains	Notes
Index build (CLI)	~15 s	download excluded; parse + dedupe + binary encode
Index size	~2.5 MB	flat suffix trie + Bloom + string pool
Load time (runtime, verify)	~13 ms	read + CRC32 + zero-copy Bloom view
Load time (skip checksum)	~5–8 ms	verifyChecksum: false or hot reload default
Large fixture test	100K domains	covered in CI integration tests

Load optimizations

Runtime load is optimized for large indexes:

Optimization	Effect
Zero-copy Bloom	Bloom filter is a view into index.bin — no duplicate buffer
Lazy metadata	JSON metadata parsed only when stats() or reason is needed
Optional checksum	Skip CRC32 on hot reload (verifyChecksumOnReload: false, default)

// Fastest startup when index is trusted (e.g. CI-built artifact)
const safe = await DomainSafe.load({
  dbPath: ".domain-safe/index.bin",
  verifyChecksum: false,
});

// Production: verify on boot, fast reload thereafter (default behavior)
const prod = await DomainSafe.load({
  dbPath: ".domain-safe/index.bin",
  verifyChecksum: true,
  verifyChecksumOnReload: false,
});

Run locally

pnpm build
pnpm benchmark              # full run: 100K domains, 1M iterations
pnpm benchmark -- --smoke   # CI smoke: 10K domains, regression guard

Smoke mode fails CI if lookup exceeds 500 µs/op or load exceeds 5 s.

Design notes

• Miss path is fastest — Bloom filter rejects domains not in the index without trie traversal
• Hit path scales with label depth — suffix match walks reverse labels (com → evil → …)
• No runtime allocations on the lookup hot path — flat ArrayBuffer + in-place reads
• Millions of domains — index build time grows linearly; load stays under ~100 ms target for ~3M entries (see docs/architecture.md)

Numbers vary by CPU, Node version, and real-world domain distribution. Treat as indicative, not a SLA.

Development

pnpm install
pnpm build
pnpm test
pnpm benchmark -- --smoke

Alternatives

domain-safe is an in-process, offline-first domain blocklist library. Depending on your use case, these alternatives may fit better:

Tool	Type	Best for
Pi-hole	DNS sinkhole	Network-wide blocking for all devices on a LAN
AdGuard Home	DNS sinkhole	Home network filtering with a web UI
dnsink	DNS proxy (Rust)	High-performance DNS-layer blocking with live feeds
agent-dns-firewall	In-process Node.js library	AI agents / apps that need runtime fetch + blocklist refresh
@gorhill/ubo-core	In-process trie engine	uBlock Origin-style filter lists and hostname matching
fabriziosalmi/blacklists	Raw blocklist feeds	Pi-hole, Unbound, BIND — use lists directly without a library
Google Safe Browsing API	Hosted API	Real-time URL/domain reputation with Google infrastructure
VirusTotal API	Hosted API	Multi-vendor threat intelligence and enrichment

When to choose domain-safe

• You need zero network access at runtime (air-gapped, edge, strict security)
• You want microsecond in-memory lookups inside Node.js/Bun/Deno
• You need hot reload without restarting your app
• You prefer a CLI build step + binary index over parsing text on every startup

When to choose something else

• Network-wide protection → Pi-hole, AdGuard Home, or dnsink at the DNS layer
• Runtime auto-download + refresh → agent-dns-firewall
• Full filter-list syntax (EasyList, uBlock rules) → @gorhill/ubo-core
• Real-time cloud reputation + scoring → Safe Browsing, VirusTotal, or similar APIs
• Simple hosts-file blocking → download fabriziosalmi/blacklists and point your resolver at it

License

MIT