feat(identity): database overlay for federation identity (#326 Phase 1)

CHANGELOG.md

@@ -2,6 +2,9 @@
 
 ## Unreleased
 
+### Added
+- **Documented what actually happens when you change your domain** (#326) — `docs/fediverse-setup.md` gains a **Changing Your Domain** section. ActivityPub has no rename: your identity is a URL, and every server that has seen you stores it. The network's answer is an alias on the new account plus a `Move` from the old one — which only works while **the old domain is still serving**, because remote servers verify the move by fetching both ends. Move first and lose the domain later and your followers can still be brought across; lose the domain first and they can't, by anyone, ever. FediHome doesn't implement that handshake yet, so for now the guidance is to treat your domain as permanent, and the recommended shape of a move (new instance alongside the old, then decommission slowly) is written down.
+
 ### Changed
 - **Setting up on an address the Fediverse can't reach is now a deliberate choice** (#326) — the wizard used to let you finish on `localhost` or a private-network address with nothing but a warning. That bakes an identity nobody can follow into your ActivityPub actor, and because posts store their full URL, into everything you publish before you move. It's still allowed — testing locally is a perfectly reasonable thing to do — but you now have to tick a box confirming that's what you're doing. Setting up on a real domain is unchanged.
 

docs/fediverse-setup.md

@@ -23,6 +23,67 @@ This is set by two environment variables in `.env.local`:
 
 Together, these form `@sam@samcorner.com`. This is what people type into Mastodon's search bar to find and follow you.
 
+## Changing Your Domain
+
+**Choose your domain before you federate, and treat it as permanent.**
+
+In ActivityPub there is no rename. Your identity is your **actor id** — a URL
+built from `SITE_URL`, like `https://samcorner.com/ap/actor`. Every remote server
+that has ever seen you stores *that URL*, along with the public key it fetched
+from it. Change the domain and, as far as the network is concerned, the old
+account simply stopped existing and an unrelated new one appeared. Your followers
+stay attached to an address that no longer answers.
+
+### How the network handles a move
+
+Mastodon (and micro.blog, and others) solve this with a two-sided handshake:
+
+1. On the **new** account you add an **alias** — `alsoKnownAs` — pointing at the
+   old actor URL.
+2. From the **old** account you publish a **`Move`** activity and set `movedTo`
+   on the old actor.
+3. Each remote server that receives the `Move` **fetches the old actor to verify
+   it, then fetches the new one and checks that its `alsoKnownAs` names the old
+   address.** Only if both agree does it move the follow across.
+
+Step 3 is the part that catches people out: **the old domain has to still be
+serving** while followers migrate. A `Move` published from a domain you no longer
+control cannot be verified, so nothing moves. Lose the domain first and your
+followers are simply gone — you keep the list of accounts *you* follow, because
+that lives in your own database, but the people following *you* cannot be
+brought across by any later action.
+
+Even done correctly, migration is best-effort: servers that are down during the
+move, or that don't implement `Move`, will keep pointing at the old address.
+
+### What this means for FediHome today
+
+FediHome does **not** implement `alsoKnownAs`, `Move`, or `movedTo` yet — in
+either direction (tracked in #326). So today there is no supported way to carry
+followers to a new domain, which makes the advice above a hard rule rather than a
+recommendation.
+
+If you must move, the shape that works with the grain of the protocol is:
+
+1. Stand up a **new instance on the new domain**, running alongside the old one.
+2. Migrate your content to it (a database restore keeps your posts and your
+   following list).
+3. Publish the move from the old instance, and **leave the old server running**
+   for a good while — weeks, not hours — so every remote server gets a chance to
+   verify and follow the redirect.
+4. Only then decommission the old domain.
+
+Note that **post URLs are stored absolutely**: a post published at
+`https://old.example/post/hello` keeps that URL in the database. A restore onto a
+new domain carries those old URLs with it, so links and federated copies of older
+posts still point at the old host. That is another reason to keep the old domain
+alive rather than cutting it over.
+
+And it's why setting up against `localhost` or a private address matters: the
+setup wizard makes you confirm it explicitly, because an identity nobody can
+reach gets written into your actor *and* into every post you publish before you
+move.
+
 ## DNS Requirements
 
 For federation to work, your domain must:

src/instrumentation.ts

@@ -9,6 +9,14 @@
  */
 export async function register() {
   if (process.env.NEXT_RUNTIME === "nodejs") {
+    // Federation identity first, and awaited (#326). getIdentity() is synchronous,
+    // so the database overrides have to be in place BEFORE anything can serve a
+    // request — a request answered mid-load would sign with the environment's
+    // identity instead of the configured one, which is precisely the silent
+    // actor-id mismatch that breaks federation with nothing in the logs.
+    const { loadIdentity } = await import("@/lib/identity-store");
+    await loadIdentity();
+
     const { startScheduler } = await import("@/lib/scheduler");
     startScheduler();
   }

src/lib/__tests__/identity-store.test.ts

@@ -0,0 +1,131 @@
+import { describe, it, expect, vi, beforeEach, afterAll } from "vitest";
+
+/**
+ * The database overlay for federation identity (#326 Phase 1).
+ *
+ * Two things matter here. First, precedence: a saved identity must win over the
+ * environment, or the overlay is pointless — but a MISSING row must fall through
+ * to the environment rather than to a default, or an instance would silently
+ * start federating as `@me@localhost:3000`.
+ *
+ * Second, it must never throw. This runs during boot, before anything can serve
+ * a request; a database that is down or mid-migration has to leave the instance
+ * on its environment identity rather than refusing to start.
+ */
+
+const { findMany } = vi.hoisted(() => ({ findMany: vi.fn() }));
+vi.mock("@/lib/db", () => ({ prisma: { siteSetting: { findMany } } }));
+
+import { loadIdentity, refreshIdentity, IDENTITY_KEYS } from "@/lib/identity-store";
+import { getIdentity, clearIdentityOverrides } from "@/lib/identity";
+
+const OLD = {
+  SITE_URL: process.env.SITE_URL,
+  FEDI_HANDLE: process.env.FEDI_HANDLE,
+  FEDI_DOMAIN: process.env.FEDI_DOMAIN,
+};
+
+const rows = (o: Record<string, string>) =>
+  Object.entries(o).map(([key, value]) => ({ key, value }));
+
+beforeEach(() => {
+  vi.clearAllMocks();
+  clearIdentityOverrides();
+  process.env.SITE_URL = "https://from-env.example";
+  process.env.FEDI_HANDLE = "envhandle";
+  delete process.env.FEDI_DOMAIN;
+  findMany.mockResolvedValue([]);
+});
+
+afterAll(() => {
+  clearIdentityOverrides();
+  for (const [k, v] of Object.entries(OLD)) {
+    if (v === undefined) delete process.env[k];
+    else process.env[k] = v;
+  }
+});
+
+describe("loadIdentity — precedence", () => {
+  it("queries exactly the identity.* keys", async () => {
+    await loadIdentity();
+    expect(findMany).toHaveBeenCalledWith({ where: { key: { in: IDENTITY_KEYS } } });
+    expect(IDENTITY_KEYS).toEqual([
+      "identity.siteUrl",
+      "identity.fediHandle",
+      "identity.fediDomain",
+    ]);
+  });
+
+  it("a saved identity wins over the environment", async () => {
+    findMany.mockResolvedValue(
+      rows({ "identity.siteUrl": "https://from-db.example", "identity.fediHandle": "dbhandle" }),
+    );
+    await loadIdentity();
+
+    const id = getIdentity();
+    expect(id.siteUrl).toBe("https://from-db.example");
+    expect(id.fediHandle).toBe("dbhandle");
+    expect(id.actorId).toBe("https://from-db.example/ap/actor");
+    // Derived values must follow the override, not straddle both sources.
+    expect(id.keyId).toBe("https://from-db.example/ap/actor#main-key");
+    expect(id.webfingerSubject).toBe("acct:dbhandle@from-db.example");
+  });
+
+  it("a field with no row falls through to the environment, not to a default", async () => {
+    // The dangerous alternative: an instance quietly federating as @me@localhost.
+    findMany.mockResolvedValue(rows({ "identity.siteUrl": "https://from-db.example" }));
+    await loadIdentity();
+
+    expect(getIdentity().fediHandle).toBe("envhandle");
+  });
+
+  it("no rows at all leaves the environment identity untouched", async () => {
+    await loadIdentity();
+    const id = getIdentity();
+    expect(id.siteUrl).toBe("https://from-env.example");
+    expect(id.fediHandle).toBe("envhandle");
+  });
+
+  it("normalisation still applies to a value that came from the database", async () => {
+    findMany.mockResolvedValue(rows({ "identity.siteUrl": "https://from-db.example/" }));
+    await loadIdentity();
+    expect(getIdentity().actorId).toBe("https://from-db.example/ap/actor");
+  });
+});
+
+describe("loadIdentity — rejects junk rows", () => {
+  it("ignores blank and whitespace-bearing values rather than building a broken actor id", async () => {
+    findMany.mockResolvedValue(
+      rows({ "identity.siteUrl": "   ", "identity.fediHandle": "has space" }),
+    );
+    await loadIdentity();
+
+    const id = getIdentity();
+    expect(id.siteUrl).toBe("https://from-env.example");
+    expect(id.fediHandle).toBe("envhandle");
+  });
+
+  it("ignores an unknown identity.* key", async () => {
+    findMany.mockResolvedValue(rows({ "identity.nonsense": "x" }));
+    await expect(loadIdentity()).resolves.toBeUndefined();
+    expect(getIdentity().siteUrl).toBe("https://from-env.example");
+  });
+});
+
+describe("loadIdentity — must not break the boot", () => {
+  it("falls back to the environment when the database is unavailable", async () => {
+    findMany.mockRejectedValue(new Error("ECONNREFUSED"));
+    await expect(loadIdentity()).resolves.toBeUndefined();
+    expect(getIdentity().siteUrl).toBe("https://from-env.example");
+  });
+
+  it("a failed reload drops a previously-loaded override rather than serving it blind", async () => {
+    findMany.mockResolvedValue(rows({ "identity.siteUrl": "https://from-db.example" }));
+    await loadIdentity();
+    expect(getIdentity().siteUrl).toBe("https://from-db.example");
+
+    findMany.mockRejectedValue(new Error("db went away"));
+    await refreshIdentity();
+    expect(getIdentity().siteUrl).toBe("https://from-env.example");
+  });
+});

src/lib/identity-store.ts

@@ -0,0 +1,85 @@
+import { prisma } from "./db";
+import { applyIdentityOverrides, clearIdentityOverrides, getIdentity } from "./identity";
+
+/**
+ * The database half of federation identity (#326 Phase 1).
+ *
+ * Kept separate from `identity.ts` on purpose: that module is imported by
+ * `site.config.ts`, which reaches client bundles, so importing `prisma` there
+ * would pull the database client into the browser. This module is server-only
+ * and *pushes* the loaded values into the accessor, which stays synchronous —
+ * that's what lets ~60 call sites, several of them in sync helpers, keep working
+ * without becoming async.
+ *
+ * **Loaded once at boot** (`src/instrumentation.ts`), not per request and not on
+ * a TTL. Identity effectively never changes, a stale value here is far worse
+ * than a stale setting elsewhere, and `getIdentity()` is synchronous so it
+ * cannot refresh itself anyway. Anything that writes these rows must call
+ * `refreshIdentity()`.
+ *
+ * **Read-only for now.** No UI writes `identity.*`, so every instance still
+ * resolves from the environment. The write path, the admin UI, and the
+ * change-domain migration are later phases — and the migration is the hard part:
+ * ActivityPub has no rename, so moving domains means `alsoKnownAs` + a `Move`
+ * activity served from the OLD instance while it is still reachable.
+ */
+
+const KEY_PREFIX = "identity.";
+const FIELDS = ["siteUrl", "fediHandle", "fediDomain"] as const;
+type IdentityField = (typeof FIELDS)[number];
+
+export const IDENTITY_KEYS = FIELDS.map((f) => `${KEY_PREFIX}${f}`);
+
+/** Reject junk so a bad row can't produce a malformed actor id. */
+function clean(value: unknown): string | undefined {
+  if (typeof value !== "string") return undefined;
+  const v = value.trim();
+  if (!v || v.length > 300 || /[\s\r\n]/.test(v)) return undefined;
+  return v;
+}
+
+/**
+ * Load the `identity.*` overrides from the database into the accessor.
+ *
+ * Never throws: a database that is down or mid-migration must not stop the
+ * server booting, and falling back to the environment is the safe direction —
+ * it's what every instance uses today.
+ */
+export async function loadIdentity(): Promise<void> {
+  try {
+    const rows = await prisma.siteSetting.findMany({ where: { key: { in: IDENTITY_KEYS } } });
+    const found: Partial<Record<IdentityField, string>> = {};
+    for (const row of rows) {
+      const field = row.key.slice(KEY_PREFIX.length) as IdentityField;
+      if (!FIELDS.includes(field)) continue;
+      const value = clean(row.value);
+      if (value) found[field] = value;
+    }
+    applyIdentityOverrides(found);
+
+    if (Object.keys(found).length > 0) {
+      // Loud on purpose: if this ever prints unexpectedly, the instance is
+      // federating under an identity that is NOT what `.env.local` says, and
+      // that is the first thing you'd want to know when debugging it.
+      console.warn(
+        `[FediHome] Federation identity overridden from the database: ${getIdentity().fediAddress} (${getIdentity().actorId})`,
+      );
+    }
+  } catch {
+    clearIdentityOverrides(); // DB unavailable → environment only
+  }
+}
+
+/**
+ * Re-read the overrides after a write.
+ *
+ * ⚠️ Process-local, like the load itself. Under a multi-process deployment
+ * (pm2 cluster) only the worker that handled the write is refreshed; the others
+ * keep serving the old identity until they restart. Whatever ships the write
+ * path has to account for that — a partial rollout of a *federation identity*
+ * across workers is exactly the silent-mismatch failure this module exists to
+ * prevent.
+ */
+export async function refreshIdentity(): Promise<void> {
+  await loadIdentity();
+}

src/lib/identity.ts

@@ -7,8 +7,12 @@
  * 12 more. A module-level `const siteUrl = process.env.SITE_URL` is evaluated
  * once at *import*, so it can never see a value resolved at runtime — which
  * makes a DB-backed identity impossible until every consumer goes through one
- * accessor. This is that accessor. It is still env-backed and (deliberately)
- * changes nothing about how values are resolved; the overlay comes later.
+ * accessor. This is that accessor.
+ *
+ * Resolution order is **runtime override → environment → built-in default**.
+ * The overrides are loaded from the database at boot by `identity-store.ts`
+ * (Phase 1); nothing writes those rows yet, so in practice this still resolves
+ * from the environment exactly as before.
  *
  * **Why the agreement matters.** The actor id, the WebFinger `subject`/`href`
  * and the HTTP-signature `keyId` must all describe the same identity. If they
@@ -28,6 +32,40 @@
 
 const DEFAULT_SITE_URL = "http://localhost:3000";
 
+/**
+ * Runtime overrides, layered over the environment (#326 Phase 1).
+ *
+ * This module deliberately imports **nothing** — `site.config.ts` imports it,
+ * and that is pulled into client bundles, so a `prisma` import here would drag
+ * the database client into the browser. The DB read therefore lives in the
+ * server-only `identity-store.ts`, which *pushes* values in through
+ * `applyIdentityOverrides`. That keeps `getIdentity()` synchronous, which is
+ * what lets ~60 call sites — several of them in sync helpers — keep working.
+ *
+ * Empty until something loads it. On the client it stays empty forever, exactly
+ * as `process.env.SITE_URL` is already absent there; client code must take
+ * identity from props or the runtime site config, never from here.
+ */
+type IdentityOverrides = Partial<Record<"siteUrl" | "fediHandle" | "fediDomain", string>>;
+let overrides: IdentityOverrides = {};
+
+/**
+ * Replace the runtime overrides. Server-side only — see `identity-store.ts`.
+ *
+ * ⚠️ Process-local. Under a multi-process deployment (pm2 cluster) a write in
+ * one worker is invisible to the others until they reload, so whatever
+ * eventually writes these rows must either refresh every worker or require a
+ * restart. Same constraint as `webpush.setVapidDetails` in push-config.ts.
+ */
+export function applyIdentityOverrides(next: IdentityOverrides): void {
+  overrides = { ...next };
+}
+
+/** Drop the overrides, falling back to the environment. */
+export function clearIdentityOverrides(): void {
+  overrides = {};
+}
+
 export interface Identity {
   /** Public origin, no trailing slash — e.g. `https://example.com`. */
   siteUrl: string;
@@ -56,15 +94,16 @@ function normalizeOrigin(raw: string): string {
 }
 
 export function getIdentity(): Identity {
-  const siteUrl = normalizeOrigin(process.env.SITE_URL || DEFAULT_SITE_URL);
-  const fediHandle = process.env.FEDI_HANDLE || "me";
+  // Precedence: runtime override -> environment -> built-in default.
+  const siteUrl = normalizeOrigin(overrides.siteUrl || process.env.SITE_URL || DEFAULT_SITE_URL);
+  const fediHandle = overrides.fediHandle || process.env.FEDI_HANDLE || "me";
   // Derive the domain from the site URL when it isn't set explicitly, matching
   // what site.config.ts has always done. WebFinger used to fall back to the
   // literal "localhost" instead, so an instance that set SITE_URL but not
   // FEDI_DOMAIN advertised @me@example.com everywhere while WebFinger answered
   // only to acct:me@localhost — i.e. 404 to every remote lookup, undiscoverable,
   // with a perfectly healthy-looking site. One derivation, no disagreement.
-  let fediDomain = process.env.FEDI_DOMAIN;
+  let fediDomain = overrides.fediDomain || process.env.FEDI_DOMAIN;
   if (!fediDomain) {
     try {
       fediDomain = new URL(siteUrl).host;
@@ -100,6 +139,6 @@ export function getSiteUrl(): string {
  * merely *reads* the identity wants `getSiteUrl()` instead.
  */
 export function getConfiguredSiteUrl(): string | undefined {
-  const raw = process.env.SITE_URL?.trim();
+  const raw = (overrides.siteUrl || process.env.SITE_URL)?.trim();
   return raw ? normalizeOrigin(raw) : undefined;
 }