chore(telemetry): remove hardcoded PostHog key default

[willwashburn]

Summary

Relaycast half of AgentWorkforce/relay#881 (P0.5). A parallel PR in the relay repo handles the relay-side changes.

Drops the baked-in phc_OAqBdey9... PostHog write key from this repo so that:

1. Forks running tests / dev no longer pollute our production PostHog project.
2. Telemetry silently no-ops when no key is configured (no scary warnings, no HTTP calls, no event buffer).
3. Production builds get the key injected from a GitHub Actions repo variable at deploy / install time.

Why a variable, not a secret: PostHog phc_* project keys are ingestion-only and public by design — same category as a Sentry DSN or Stripe publishable key. The structural problem is fork pollution, and that's already solved by (a) the new no-op-when-missing behavior and (b) the fact that GitHub gates both vars and secrets from fork PRs. Calling it a secret was misleading.

What changed

packages/mcp/src/telemetry.ts

• Removed the DEFAULT_POSTHOG_PUBLIC_KEY constant.
• getPosthogApiKey() now falls back to '' (empty string) when no env / option is provided.
• telemetryEnabled() now takes the resolved API key and returns false when it's empty — same code path as the existing DO_NOT_TRACK / RELAYCAST_TELEMETRY_DISABLED opt-outs, so capture() silently no-ops with no HTTP calls.
• Env precedence is unchanged: RELAYCAST_POSTHOG_API_KEY > POSTHOG_API_KEY > options.posthogApiKey.

packages/mcp/src/__tests__/telemetry.test.ts

• Existing "prefers fetch transport" test now passes an explicit posthogApiKey: 'phc_example' (previously relied on the hardcoded default).
• New test: silently no-ops when no PostHog API key is configured.

site/index.html

• Replaced the hardcoded phc_OAqBdey9... meta value with a __RELAYCAST_POSTHOG_API_KEY__ placeholder that CI substitutes at deploy time.

site/script.js

• Removed the POSTHOG_PUBLIC_KEY constant fallback.
• Added a startsWith('phc_') guard so the unsubstituted __RELAYCAST_POSTHOG_API_KEY__ placeholder (or any other non-phc value) is treated as "no key configured" and the analytics layer returns a no-op capture. Forks deploying their own copy of the static site, and local previews via file://, both get no-op behavior.

.github/workflows/deploy.yml

• New Inject PostHog API key into marketing site step before pages deploy site/. Uses a small Node script to do a literal __RELAYCAST_POSTHOG_API_KEY__ -> value substitution in site/index.html. The source is vars.POSTHOG_PROJECT_KEY (a repository variable, not a secret — see "why" above). If the variable is unset, logs a warning and ships the site with telemetry disabled (no failure).
• The existing deploy-production job already pipes secrets.POSTHOG_API_KEY from a GitHub secret into the Cloudflare Worker via wrangler secret put — left as-is. The Cloudflare-side is still a runtime secret (that's a Cloudflare concept), and the GitHub source for that step is a separate value managed independently from this PR.

Workflows intentionally NOT modified

• publish-npm.yml — the MCP package reads RELAYCAST_POSTHOG_API_KEY from the user's runtime env at install time; nothing is baked into the npm artifact.
• publish-rust.yml, local-binary-release.yml — no PostHog integration in those artifacts.
• ci.yml and the ci job in deploy.yml — left without the key so test events from CI don't pollute the prod project. DO_NOT_TRACK=1 was already set on test runs there.

Action required after merge

Add POSTHOG_PROJECT_KEY as a repository Variable (Settings → Secrets and variables → Actions → Variables tab, NOT Secrets) before the next release. The deploy-pages job injects this value into the marketing site at deploy time. Without it, the site ships cleanly but with telemetry disabled.

The Cloudflare Worker still receives its PostHog value as a runtime secret via wrangler secret put POSTHOG_API_KEY (that's a Cloudflare-side concept, not a GitHub-side one). The existing secrets.POSTHOG_API_KEY GitHub Actions secret feeding that step is unchanged by this PR.

Users running the MCP locally won't have a key set by default and will silently no-op. If we want to opt them in via documentation, that's a follow-up.

Notes / judgment calls

• I made site/script.js reject any meta value that doesn't start with phc_. This handles the __RELAYCAST_POSTHOG_API_KEY__ placeholder cleanly and also defensively rejects garbage values. Felt cleaner than a value === '__RELAYCAST_POSTHOG_API_KEY__' string compare.
• The substitution step in the workflow uses a Node script (rather than sed -i) to avoid GNU vs. BSD sed differences and any regex-special-char surprises in the key. It also validates the placeholder is present and exits non-zero if it isn't.
• No docs/ or marketing-copy mentions of the key needed updating (verified by grep -r phc_OAqBdey9 .).

Test plan

• npm run build — succeeds (turbo, 9 tasks).
• DO_NOT_TRACK=1 npm test — 223/223 MCP tests pass; 437/437 server tests pass; full suite green.
• npm run lint — 0 errors (64 pre-existing warnings unrelated to this change).
• grep -r 'phc_OAqBdey9' returns 0 hits across the repo.
• After merge + variable config: trigger a deploy, confirm <meta name="relaycast-posthog-key"> contains the real key in the deployed site/index.html.
• After merge: run npx @relaycast/mcp locally with no env, confirm no PostHog requests in network logs.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 561c628d-f20d-43d4-ae50-356ece3face6

📥 Commits

Reviewing files that changed from the base of the PR and between c4050ff and 0edc908.

📒 Files selected for processing (3)

• packages/mcp/src/__tests__/telemetry.test.ts
• packages/mcp/src/server.ts
• packages/server/src/durable-objects/mcpSession.ts

🚧 Files skipped from review as they are similar to previous changes (1)

• packages/server/src/durable-objects/mcpSession.ts

---

📝 Walkthrough

Walkthrough

Telemetry is opt-in via a PostHog API key: backend gating and server wiring skip capture when no key is present, the frontend reads a meta placeholder and disables tracking if invalid/missing, and CI can inject the key into the HTML placeholder at deploy time.

Changes

Conditional PostHog Telemetry

Layer / File(s)	Summary
Backend telemetry API key gating and wiring packages/mcp/src/telemetry.ts, packages/mcp/src/server.ts, packages/server/src/durable-objects/mcpSession.ts, packages/mcp/src/__tests__/telemetry.test.ts	telemetryEnabled now accepts the API key and returns false when it's falsy; getPosthogApiKey falls back to empty string; capture checks the resolved key; server options and MCP session thread the posthogApiKey; tests added/updated for key-present and key-absent cases.
Frontend telemetry key initialization from meta tag site/script.js, site/index.html	Removed hardcoded public key; runtime reads relaycast-posthog-key meta tag, validates phc_ prefix, and returns disabled telemetry with a no-op capture when no valid key is present. HTML meta tag now contains __RELAYCAST_POSTHOG_API_KEY__ placeholder.
Deployment secret injection .github/workflows/deploy.yml	Adds a deploy step that reads RELAYCAST_POSTHOG_API_KEY and conditionally replaces __RELAYCAST_POSTHOG_API_KEY__ in site/index.html with the secret using an inline Node script; logs/skips when empty and fails if the placeholder is missing.

Sequence Diagram

sequenceDiagram
  participant GitHub as GitHub Actions
  participant HTML as site/index.html
  participant Frontend as site/script.js
  participant Backend as packages/mcp/src/telemetry.ts

  GitHub->>HTML: inject RELAYCAST_POSTHOG_API_KEY into __RELAYCAST_POSTHOG_API_KEY__
  Frontend->>HTML: read meta tag 'relaycast-posthog-key'
  alt valid key (phc_ prefix)
    Frontend->>Backend: createMcpTelemetry with posthogApiKey
    Backend->>Backend: telemetryEnabled checks key → true
    Backend->>Backend: capture sends events
  else no key or invalid
    Frontend->>Backend: createMcpTelemetry without key
    Backend->>Backend: telemetryEnabled checks key → false
    Backend->>Backend: capture is no-op
  end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 I nudged a placeholder, tiny and meek,

CI brings a key when the nights are sleek.
No key — we nap, no pings on the wire,
With key — we whisper metrics like choir.
Hoppity hops, the telemetry's sweet!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'chore(telemetry): remove hardcoded PostHog key default' accurately and concisely summarizes the main change in the changeset—removing the baked-in PostHog write key.
Description check	✅ Passed	The description is comprehensive and directly related to the changeset, covering the rationale, technical changes, required post-merge actions, and test plan for removing the hardcoded PostHog key.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch chore/telemetry-key-from-env

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[cubic-dev-ai]

No issues found across 5 files

_{Re-trigger cubic}

[devin-ai-integration]

Devin Review found 1 potential issue.

View 4 additional findings in Devin Review.

[devin-ai-integration]

🔴 MCP telemetry silently disabled in Cloudflare Worker production after removing hardcoded key

The removal of DEFAULT_POSTHOG_PUBLIC_KEY breaks MCP telemetry in the Cloudflare Worker context (McpSessionDO). The getPosthogApiKey function falls through process.env.RELAYCAST_POSTHOG_API_KEY → process.env.POSTHOG_API_KEY → options.posthogApiKey → ''. In Cloudflare Workers, secrets set via wrangler secret put are available through the env bindings parameter (as used by the server package at packages/server/src/lib/telemetry.ts:36), not through process.env. Since McpSessionDO (packages/server/src/durable-objects/mcpSession.ts:51-57) calls createRelayMcpServer without passing posthogApiKey, and the MCP server's createRelayMcpServer (packages/mcp/src/server.ts:83-87) doesn't pass it to createMcpTelemetry either, the API key resolves to ''. With the new !apiKey check at packages/mcp/src/telemetry.ts:47, telemetry is disabled. This silently drops all MCP-specific events (relaycast_mcp_server_started, relaycast_mcp_session_authenticated, etc.) in production. Previously, the hardcoded DEFAULT_POSTHOG_PUBLIC_KEY ensured these events were always captured.

Prompt for agents

The core issue is that the MCP package's telemetry uses process.env to read the PostHog API key, but in the Cloudflare Worker runtime (McpSessionDO), secrets are only available through the env bindings parameter, not process.env. The McpSessionDO has access to this.env.POSTHOG_API_KEY, but it never passes it through to createRelayMcpServer or createMcpTelemetry.

To fix this, you need to thread the PostHog API key from the Worker env bindings into the MCP telemetry. One approach:

1. Add a posthogApiKey field to McpServerOptions in packages/mcp/src/server.ts
2. Pass it through when creating telemetry at server.ts:83-87
3. In McpSessionDO.ensureInitialized() (packages/server/src/durable-objects/mcpSession.ts:51), pass posthogApiKey: this.env.POSTHOG_API_KEY to createRelayMcpServer

This mirrors how the server package's own telemetry (packages/server/src/lib/telemetry.ts:36) explicitly reads env.POSTHOG_API_KEY from the Cloudflare bindings rather than relying on process.env.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[github-actions]

Preview deployed!

Environment	URL
API	https://pr131-api.relaycast.dev
Health	https://pr131-api.relaycast.dev/health
Observer	https://pr131-observer.relaycast.dev/observer

This preview shares the staging database and will be cleaned up when the PR is merged or closed.

Run E2E tests

npm run e2e -- https://pr131-api.relaycast.dev --ci

Open observer dashboard

https://pr131-observer.relaycast.dev/observer

[willwashburn]

@copilot please fix issues raised by comments in this PR

[Copilot]

@copilot please fix issues raised by comments in this PR

Fixed in c4050ff. The review identified that in Cloudflare Workers, wrangler secret put values are only accessible via the env bindings parameter — not process.env. Without the hardcoded default, MCP telemetry was silently disabled in production.

The fix threads the key through three layers:

1. McpSessionDO.ensureInitialized() now passes posthogApiKey: this.env.POSTHOG_API_KEY to createRelayMcpServer
2. McpServerOptions gains a new posthogApiKey? field
3. createRelayMcpServer forwards it to createMcpTelemetry via the existing options.posthogApiKey path in McpTelemetryOptions

This mirrors how the server package's own telemetry already reads env.POSTHOG_API_KEY directly from Cloudflare bindings.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• api.relaycast.dev
  • Triggering command: /usr/local/bin/node /usr/local/bin/node --experimental-import-meta-resolve --require /home/REDACTED/work/relaycast/relaycast/node_modules/vitest/suppress-warnings.cjs --conditions node --conditions development /home/REDACTED/work/relaycast/relaycast/node_modules/vitest/dist/workers/forks.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[willwashburn]

Thanks @devin-ai-integration[bot] — the regression you flagged is real and is now fixed.

Resolution

• c4050ff (Copilot) threaded posthogApiKey: this.env.POSTHOG_API_KEY from the CF Worker env bindings into createRelayMcpServer -> createMcpTelemetry, which solves the core issue.
• 0edc908 (just pushed) rounds it out by also forwarding POSTHOG_HOST through the same path (so a custom host like the EU region works in Workers too) and adds an explicit regression test that mirrors the Cloudflare scenario: process.env.* unset, options.posthogApiKey provided, capture HTTP call fires with the expected api_key.

Verification:

• DO_NOT_TRACK=1 npx turbo test --filter=@relaycast/mcp --filter=@relaycast/server — green (223 MCP + 437 server tests pass, including the new uses options.posthogApiKey when process.env is unset (Cloudflare Worker path) test).
• npx turbo build and npx turbo lint — clean (0 errors).

Heads-up on the other findings

Your comment mentions "4 additional findings in Devin Review" living on the external dashboard at app.devin.ai/review/.... We don't have access to that surface via the GitHub API. If any of those findings are also legitimate, please re-surface them as GitHub PR comments so we can triage and act on them on this branch before merge.