diff --git a/notebooks/bill-shock-tracker.md b/notebooks/bill-shock-tracker.md new file mode 100644 index 0000000..1b0cc05 --- /dev/null +++ b/notebooks/bill-shock-tracker.md @@ -0,0 +1,126 @@ +# Bill Shock Wall — Outreach Tracker + +Curated real-world "surprise cloud / AI bill" incidents used in the **Why This Exists** section +of the marketing site (`site/index.html`), plus a reply/outreach log. + +**Goal:** social proof of the *problem* (not testimonials for us), and a respectful place to share +the free + open-source kill switch with people who just got burned. + +**Outreach account:** [@KillSwitchCloud](https://x.com/KillSwitchCloud) (X) — post replies from here. +**Story inbox:** hello@kill-switch.net → Proton (CF Email Routing). +**Site "Tell us" link:** pre-filled tweet → `https://x.com/intent/post?text=Hey%20%40KillSwitchCloud...` + +**Reply etiquette (read before posting):** +- Help first, pitch second. Lead with empathy + a concrete tip; mention the tool once, briefly. +- One reply per post. Never copy-paste the same text across threads (spam/astroturfing risk). +- Disclose the affiliation ("I work on…"). Don't reply to posts that are already resolved/old unless it adds value. +- You post from your own accounts. Mark status here so we never double-reply. + +Legend — **Status:** `todo` · `drafted` · `posted` · `skip` (too old / resolved / off-topic) + +--- + +## Embeddable social posts (on the site now) + +| # | Source | Person | $ | Post URL | Embed | Reply status | +|---|--------|--------|---|----------|-------|--------------| +| 1 | X | Jingna Zhang (@zemotion) — Cara/Vercel | $96,280 | https://x.com/zemotion/status/1798558292681343039 | ✅ live | todo | +| 2 | X | Jason Lemkin (@jasonlk) — Replit DB wipe | data loss | https://x.com/jasonlk/status/1946069562723897802 | ✅ live | skip* | +| 3 | Reddit r/webdev | u/laubonghaudoi — Netlify | $104,500 | https://www.reddit.com/r/webdev/comments/1b14bty/netlify_just_sent_me_a_104k_bill_for_a_simple/ | ✅ live | skip* | + +\* #2 and #3 are old/resolved and high-profile; replying now reads as opportunistic. Use as wall proof only. + +--- + +## Cited cards (article/news — on the site as linked story-cards) + +| # | Incident | $ | Source URL | Reply-able? | +|---|----------|---|------------|-------------| +| 4 | AI agent retried failing call 11,000× overnight | $50,000 | https://dev.to/lfariaus/building-irl-from-a-50k-aws-horror-story-to-human-centered-ai-governance-1jdg | author blog — maybe | +| 5 | Two agents in recursive loop | $47,000 | https://dev.to/utibe_okodi_339fb47a13ef5/the-ai-agent-that-cost-47000-while-everyone-thought-it-was-working-1lg6 | DEV comment | +| 6 | Claude Code left running overnight | $6,000 | https://www.makeuseof.com/someone-left-claude-code-running-overnight-and-it-cost-6000/ | no (news) | +| 7 | Replit agent ran unattended | $607 | https://blog.vibecoder.me/607-replit-bill-avoiding-runaway-ai-costs | blog | +| 8 | Recursive tool-call loop, 14k requests | $437 | https://earezki.com/ai-news/2026-04-29-i-let-my-ai-agent-run-overnight-it-cost-437/ | author blog | +| 9 | AWS Bedrock + Claude Opus despite cost cap | $30,000 | https://www.theregister.com/ai-ml/2026/05/18/surprise-ai-bills-leave-aws-and-google-cloud-users-aghast/ | no (news) | +| 10 | AU dev, $10k despite spending limits | $10,000 | https://www.theregister.com/ai-ml/2026/05/18/surprise-ai-bills-leave-aws-and-google-cloud-users-aghast/ | no (news) | +| 11 | Firebase 2-yr-old mistake → AI bill overnight | €3,167 | https://medium.com/flutter-community/how-a-two-year-old-firebase-mistake-led-to-a-3-167-ai-bill-overnight-89adfab1dad3 | Medium | +| 12 | Uber burned full-year AI budget in 4 months (Claude Code) | enterprise | https://www.theregister.com/ai-ml/2026/05/18/surprise-ai-bills-leave-aws-and-google-cloud-users-aghast/ | no (news) | + +--- + +## Reply drafts + +### #1 — Jingna Zhang / Cara (X) — `todo` +> Gut-wrenching to read — going viral shouldn't mean a $96K surprise. The core problem is that none +> of these platforms ship a real spending *cap*, just alerts after the fact. We built a free + open-source +> kill switch for exactly this: it watches spend across Vercel/AWS/GCP/CF and auto-pauses the runaway +> resource before the bill lands. No affiliation pressure — it's OSS, self-host it: kill-switch.net. +> (I work on it; happy to help you wire it up.) + +### #5 — $47K recursive-loop story (DEV comment) — `todo` +> The "everyone thought it was working" part is the scary bit — agentic loops burn quietly. We made a +> free OSS tool (`agent-guard`) that meters token spend per session + per rolling day and hard-stops the +> agent at a cap you set, plus cloud-side kill switches for the infra. Disclosure: I work on it — +> kill-switch.net. Hope it saves the next person the $47K. + +### #8 — $437 overnight tool-call loop (author blog) — `todo` +> "Agentic brakes" is exactly the right frame. We shipped a free, open-source version: a Claude Code hook +> + token-metering proxy that caps per-session and daily LLM spend, plus auto-kill on the cloud side. +> (I work on it, it's OSS: kill-switch.net.) Your 14k-redundant-requests breakdown is a great case study — +> mind if we link it on our incidents wall? + +> _Reuse note:_ each draft is intentionally different. Tailor the first line to what THEY said before posting. + +--- + +## To find next round (fresh, reply-able posts) +Web search surfaced mostly news write-ups; the live, recent, reply-able threads need on-platform search: +- [ ] r/aws, r/SaaS, r/webdev, r/selfhosted — sort by New, query "bill" / "overnight" / "AI agent" +- [ ] r/Cursor, r/ClaudeAI, r/ChatGPTCoding — "credits gone", "token usage", "API bill" +- [ ] X search: `vercel bill`, `aws bill overnight`, `claude code $`, `cursor credits` (filter: latest) +- [ ] HN: Algolia search "surprise bill" past month +- [ ] Add a screenshot/permalink the moment you find one — posts get deleted. + +## Fresh reply targets — research pass (May 2026) +HN items verified via Algolia API (real IDs/authors/dates); Cursor forum threads fetched & confirmed. +Reddit/X permalinks could NOT be verified in this pass (reddit blocked to fetch, X needs login) — see gaps below. +Reply from [@KillSwitchCloud](https://x.com/KillSwitchCloud). Lead with help; disclose affiliation; one reply each. + +| Pri | platform | url | poster | date | $ | hook | status | +|-----|----------|-----|--------|------|---|------|--------| +| ⭐ | HN | https://news.ycombinator.com/item?id=47933355 | Zephyr0x | 2026-04-28 | $37,901 | AWS Bedrock prompt-cache miss, "allowed to fail silently," wants hard caps | todo | +| ⭐ | HN | https://news.ycombinator.com/item?id=48120294 | serial_dev | 2026-05-13 | €3,167 | Firebase mistake → overnight AI bill (Flutter); low comments | todo | +| ⭐ | HN | https://news.ycombinator.com/item?id=47943738 | — | 2026-04-29 | $67k | Google API change → $67k Gemini bill in 19h; 0 comments | todo | +| ⭐ | HN | https://news.ycombinator.com/item?id=48200057 | shanirshad | 2026-05-19 | fear | Show HN PrismoDev — worried about Claude Code surprise bills | todo | +| ◦ | HN | https://news.ycombinator.com/item?id=48176365 | — | 2026-05-18 | $10k+ | "Surprise AI bills leave AWS & GCP users aghast" (Register) thread | todo | +| ◦ | HN | https://news.ycombinator.com/item?id=47820834 | — | 2026-04-19 | fear | "Budget alerts come to Cloudflare" — note: alerts ≠ auto-pause | todo | +| ◦ | HN | https://news.ycombinator.com/item?id=47704305 | — | 2026-04-09 | fear | Show HN BillSpike — adjacent founder, collab angle | todo | +| ◦ | HN | https://news.ycombinator.com/item?id=46966879 | hardwellvibe | 2026-02-10 | ~$42+ | Cursor silent pay-per-token overage complaint | todo | +| ◦ | HN | https://news.ycombinator.com/item?id=46544838 | throwawayround | 2026-01-08 | $500+ | Cancelled Cursor Ultra — $60→$500 via hidden cache billing | todo | +| ◦ | HN (viral) | https://news.ycombinator.com/item?id=47791871 | zanbezi | 2026-04-16 | €54k | Unrestricted Firebase key → Gemini; ~297 comments (saturated) | todo | +| ◦ | Cursor | https://forum.cursor.com/t/it-is-unacceptable-that-in-1-day-monthly-credits-are-gone/149522 | Maxim | 2026-01-21 | $20 plan/1 day | Credits gone in a day on Agent+Opus | todo | +| ◦ | Cursor | https://forum.cursor.com/t/running-out-of-ultra-plan-credits-before-renewal/150012 | huynh_ductrung | 2026-01-27 | Ultra/20d | Burned Ultra before renewal, wants caps | todo | +| ⚠ | Reddit | r/ClaudeAI `1t11mmy` — **UNVERIFIED permalink** (from MakeUseOf article) | unknown | ~May 2026 | ~$6,000 | `/loop` ran 46× overnight on Opus 4.7 → $6k | verify-first | + +**Named victims w/o linkable post (track down handles manually):** Rod Danan ($10,138 Gemini in ~30 min) and Isuru Fonseka (~AUD $17k despite believed $250 cap) — from The Register (2026-05-13 / 05-18). + +**Gaps to close next round (needs logged-in access):** +- [ ] Reddit: r/aws, r/Cursor, r/Firebase, r/Supabase, r/ClaudeAI — use Reddit's own search or API (fetch was blocked here). +- [ ] X/Twitter: search behind login for recent first-person bill posts. +- [ ] Richest fresh veins: leaked Google Maps→Gemini API-key bills (Feb–May 2026) and AI-coding-agent overruns (Claude Code loops, Cursor on-demand, Bedrock cache misses). + +## Tweet carousel — verified embeddable tweets (syndication-CDN confirmed) +Live in the homepage "Straight From the Source" carousel (6): @KillSwitchCloud (pinned), @mikeumus, +@zemotion (Cara $96k), @jasonlk (Replit), @tamarajtran (Firebase ~$70k), @GergelyOrosz (AWS, wants hard caps). + +Verified alternates to swap in (all status IDs confirmed via cdn.syndication.twimg.com): +- @rtwlz/2020957597810254052 — Jmail/Vercel ~$46–50k (the creator) +- @MatthewBerman/2043851194301592022 — "this @vercel bill came as an absolute surprise" + screenshot +- @radjathaher/2021021709630148667 — "$46K for serving static content is demonic" (Jmail reaction) +- @zak123/2021035298667495465 — "never use vercel… you owe absurd amounts" (reaction) +- @theburningmonk/1798703655908192570 — Yan Cui commentary on Vercel surprises +Couldn't verify an X status ID (excluded): $82k stolen-Gemini-key (origin Reddit), Cursor "$71/day" (forum). + +## Meta (our own dogfood story — consider posting!) +While building this very wall, our own `agent-guard` kill switch tripped its $300 daily hard cap and +halted the research session. On-brand proof the thing works. Candidate for a "we killed our own agent" post. diff --git a/notebooks/paid-service-audit.md b/notebooks/paid-service-audit.md new file mode 100644 index 0000000..4d051e0 --- /dev/null +++ b/notebooks/paid-service-audit.md @@ -0,0 +1,76 @@ +# Paid-Service Audit — Kill Switch (end-to-end) + +Date: 2026-05-29. Scope: signup → subscribe → payment → provisioning → gated access, across +`packages/api` + `packages/web`. Read-only audit (3 parallel passes: billing, entitlements, signup/UX). + +## Verdict +The paid funnel is **fundamentally sound**: JIT account creation works, Stripe webhook signature +verification + raw-body handling are correct, tier is derived from the **verified price ID** (not +client metadata) and read from a trusted DB source, and most limits are enforced **server-side** +(so `ks_live_` API keys are gated identically to the UI). **Not yet "100%"** — 2 billing BLOCKERs and +one security HIGH should be fixed before charging real cards at scale. + +## Flow map +1. **Signup** — Clerk hosted signup → first authed call hits `resolveOrg`, which upserts a personal + `GuardianAccount` (tier `free`) + `UserProfile` (`middleware/auth.ts:214-251`). No orphaned-user gap. +2. **Onboarding** — 6-step wizard; a free user can reach a working monitored account (1 account, 6h checks). +3. **Billing entry** — `/billing?plan=pro` only *highlights* a card; upgrade click → `POST /billing/checkout` + → Stripe Checkout → redirect (correct). +4. **Provisioning** — webhook `checkout.session.completed` sets tier/sub IDs/check-interval from the + verified price ID (`billing/index.ts:218-256`). Automatic, no manual step. +5. **Gating** — cloud accounts, alert channels, team members, org creation, check interval all enforced + in the API. Pricing matches marketing (Pro $29/$290, Team $99/$990, Enterprise custom). + +## Gap list (severity-ranked) + +### BLOCKER — fix before charging at scale +- **B1. No payment-failure handling.** `customer.subscription.updated` only acts on `status==="active"`, + and `invoice.payment_failed` isn't handled → a failed card stays on the paid tier indefinitely + (`routes/billing/index.ts:259-273`). Fix: handle `invoice.payment_failed` + non-active statuses; + downgrade/flag after a grace period. +- **B2. Stripe v17 field-location bug.** `/billing/status` reads `current_period_end` / + `cancel_at_period_end` off the top-level Subscription, but SDK v17's default API version moved them + onto subscription items → UI gets `undefined` renewal/cancel info (`billing/index.ts:87-88`). Root cause: + no explicit `apiVersion` pin on the billing client (`billing/index.ts:16`; cf. `app.ts:338` which pins). + +### HIGH +- **H1 (security). Activity Log is UI-gated only.** `GET /activity` checks role but not tier + (`routes/activity/index.ts:22`), so a free/pro owner/admin can read the full audit log via direct API / + `ks_live_` key, though it's marketed Team/Enterprise-only. Fix: add a `requireTier("team","enterprise")` + check on the route. +- **H2 (conversion). `/billing?plan=` doesn't launch checkout** — only sets a CSS highlight + (`web/.../BillingPage.tsx:149`); marketing "Upgrade" buttons imply one-click. Fix: auto-call + `handleCheckout` when `?plan=` matches a paid tier (or a one-click "Continue to checkout" CTA). +- **H3. Hardcoded test-mode fallback price IDs.** If `STRIPE_PRICE_*` env vars are unset in prod, + checkout silently uses baked-in `price_1TE2hN...` test IDs (`billing/index.ts:20-23`). Fix: throw on + startup if missing in production. +- **H4. No webhook idempotency / event de-dup.** Stripe retries re-run handlers (`billing/index.ts:216-287`). + Fix: persist processed `event.id`s and short-circuit duplicates. +- **H5. No explicit `apiVersion` pin** on the billing Stripe client (root cause of B2) — behavior floats on + `npm update` (`billing/index.ts:16`). + +### MEDIUM +- **M1.** No `subscriptionStatus`/`currentPeriodEnd` persisted; every status read hits Stripe live + (`models/guardian-account/schema.ts:29-30`). Persist from webhooks. +- **M2.** Downgrade doesn't reconcile overages — excess cloud accounts/alert channels left active after + dropping to free (enforcement only triggers on new POSTs). +- **M3.** Free-tier expectation mismatch — marketing pricing lists only Pro/Team/Enterprise; no Free card, + yet signup lands on a usable Free tier (`site/index.html:1176-1219`). +- **M4.** Onboarding offers only email/discord/slack channels; platform also supports PagerDuty/GitHub. + +### LOW +- **L1.** `enforceTierLimits("alertChannels")` branch is **dead code** — the alertChannels limit is actually + enforced separately in `routes/alerts/index.ts:70` (so channels *are* gated; the billing-side branch is + just unused). Remove the unused param/branch. +- **L2.** `requireTeamTier` duplicated in `team/index.ts:18` and inline in `orgs/index.ts:94,312` — extract a + shared `requireTier(...)` middleware (also fixes H1 cleanly). +- **L3.** Team-member limit hardcoded `{team:10,enterprise:100}` in `team/index.ts:115` instead of `TIER_LIMITS`. +- **L4.** `success=true` banner says "plan active" before the async webhook may have landed + (`BillingPage.tsx:80`) — poll/refetch status a few times. +- **L5.** Webhook handler `console.error` on success paths; 500-on-error + no idempotency → double-apply risk. + +## Correct as-is (no action) +Webhook signature verification (fail-closed) + raw-body preservation; tier derived from verified price ID; +tier read only from DB (never client headers/body); RBAC `requirePermission` on all mutations; IDOR/ownership +checks on org routes; redirect-URL allowlisting; account deletion cancels Stripe subs; check-interval not +user-editable; dev auth-bypass triple-gated to non-prod; API-key path gated identically to UI. diff --git a/site/index.html b/site/index.html index fb28066..ddd5c89 100644 --- a/site/index.html +++ b/site/index.html @@ -29,7 +29,7 @@ /* ── Nav ─────────────────────────────────────── */ .navbar { - position: fixed; top: 0; left: 0; width: 100%; + position: sticky; top: 0; left: 0; width: 100%; z-index: 1000; display: flex; justify-content: space-between; align-items: center; padding: 0 2em; height: 58px; @@ -587,34 +587,38 @@ backdrop-filter: blur(4px); } .embed-cell .embed-more a:hover { background: rgba(92,226,231,0.15); } - /* overflow story-cards in a single-row horizontal scroll */ - .stories-scroll { + /* two horizontal-scroll rows: 3 cards visible per row, the 4th scrolls in */ + .stories-row { display: flex; gap: 20px; overflow-x: auto; overflow-y: hidden; - margin: 20px -24px 0; padding: 4px 24px 14px; + margin: 0 0 20px; padding: 4px 0 14px; scroll-snap-type: x proximity; scrollbar-width: thin; scrollbar-color: rgba(255,107,107,0.4) transparent; } - .stories-scroll .story-card { flex: 0 0 320px; scroll-snap-align: start; } - .stories-scroll::-webkit-scrollbar { height: 8px; } - .stories-scroll::-webkit-scrollbar-thumb { background: rgba(255,107,107,0.35); border-radius: 999px; } - .stories-scroll::-webkit-scrollbar-track { background: transparent; } - .scroll-hint { - font-size: 12px; color: #6b7280; margin: 22px 0 0; - text-transform: uppercase; letter-spacing: 1.5px; font-weight: 600; + .stories-row:last-of-type { margin-bottom: 0; } + .stories-row .story-card { + flex: 0 0 calc((100% - 40px) / 3); /* 3 visible; two 20px gaps */ + min-width: 260px; /* stay readable on narrow screens */ + scroll-snap-align: start; } - /* let the header sit over the visible video (shorter top fade) */ - .video-bg-section.show-video-top::before { height: 72px; } + .stories-row::-webkit-scrollbar { height: 8px; } + .stories-row::-webkit-scrollbar-thumb { background: rgba(255,107,107,0.35); border-radius: 999px; } + .stories-row::-webkit-scrollbar-track { background: transparent; } + /* header overlaps the video: minimal top fade + push the heading down into it */ + .video-bg-section.show-video-top::before { height: 36px; } + .video-bg-section.show-video-top > section { padding-top: 170px; } + .video-bg-section.show-video-top > video { opacity: 0.22; } /* a touch more so the texture reads behind the heading */ .embed-wall .twitter-tweet, .embed-wall .reddit-embed-bq { margin: 0 auto !important; width: 100% !important; } .embed-note { - text-align: center; font-size: 12px; color: #6b7280; - margin: 20px auto 0; max-width: 640px; + text-align: center; font-size: 15px; color: #8b8d96; + margin: 24px auto 0; max-width: 1000px; } + .embed-note a { font-weight: 700; } /* ── Pricing ─────────────────────────────────── */ .pricing-grid { @@ -1061,7 +1065,8 @@

$91K Cloud Bill -
+ +
$91,316
Cloud compute bill from runaway containers generating 1.38 billion requests in one month. $88K in compute duration alone.
@@ -1077,6 +1082,13 @@

Uncontrolled writes from a retry loop bug. No native circuit breaker to stop it.

Hacker News
+
+
$30,000
+
AWS Bedrock + Claude Opus billed $30K despite a cost-cap setting the customer believed was enabled.
+
The Register →
+
+
+
$104,500
Free-tier static site hit 60.7 TB of bandwidth in four days. One MP3 downloaded millions of times.
@@ -1092,14 +1104,6 @@

Two AI agents locked in a recursive loop, exchanging clarification requests thousands of times around the clock.

DEV Community →
- -

More incidents — scroll →

-
-
-
$30,000
-
AWS Bedrock + Claude Opus billed $30K despite a cost-cap setting the customer believed was enabled.
-
The Register →
-
$6,000
Someone left Claude Code running overnight. Token-based billing plus an agentic loop did the rest.
@@ -1110,12 +1114,12 @@

Straight From the Source

People are posting these in real time
-
From founders, indie devs, and us — the bill-shock conversation in real time. Scroll →
+
From founders and indie devs who got the surprise bill — the conversation, in real time. Scroll →
- +
- -
View on X →
+ +
View on X →
@@ -1132,18 +1136,13 @@

View on X →

- -
- -
View on X →
-
View on X →
-

Embedded directly from the original posts — nothing here is altered or out of context. Got a bill-shock story? Tell @KillSwitchCloud.

+

Embedded directly from the original posts — nothing here is altered or out of context. Got a bill-shock story? 💌 Tell @KillSwitchCloud