diff --git a/.claude/commands/review.md b/.claude/commands/review.md index a8a7b4f..a4e8828 100644 --- a/.claude/commands/review.md +++ b/.claude/commands/review.md @@ -40,7 +40,6 @@ touching the auth bridge — a subtle bug here is a security issue: - `backendServer/backend/jwt_auth.py` - `backendServer/backend/permissions.py` - `theCommonsWeb/src/lib/auth.ts` -- `theCommonsWeb/src/lib/lazy-auth-plugin.ts` For these, reason explicitly about token verification, the `BearerTokenAuthentication` path, permission classes, and the `neon_auth` (`BetterAuthUser`) mirror. diff --git a/AGENTS.md b/AGENTS.md index fd8d4ee..06036de 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -71,7 +71,7 @@ Run backend + theCommonsWeb together for end-to-end auth (Django validates JWTs | Concern | Key files | Deep dive | |---------|-----------|-----------| -| Auth bridge | `backend/jwt_auth.py`, `backend/permissions.py`, `src/lib/auth.ts`, `src/lib/lazy-auth-plugin.ts` | [ARCHITECTURE.md §Authentication](ARCHITECTURE.md#authentication) | +| Auth bridge | `backend/jwt_auth.py`, `backend/permissions.py`, `src/lib/auth.ts` | [ARCHITECTURE.md §Authentication](ARCHITECTURE.md#authentication) | | Data models | `events/models.py`, `ingestion/models.py`, `broadcast/models.py` | [ARCHITECTURE.md §Data Models](ARCHITECTURE.md#data-models) | | API endpoints | `backend/urls.py`, `events/urls.py`, `broadcast/urls.py` | [ARCHITECTURE.md §API Endpoints](ARCHITECTURE.md#api-endpoints) | | Ingestion pipeline | `ingestion/services.py`, `ingestion/standardizer.py`, `ingestion/importers/`, `ingestion/safety_scorer.py` | [docs/ingestion-pipeline.md](docs/ingestion-pipeline.md) | diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 7232bf3..b3558da 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -30,7 +30,7 @@ Data lives in Postgres on Neon (`public` schema owned by Django, `neon_auth` sch | `Category` | `slug` (unique), `display_name` | M2M with `Event` | | `UserProfile` | `uuid`, `user_type` (LOCAL/BUSINESS/VENUE), `primary_city`, `address`, `email_preference` (WEEKLY/MONTHLY/NEVER) | OneToOne→`BetterAuthUser` (`db_constraint=False`); M2M→`Tag` | | `BusinessProfile` | `uuid`, `business_name`, `description`, `contact_email/phone`, `is_published`, timestamps | OneToOne→`BetterAuthUser`; M2M→`Tag`; M2M→`Town` (`service_area`) | -| `NewsletterSubscriber` | `email` (unique), `frequency`, `is_active`, `subscribed_at` | — | +| `NewsletterSubscriber` | `email` (unique), `frequency`, `is_active`, `manage_token` (UUID, unique — unguessable credential for the manage link), `subscribed_at` | — | | `Event` | `uuid` (PK), `title`, `date` (indexed), `venue`, `description`, `price`, `photo`, `link`, `is_verified`, `source_name` | FK→`Town` (SET_NULL); M2M→`Tag`, `Category`; FK→`BetterAuthUser` (`created_by`) | ### Better Auth mirrors — `neon_auth` schema (`managed = False`) @@ -85,7 +85,8 @@ Notes that apply throughout: | POST | `/api/events/publish-approved` | API key | Queue bulk publish of approved staged events | | POST | `/api/events/direct-submit` | JWT optional (anonymous allowed) | Direct host event submission — fire-and-forget from broadcast SPA; 10/m by IP; invalid token → 401 | | GET/PATCH | `/auth/me` | user | Read / update own profile | -| POST | `/auth/subscribe` | — | Newsletter signup | +| POST | `/newsletter/subscribe` | — | Newsletter signup (`{email, frequency}`); sends a welcome email with a manage link | +| GET/PATCH | `/newsletter/manage` | — (token) | Manage a subscription via `?token=` — GET returns `{email, frequency, is_active}`; PATCH body `{frequency: WEEKLY\|MONTHLY\|NEVER}` (`NEVER` sets `is_active=false`) | | GET/POST | `/businesses` | user | Browse published businesses / create a listing | | GET | `/businesses/me` | user | Own business listing | | GET/PATCH/DELETE | `/businesses/` | user | Business listing CRUD | @@ -99,7 +100,7 @@ Notes that apply throughout: | GET | `/events/` | — | Paginated published events (window/after/before/category filters, Redis-cached) | | GET | `/events/towns/` | — | Town list (cached) | | GET | `/events/categories/` | — | Category list (cached) | -| GET | `/events/me/profile` | user | Own profile summary (includes derived `has_password`) | +| GET | `/events/me/profile` | user | Own profile summary | | GET | `/events/me/events` | user | Own staged + published events | | GET/PATCH/DELETE | `/events/staged/` | user | Manage own staged submission | | GET/DELETE | `/events/` | user (delete) | Event detail / owner delete | @@ -124,13 +125,13 @@ Auth via Bearer JWT or `X-Broadcast-Access-Code` header, resolved to a tier by ` | GET | `/broadcast/jobs//manual/` | tier ≥ 1 | Recipe JSON for a `needs_manual` target | | GET | `/broadcast/mock-form` | — (`DEBUG` only) | Dev-only mock submission form | -> Login/signup/logout are handled by Better Auth in **Next.js** at `/api/auth/*` (including lazy `POST /api/auth/enter` and `POST /api/auth/set-password`). +> Login/signup/logout are handled by Better Auth in **Next.js** at `/api/auth/*` (standard `emailAndPassword` sign-up/sign-in, fronted by the portal). --- ## Authentication -**Key files:** `backend/jwt_auth.py`, `backend/permissions.py`, `src/lib/auth.ts`, `src/lib/lazy-auth-plugin.ts`, `src/lib/redirect-allowlist.ts`, `src/hooks/useAuth.tsx`, `src/app/(portal)/`, `src/components/layout/SiteChrome.tsx`, `src/app/api/auth/set-password/route.ts` +**Key files:** `backend/jwt_auth.py`, `backend/permissions.py`, `src/lib/auth.ts`, `src/lib/redirect-allowlist.ts`, `src/hooks/useAuth.tsx`, `src/app/(portal)/`, `src/components/layout/SiteChrome.tsx` Auth is owned by **Better Auth running inside Next.js**, fronted by a standalone **portal** — there are no Django login/signup endpoints, and no app renders its own embedded auth form anymore. Django only *verifies* tokens. @@ -140,7 +141,7 @@ Better Auth — and the portal UI in front of it — is served at **`https://aut ### The portal -The portal is a route group, `src/app/(portal)/`, inside the same Next.js app — not a separate service. Routes: `/signin`, `/join` (passwordless lazy-auth create-account), `/set-password`, `/forgot-password`. `PortalShell` (`src/app/(portal)/PortalShell.tsx`) renders standalone split-panel chrome with a SIGN IN / CREATE ACCOUNT tab switcher; a client gate, `src/components/layout/SiteChrome.tsx` (checks `usePathname()` against the portal paths), hides the apex `Header`/`Footer`/banners on those routes so the portal has its own chrome, while every other route is unchanged. +The portal is a route group, `src/app/(portal)/`, inside the same Next.js app — not a separate service. Routes: `/signin`, `/join` (create account: email + password + confirm, one step), `/forgot-password`. `PortalShell` (`src/app/(portal)/PortalShell.tsx`) renders standalone split-panel chrome with a SIGN IN / CREATE ACCOUNT tab switcher; a client gate, `src/components/layout/SiteChrome.tsx` (checks `usePathname()` against the portal paths), hides the apex `Header`/`Footer`/banners on those routes so the portal has its own chrome, while every other route is unchanged. Every service that needs a user to authenticate redirects into the portal with `?redirect_to=`. `src/lib/redirect-allowlist.ts` exports `resolveRedirect(raw, fallback='/')`, which validates the destination against an allowlist (`thecommons.town`, `*.thecommons.town`, `localhost`/`127.0.0.1` in dev) as an open-redirect guard; the portal completes sign-in with `window.location.href = resolveRedirect(...)` — a full cross-subdomain navigation, not a client-side route change. The apex app's own former embedded flow (`src/app/auth/AuthFlow.tsx`, `src/app/auth/google-popup/`) was removed; `/auth`, `/auth/login`, `/auth/signup` are now thin server-redirect shims that map the old `?redirect=`/`?intent=` params to an absolute `redirect_to` and bounce into the portal. In-app "Sign in"/"Sign up" entry points (Header, sidebar, post gate, digest CTA) navigate straight to the portal. `broadcastWeb` does the same: its former inline `AuthModal` was removed, and its "Sign in / Create account" button does a full navigation to `${VITE_BETTER_AUTH_URL}/signin?redirect_to=` — the shared session brings the user back. @@ -155,16 +156,8 @@ Every service that needs a user to authenticate redirects into the portal with ` ### User-creation side effect `src/lib/auth.ts` defines `databaseHooks.user.create.after`, which inserts a matching `public.events_userprofile` row whenever Better Auth creates a user — so every account has a Django profile. -### Lazy (passwordless) accounts -Signup is email-first, password-optional. The custom plugin `src/lib/lazy-auth-plugin.ts` exposes `POST /api/auth/enter`: -- New email → creates a Better Auth user (no credential) + session; the `databaseHook` fires. -- Existing passwordless email → fresh session. -- Existing email with a password → returns `requiresPassword: true` (no session); frontend collects the password and uses normal `signIn.email`. - -Users secure the account later via `POST /api/auth/set-password` (links a `credential` account). **No email verification for MVP.** - -### `has_password` is derived -Django computes it from the `BetterAuthAccount` mirror (`provider_id='credential'` with a non-null password) and returns it on `/auth/me` and `/events/me/profile`. No column, no migration. +### Account creation is password-required +Signup collects **email + password + confirm** in one step on `/join`, via Better Auth's standard `emailAndPassword` flow (`autoSignIn: true` in `src/lib/auth.ts`) — `signUp.email` creates the Better Auth user + `credential` account and signs the user in immediately; the `databaseHook` fires as usual. There is no passwordless/email-only path and no separate set-password step. **No email verification for MVP.** ### Google sign-in — DISABLED Commented out in `src/lib/auth.ts`. The client popup flow that used to live at `src/app/auth/google-popup/` was removed along with the rest of the pre-portal embedded auth UI; re-enabling Google sign-in needs a new post-OAuth account-type step built into the portal. Revisit later. @@ -211,14 +204,15 @@ Flow: tier-based auth (Bearer JWT or access code, resolved by `broadcast/access. - **Celery** app is built in `backend/celery.py`, loaded eagerly via `backend/__init__.py`, and autodiscovers tasks. `CELERY_TIMEZONE = UTC` (beat entries carry their own tz). - **Beat** uses `django_celery_beat`'s `DatabaseScheduler` — schedules live in Postgres and are editable in admin. Seeded by migrations: - `weekly-digest-sunday` → `events.tasks.fan_out_weekly_digest`, Sun 18:00 America/New_York (`events/migrations/0015_seed_digest_beat.py`). + - `monthly-digest` → `events.tasks.fan_out_monthly_digest`, 1st of month 18:00 America/New_York (`events/migrations/0020_seed_monthly_digest_beat.py`). - `ingest-events-daily` → `ingestion.tasks.run_ingestion_pipeline`, 04:00 America/New_York (`ingestion/migrations/0007_seed_ingest_beat.py`). -- **Tasks:** `events.tasks` (`ping`, `send_one_digest`, `fan_out_weekly_digest`), `ingestion.tasks` (`run_ingestion_pipeline`, `publish_all_approved_task`). +- **Tasks:** `events.tasks` (`ping`, `send_one_digest`, `fan_out_weekly_digest`, `fan_out_monthly_digest`), `ingestion.tasks` (`run_ingestion_pipeline`, `publish_all_approved_task`). - **Read-endpoint cache:** `events/cache.py` is a version-keyed Redis cache for the hot list endpoints; `events/signals.py` bumps the version on `Event`/`Town`/`Category` writes to invalidate. See [docs/redis-celery-handoff.md](docs/redis-celery-handoff.md). ### Email digests -`events/email_service.py` wraps **Brevo** transactional email and builds digest HTML from `templates/email/`. `fan_out_weekly_digest` queues one `send_one_digest` per WEEKLY `UserProfile`. Management commands (`send_digest`, `send_test_digest`, `send_weekly_digest`) cover synchronous/test sends. +`events/email_service.py` wraps **Brevo** transactional email and builds digest HTML from `templates/email/`. `NewsletterSubscriber` is the single source of truth for both weekly and monthly digests: `_build_recipients(frequency)` resolves the recipient list (deduped by email) from active subscriber rows — anonymous newsletter subscribers get all events, account holders (`UserProfile.email_preference`) are tag-filtered — and returns `{email, tags, manage_token}` per recipient. This one resolver backs both the Celery path (`fan_out_weekly_digest` / `fan_out_monthly_digest` queue one `send_one_digest` per recipient) and the synchronous `send_digest`/`send_weekly_digest` management commands (`send_test_digest` sends a one-off test). Every digest email carries a "Manage preferences / Unsubscribe" link built from the recipient's `manage_token` (`/newsletter/manage?token=`). --- @@ -239,12 +233,10 @@ The main site is **Next.js 16 App Router**. Root layout (`src/app/layout.tsx`) w | `/dashboard` | `app/dashboard/page.tsx` | client | Manage submitted events + business listing | | `/auth`, `/auth/login`, `/auth/signup` | `app/auth/{page,login/page,signup/page}.tsx` | server redirect shim | Legacy entry points — map old `?redirect=`/`?intent=` to `redirect_to` and bounce into the portal (`/join` or `/signin`) | | `/signin` | `app/(portal)/signin/page.tsx` | client (`PortalShell` + `SignInForm`) | Portal sign-in | -| `/join` | `app/(portal)/join/page.tsx` | client (`PortalShell` + `JoinForm`) | Portal passwordless create-account | -| `/set-password` | `app/(portal)/set-password/page.tsx` | client | Set a password on a passwordless account | +| `/join` | `app/(portal)/join/page.tsx` | client (`PortalShell` + `JoinForm`) | Portal create-account (email + password + confirm) | | `/forgot-password` | `app/(portal)/forgot-password/page.tsx` | client | Password reset request | | `/events/[uuid]` | `app/events/[uuid]/page.tsx` | server (async) | Event detail (`generateMetadata` + OpenGraph) | | `/api/auth/[...all]` | `app/api/auth/[...all]/route.ts` | route | Better Auth handler | -| `/api/auth/set-password` | `app/api/auth/set-password/route.ts` | route | Set password on a passwordless account | `/auth/google-popup/` (the disabled Google OAuth popup) was removed along with the rest of the pre-portal embedded auth UI — see [§Authentication](#authentication). diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md index 6757705..bc7700a 100644 --- a/PROJECT_CONTEXT.md +++ b/PROJECT_CONTEXT.md @@ -69,7 +69,7 @@ docs/ # broadcast, ingestion-pipeline, safety-scoring, admin-backe - **`Event`** — UUID PK · title · town (FK, SET_NULL) · date (indexed) · venue · description · price · photo · link · `tags`/`categories` (M2M) · `is_verified` · `source_name` · `created_by` (FK → `BetterAuthUser`). - **`UserProfile`** — OneToOne → `BetterAuthUser` · `user_type` (LOCAL/BUSINESS/VENUE) · `primary_city` · `address` · `email_preference` (WEEKLY/MONTHLY/NEVER) · `tags` (M2M). Created automatically via a Better Auth `databaseHook`. - **`BusinessProfile`** — OneToOne → `BetterAuthUser` · `business_name` · `description` · `tags` (M2M) · `service_area` (M2M Town) · contacts · `is_published`. -- **`NewsletterSubscriber`** — `email` · `frequency` · `is_active` · `subscribed_at`. +- **`NewsletterSubscriber`** — `email` · `frequency` · `is_active` · `manage_token` (UUID, unique — unguessable manage-link credential) · `subscribed_at`. Single source of truth for both weekly and monthly digests. ### Better Auth mirrors — `neon_auth` schema, `managed = False` `BetterAuthUser`, `BetterAuthSession`, `BetterAuthAccount`, `BetterAuthVerification`, `BetterAuthJwks`. Django maps them read-only via the cross-schema `db_table` trick (`'neon_auth"."user'`); FKs use `db_constraint=False`. **Never migrate them.** @@ -105,27 +105,27 @@ docs/ # broadcast, ingestion-pipeline, safety-scoring, admin-backe | GET/DELETE | `/events/` | user (delete) | Event detail / owner delete | | POST | `/events/create` | user or key | Submit an event → StagedEvent | | GET/PATCH | `/auth/me` | user | Read / update profile | -| POST | `/auth/subscribe` | — | Newsletter signup | +| POST | `/newsletter/subscribe` | — | Newsletter signup (`{email, frequency}`, sends welcome email + manage link) | +| GET/PATCH | `/newsletter/manage` | — (token) | View / change subscription via `?token=` | | GET/POST | `/businesses` · `/businesses/me` · `/businesses/` | user | Business listing CRUD | | GET | `/api/cron/ingest` | CRON_SECRET | Queue ingestion pipeline | | POST | `/api/events/publish-approved` | key | Queue bulk publish | | POST/GET | `/broadcast/...` | code | Preview/submit/jobs/screenshots/manual (see §10) | -> Login/signup/logout are handled by Better Auth in **Next.js** at `/api/auth/*` (incl. lazy `/api/auth/enter` and `/api/auth/set-password`). Django admin at `/admin/` (django-unfold). +> Login/signup/logout are handled by Better Auth in **Next.js** at `/api/auth/*` (standard `emailAndPassword` sign-up/sign-in). Django admin at `/admin/` (django-unfold). --- ## 6. Authentication — Better Auth ↔ Django Bridge -**Key files:** `backend/jwt_auth.py`, `backend/permissions.py`, `src/lib/auth.ts`, `src/lib/lazy-auth-plugin.ts`, `src/hooks/useAuth.tsx`, `src/app/api/auth/set-password/route.ts` +**Key files:** `backend/jwt_auth.py`, `backend/permissions.py`, `src/lib/auth.ts`, `src/hooks/useAuth.tsx` Auth is owned by **Better Auth inside Next.js** — no Django login/signup endpoints. Django only *verifies* tokens. - Browser holds a Better Auth session cookie; to call Django it fetches a short-lived **JWT** from `/api/auth/token` and sends `Authorization: Bearer `. - `BearerTokenAuthentication` accepts either a **Better Auth JWT** (verified statelessly against the frontend JWKS, in-process cache with TTL + stale-grace; `sub` → `BetterAuthUser`) **or** the shared **`THE_COMMONS_API_KEY`** (no user). - `databaseHooks.user.create.after` inserts a matching `events_userprofile` on user creation. -- **Lazy passwordless accounts:** email-first signup via `POST /api/auth/enter`; users secure the account later via `POST /api/auth/set-password`. No email verification for MVP. -- **`has_password`** is derived from the `BetterAuthAccount` mirror — no column, no migration. +- **Password required at signup:** the portal's `/join` collects email + password + confirm in one step via Better Auth's standard `emailAndPassword` (`autoSignIn`). No passwordless path, no separate set-password step, no email verification for MVP. - **Google sign-in is DISABLED** (commented out in `auth.ts`, `AuthFlow.tsx`, `google-popup/`). --- @@ -149,15 +149,15 @@ Runs daily via Celery beat (04:00 ET) or `POST /api/cron/ingest` (`CRON_SECRET`) **Key files:** `backend/celery.py`, `events/tasks.py`, `ingestion/tasks.py`, `events/cache.py`, `events/signals.py` - One Redis instance: **DB 0** = Celery broker + results (`REDIS_URL`), **DB 1** = Django cache (`REDIS_CACHE_URL`). -- Celery autodiscovers tasks; `django_celery_beat` `DatabaseScheduler` holds schedules in Postgres, seeded by migrations: weekly digest (Sun 18:00 ET), ingest (04:00 ET). -- Tasks: `events.tasks` (`ping`, `send_one_digest`, `fan_out_weekly_digest`), `ingestion.tasks` (`run_ingestion_pipeline`, `publish_all_approved_task`). +- Celery autodiscovers tasks; `django_celery_beat` `DatabaseScheduler` holds schedules in Postgres, seeded by migrations: weekly digest (Sun 18:00 ET), monthly digest (1st of month 18:00 ET), ingest (04:00 ET). +- Tasks: `events.tasks` (`ping`, `send_one_digest`, `fan_out_weekly_digest`, `fan_out_monthly_digest`), `ingestion.tasks` (`run_ingestion_pipeline`, `publish_all_approved_task`). - Read-endpoint cache (`events/cache.py`) is version-keyed; `events/signals.py` invalidates on Event/Town/Category writes. - The **broadcast worker is NOT Celery** — it has its own Postgres queue. See `docs/redis-celery-handoff.md`. ### Email digests -`events/email_service.py` wraps **Brevo**; digest HTML in `templates/email/`. Commands: `send_digest`, `send_test_digest --email`, `send_weekly_digest`. +`events/email_service.py` wraps **Brevo**; digest HTML in `templates/email/`. `NewsletterSubscriber` is the single source of truth for both weekly and monthly digests — `_build_recipients(frequency)` dedupes anonymous subscribers and tag-filtered account holders by email and returns each recipient's `manage_token` for the "Manage preferences / Unsubscribe" link (`/newsletter/manage?token=`). Commands: `send_digest`, `send_test_digest --email`, `send_weekly_digest`. --- @@ -176,7 +176,7 @@ Next.js 16 App Router; root layout wraps `QueryProvider → AuthProvider → Mes | `/dashboard` | `app/dashboard/page.tsx` | client | Manage events + business listing | | `/auth[/login\|/signup]` | `app/auth/` | server → client `AuthFlow` | Login / signup | | `/events/[uuid]` | `app/events/[uuid]/page.tsx` | server | Event detail (OpenGraph) | -| `/api/auth/[...all]` · `/api/auth/set-password` | `app/api/auth/` | route | Better Auth handler / set-password | +| `/api/auth/[...all]` | `app/api/auth/` | route | Better Auth handler | ### Data layer TanStack Query (`lib/queryClient.ts`: `staleTime/gcTime: Infinity`, `retry: 1`), provided by `QueryProvider`. Keys: `['towns']`, `['categories']`, `['profile', token]`, `['events', …]`, `['myEvents', token]`, `['myBusiness', token]`. Services (`src/services/`) call Django over `fetch` at `NEXT_PUBLIC_API_BASE_URL`; `fetchWithRetry` covers Neon cold-starts. Auth combined in `useAuth` (session + JWT + profile); no `middleware.ts` (client-side route guards). diff --git a/backendServer/AGENTS.md b/backendServer/AGENTS.md index b3a111b..f339514 100644 --- a/backendServer/AGENTS.md +++ b/backendServer/AGENTS.md @@ -20,7 +20,7 @@ backendServer/ │ ├── views.py / serializers.py / urls.py │ ├── cache.py # Version-keyed Redis cache for hot read endpoints │ ├── signals.py # Cache invalidation on Event/Town/Category writes -│ ├── tasks.py # Celery: ping, send_one_digest, fan_out_weekly_digest +│ ├── tasks.py # Celery: ping, send_one_digest, fan_out_weekly_digest, fan_out_monthly_digest │ ├── email_service.py # Brevo transactional email + digest builder │ └── management/commands/ # devserver, seed_dev, healthcheck, delete_user, send_*digest ├── ingestion/ # Pipeline app @@ -61,7 +61,8 @@ Auth: `—` public · `user` Better Auth JWT · `key` `THE_COMMONS_API_KEY` · ` | GET/DELETE | `/events/` | user (delete) | Event detail / owner delete | | POST | `/events/create` | user or key | Submit event → StagedEvent | | GET/PATCH | `/auth/me` | user | Read / update profile | -| POST | `/auth/subscribe` | — | Newsletter signup | +| POST | `/newsletter/subscribe` | — | Newsletter signup (welcome email + manage link) | +| GET/PATCH | `/newsletter/manage` | — (token) | View / change a subscription via `?token=` | | GET/POST | `/businesses` · `/businesses/me` · `/businesses/` | user | Business listing CRUD | | GET | `/api/cron/ingest` | CRON_SECRET | Queue ingestion pipeline | | POST | `/api/events/publish-approved` | key | Queue bulk publish | diff --git a/backendServer/backend/urls.py b/backendServer/backend/urls.py index a958562..85375c6 100644 --- a/backendServer/backend/urls.py +++ b/backendServer/backend/urls.py @@ -9,7 +9,14 @@ from django.contrib import admin from django.urls import include, path -from events.views import business_detail, businesses, me, my_business, subscribe +from events.views import ( + business_detail, + businesses, + me, + my_business, + newsletter_manage, + subscribe, +) from ingestion.views import ( admin_docs, cron_ingest, @@ -29,7 +36,8 @@ path("api/cron/ingest", cron_ingest, name="cron-ingest"), path("api/events/publish-approved", publish_approved_events, name="publish-approved-events"), path("api/events/direct-submit", direct_submit, name="direct-submit"), - path("auth/subscribe", subscribe, name="subscribe"), + path("newsletter/subscribe", subscribe, name="subscribe"), + path("newsletter/manage", newsletter_manage, name="newsletter-manage"), path("auth/me", me, name="auth-me"), path("businesses", businesses, name="businesses"), path("businesses/me", my_business, name="my-business"), diff --git a/backendServer/events/email_service.py b/backendServer/events/email_service.py index 2c16a21..a6bc0a2 100644 --- a/backendServer/events/email_service.py +++ b/backendServer/events/email_service.py @@ -50,36 +50,64 @@ def send_email(to: str, subject: str, html: str, text: str | None = None) -> boo return False -def _build_recipients(frequency: str) -> list[dict]: - """Return [{email, tags: set[str]}] for all subscribers of this frequency. +def send_newsletter_welcome(email: str, manage_token) -> bool: + """Send the welcome email for a new (or re-)subscription. - UserProfile entries (authenticated users) take priority over - NewsletterSubscriber rows when both share an email address. UserProfile - subscribers get tag-filtered content; anonymous subscribers get everything. + Includes the login-free manage link keyed by the subscriber's manage_token. + Best-effort — a Brevo failure here should never block the subscribe response. """ - from .models import NewsletterSubscriber, UserProfile + manage_url = manage_url_for(manage_token) + subject = "You're subscribed to The Commons" + html = ( + "

Thanks for subscribing to The Commons newsletter.

" + "

You can change your frequency or unsubscribe anytime, no login required, " + f'at {manage_url}.

' + ) + return send_email(email, subject, html) - pref_map = {"WEEKLY": "WEEKLY", "MONTHLY": "MONTHLY"} - db_pref = pref_map[frequency] - # Authenticated users with a profile - profiles = ( - UserProfile.objects.filter(email_preference=db_pref) - .select_related("user") - .prefetch_related("tags") - ) - seen = {} - for profile in profiles: - email = profile.user.email.lower() - seen[email] = {t.name for t in profile.tags.all()} +def digest_window(frequency: str) -> tuple: + """Return (cutoff, subject) for a digest frequency. Shared by send_digest + and the per-recipient Celery task so the two paths can't drift apart. + """ + if frequency == "WEEKLY": + return timezone.now() + timedelta(days=7), "This Week in The Commons" + return timezone.now() + timedelta(days=31), "This Month in The Commons" - # Anonymous newsletter subscribers — skip if already covered by a profile - for sub in NewsletterSubscriber.objects.filter(frequency=frequency, is_active=True): - email = sub.email.lower() - if email not in seen: - seen[email] = set() # empty = no tag filter → send all events - return [{"email": email, "tags": tags} for email, tags in seen.items()] +def manage_url_for(manage_token) -> str: + site_url = os.environ.get("SITE_URL", "https://www.thecommons.town") + return f"{site_url}/newsletter/manage?token={manage_token}" + + +def _build_recipients(frequency: str) -> list[dict]: + """Return [{email, tags: set[str], manage_token}] for all active subscribers. + + NewsletterSubscriber is the single source of truth for digest recipients — + every row (anonymous or account-holding) carries a manage_token, so this is + the only resolver that can produce a working manage/unsubscribe link. When + a subscriber's email also has a UserProfile, its tags narrow the digest to + matching events; otherwise (anonymous) the empty set sends everything. + Both the Celery fan-out and the send_digest command call this — no + divergent recipient logic anywhere else. + """ + from .models import NewsletterSubscriber, UserProfile + + profile_tags_by_email = { + profile.user.email.lower(): {t.name for t in profile.tags.all()} + for profile in UserProfile.objects.select_related("user").prefetch_related("tags") + } + + recipients = [] + for sub in NewsletterSubscriber.objects.filter(frequency=frequency, is_active=True): + recipients.append( + { + "email": sub.email, + "tags": profile_tags_by_email.get(sub.email.lower(), set()), + "manage_token": sub.manage_token, + } + ) + return recipients def send_digest(frequency: str) -> dict: @@ -89,12 +117,7 @@ def send_digest(frequency: str) -> dict: """ from .models import Event - if frequency == "WEEKLY": - cutoff = timezone.now() + timedelta(days=7) - subject = "This Week in The Commons" - else: - cutoff = timezone.now() + timedelta(days=31) - subject = "This Month in The Commons" + cutoff, subject = digest_window(frequency) all_events = list( Event.objects.filter(date__gte=timezone.now(), date__lte=cutoff) @@ -127,6 +150,7 @@ def send_digest(frequency: str) -> dict: "frequency": frequency, "subject": subject, "site_url": site_url, + "manage_url": manage_url_for(recipient["manage_token"]), }, ) diff --git a/backendServer/events/management/commands/rollover_passwordless_accounts.py b/backendServer/events/management/commands/rollover_passwordless_accounts.py new file mode 100644 index 0000000..907387d --- /dev/null +++ b/backendServer/events/management/commands/rollover_passwordless_accounts.py @@ -0,0 +1,183 @@ +"""Suite 38.A4 — roll over accounts stranded by the removed passwordless flow. + +Before suite 38 (38.A1/38.A2), users could sign up/sign in with just an email +(no password) — a `neon_auth.user` row existed but with no matching +`neon_auth.account` row for `provider_id='credential'`, or a credential row +with a null password. Now that the passwordless flow is gone (SignInForm.tsx +requires email + password), those users are locked out: there is no +password for them to enter. + +This command identifies those users and, with --send, emails each one a +rollover notice via Brevo (events.email_service.send_email). Defaults to a +dry run: it only prints who would be emailed. + +IMPORTANT — read docs/suite-38-passwordless-rollover.md before ever passing +--send. As of 2026-07-31 the link this email points users to does NOT yet +lead anywhere functional: Better Auth's reset-password endpoint is disabled +(`emailAndPassword.sendResetPassword` is unset in theCommonsWeb/src/lib/auth.ts), +there is no `/reset-password/[token]` page, and the existing `/forgot-password` +page is a stale stub left over from the passwordless era (it tells users "we +don't send reset emails, just sign in without a password" — which is no +longer true and doesn't call any Better Auth API). Sending this email before +that wiring lands would hand users a dead-end link. See the doc for the +exact prerequisite changes and how to verify them before an actual --send. +""" + +import os + +from django.core.management.base import BaseCommand +from django.db import connection + +from events.email_service import send_email + +# Where the "set your password" call to action points. This is the frontend +# self-serve entry point named in the 38.A4 decision. As documented above and +# in docs/suite-38-passwordless-rollover.md, this page must be rewired to +# actually call Better Auth's requestPasswordReset before this link works — +# do not --send until that prerequisite is done and verified. +RESET_PATH = "/forgot-password" + +EMAIL_SUBJECT = "Action needed: set a password for your Commons account" + + +def _reset_url() -> str: + site_url = os.environ.get("SITE_URL", "https://www.thecommons.town") + return f"{site_url}{RESET_PATH}" + + +def _email_html(name: str) -> str: + reset_url = _reset_url() + return ( + f"

Hi {name},

" + "

We've changed how sign-in works on The Commons. Your account was created " + 'before we required a password, so the old "just enter your email" sign-in ' + "no longer works for you.

" + f'

Set your password to keep using your account — ' + "it only takes a minute.

" + "

If you don't recognize this account, you can safely ignore this email.

" + "

— The Commons

" + ) + + +def _email_text(name: str) -> str: + reset_url = _reset_url() + return ( + f"Hi {name},\n\n" + "We've changed how sign-in works on The Commons. Your account was created " + 'before we required a password, so the old "just enter your email" sign-in ' + "no longer works for you.\n\n" + f"Set your password here: {reset_url}\n\n" + "If you don't recognize this account, you can safely ignore this email.\n\n" + "— The Commons" + ) + + +def find_affected_users() -> list[dict]: + """Return neon_auth users with no usable credential-provider password. + + Raw SQL, not the ORM, and for a more concrete reason than "it's a mirror + table": BetterAuthAccount.user_id (events/models.py) is declared as a + Django TextField, but the live `neon_auth.account."userId"` column is + actually `uuid` (confirmed via information_schema.columns against the + test DB — the model's type annotation has drifted from the schema it + mirrors). An ORM anti-join such as + + BetterAuthUser.objects.exclude( + id__in=BetterAuthAccount.objects.filter( + provider_id="credential", password__isnull=False, + ).values("user_id") + ) + + asks Django to bind the subquery's "userId" values as text (per the + model field) against `u.id` (uuid), and Postgres has no `uuid = text` + operator for a subquery comparison — it raises + `operator does not exist: uuid = text` at query time (reproduced while + building this command). Raw SQL sidesteps the model's stale type + entirely: both columns really are `uuid`, so a plain LEFT JOIN with no + cast is correct. Both mirror tables are managed=False; this is a + read-only SELECT, never a migration. + """ + query = """ + SELECT u.id, u.email, u.name + FROM neon_auth."user" u + LEFT JOIN neon_auth."account" a + ON a."userId" = u.id + AND a."providerId" = 'credential' + AND a.password IS NOT NULL + WHERE a.id IS NULL + ORDER BY u.email + """ + with connection.cursor() as cursor: + cursor.execute(query) + rows = cursor.fetchall() + return [{"id": row[0], "email": row[1], "name": row[2]} for row in rows] + + +class Command(BaseCommand): + help = ( + "Suite 38.A4: find neon_auth users left stranded by the removed " + "passwordless flow (no credential-provider account with a non-null " + "password) and, with --send, email each a rollover notice via Brevo. " + "Defaults to a dry run that only lists affected users — sends nothing. " + "Read docs/suite-38-passwordless-rollover.md before ever passing --send." + ) + + def add_arguments(self, parser): + parser.add_argument( + "--send", + action="store_true", + help="Actually send the rollover email via Brevo. Omit for a dry " + "run (default): prints the affected users and sends nothing.", + ) + parser.add_argument( + "--limit", + type=int, + default=None, + help="Only process the first N affected users — useful for a " + "small canary batch before sending to everyone.", + ) + + def handle(self, *args, **options): + send = options["send"] + limit = options["limit"] + + affected = find_affected_users() + if limit is not None: + affected = affected[:limit] + + self.stdout.write( + f"Found {len(affected)} account(s) with no usable password " + f"(neon_auth.user with no credential/password neon_auth.account row)." + ) + for row in affected: + self.stdout.write(f" - {row['email']} ({row['id']})") + + if not send: + self.stdout.write( + self.style.WARNING( + "Dry run only — no emails sent. Re-run with --send to actually " + "email these users (see docs/suite-38-passwordless-rollover.md " + "for the prerequisite wiring to check first)." + ) + ) + return + + if not affected: + self.stdout.write("Nothing to send.") + return + + sent, failed = 0, 0 + for row in affected: + display_name = row["name"] or row["email"] + ok = send_email( + to=row["email"], + subject=EMAIL_SUBJECT, + html=_email_html(display_name), + text=_email_text(display_name), + ) + if ok: + sent += 1 + else: + failed += 1 + + self.stdout.write(self.style.SUCCESS(f"Done. Sent: {sent}, Failed: {failed}")) diff --git a/backendServer/events/migrations/0019_newslettersubscriber_manage_token.py b/backendServer/events/migrations/0019_newslettersubscriber_manage_token.py new file mode 100644 index 0000000..a4ddbb6 --- /dev/null +++ b/backendServer/events/migrations/0019_newslettersubscriber_manage_token.py @@ -0,0 +1,42 @@ +import uuid + +from django.db import migrations, models + + +# Adds an unguessable manage_token per NewsletterSubscriber so prefs can be +# managed via a login-free URL (38.B1). A plain AddField with a callable +# default evaluates uuid.uuid4() once and writes the *same* value to every +# existing row, which would violate unique=True on a non-empty table. So the +# field lands first without the uniqueness constraint, gets a distinct value +# backfilled per row, and only then has unique=True enforced. +def backfill_manage_tokens(apps, schema_editor): + NewsletterSubscriber = apps.get_model("events", "NewsletterSubscriber") + for subscriber in NewsletterSubscriber.objects.all(): + subscriber.manage_token = uuid.uuid4() + subscriber.save(update_fields=["manage_token"]) + + +def noop_reverse(apps, schema_editor): + # Tokens are opaque and only meaningful going forward; nothing to restore. + pass + + +class Migration(migrations.Migration): + + dependencies = [ + ("events", "0018_seed_apex_durham_towns"), + ] + + operations = [ + migrations.AddField( + model_name="newslettersubscriber", + name="manage_token", + field=models.UUIDField(default=uuid.uuid4, editable=False, unique=False, db_index=False), + ), + migrations.RunPython(backfill_manage_tokens, noop_reverse), + migrations.AlterField( + model_name="newslettersubscriber", + name="manage_token", + field=models.UUIDField(default=uuid.uuid4, editable=False, unique=True, db_index=True), + ), + ] diff --git a/backendServer/events/migrations/0020_seed_monthly_digest_beat.py b/backendServer/events/migrations/0020_seed_monthly_digest_beat.py new file mode 100644 index 0000000..6e59b55 --- /dev/null +++ b/backendServer/events/migrations/0020_seed_monthly_digest_beat.py @@ -0,0 +1,52 @@ +"""Seed the django-celery-beat schedule for the monthly digest fan-out. + +Mirrors 0015_seed_digest_beat.py's weekly schedule — 1st of the month, 18:00 +ET. The timezone is set on the CrontabSchedule so beat tracks US Eastern DST +exactly as intended. + +This is a one-time seed — schedules live in Postgres and are edited live in +the django admin (Periodic Tasks). See docs/redis-celery-handoff.md. +""" +from django.db import migrations + +TASK_NAME = "monthly-digest-first" +TASK_PATH = "events.tasks.fan_out_monthly_digest" + + +def create_schedule(apps, schema_editor): + CrontabSchedule = apps.get_model("django_celery_beat", "CrontabSchedule") + PeriodicTask = apps.get_model("django_celery_beat", "PeriodicTask") + + crontab, _ = CrontabSchedule.objects.get_or_create( + minute="0", + hour="18", + day_of_week="*", + day_of_month="1", + month_of_year="*", + timezone="America/New_York", + ) + PeriodicTask.objects.update_or_create( + name=TASK_NAME, + defaults={ + "task": TASK_PATH, + "crontab": crontab, + "enabled": True, + }, + ) + + +def remove_schedule(apps, schema_editor): + PeriodicTask = apps.get_model("django_celery_beat", "PeriodicTask") + PeriodicTask.objects.filter(name=TASK_NAME).delete() + + +class Migration(migrations.Migration): + + dependencies = [ + ("events", "0019_newslettersubscriber_manage_token"), + ("django_celery_beat", "0019_alter_periodictasks_options"), + ] + + operations = [ + migrations.RunPython(create_schedule, remove_schedule), + ] diff --git a/backendServer/events/models.py b/backendServer/events/models.py index dd866ab..4008ed9 100644 --- a/backendServer/events/models.py +++ b/backendServer/events/models.py @@ -200,6 +200,7 @@ class Frequency(models.TextChoices): frequency = models.CharField(max_length=10, choices=Frequency.choices, default=Frequency.WEEKLY) is_active = models.BooleanField(default=True) subscribed_at = models.DateTimeField(auto_now_add=True) + manage_token = models.UUIDField(default=uuid.uuid4, editable=False, unique=True, db_index=True) def __str__(self): return f"{self.email} ({self.frequency})" diff --git a/backendServer/events/tasks.py b/backendServer/events/tasks.py index 8b0d5de..84d3667 100644 --- a/backendServer/events/tasks.py +++ b/backendServer/events/tasks.py @@ -1,12 +1,11 @@ import logging import os -from datetime import timedelta from celery import shared_task from django.template.loader import render_to_string from django.utils import timezone -from events.email_service import send_email +from events.email_service import _build_recipients, digest_window, manage_url_for, send_email logger = logging.getLogger(__name__) @@ -17,63 +16,49 @@ def ping(): @shared_task(bind=True, max_retries=3, default_retry_delay=300) -def send_one_digest(self, profile_id): - """Render and send the personalized weekly digest to one UserProfile. - - Mirrors the per-profile body of the send_weekly_digest command: resolve the - user's town, pull upcoming events for it, tag-filter against their interests, - and email it. send_email swallows Brevo errors and returns False, so a falsy - return triggers a retry (3x, 5-min backoff) without affecting other recipients. +def send_one_digest(self, email, tags, manage_token, frequency): + """Render and send the personalized digest to one resolved recipient. + + Takes an already-resolved recipient (email/tags/manage_token/frequency) + rather than a UserProfile id, so it serves both authenticated and + anonymous NewsletterSubscriber rows alike — the fan-out tasks are the only + callers, and both go through email_service._build_recipients first. `tags` + arrives as a list (Celery JSON-serializes task args) and is treated as a + set of interest-tag names to filter events by; an empty list sends + everything. send_email swallows Brevo errors and returns False, so a + falsy return triggers a retry (3x, 5-min backoff) without affecting other + recipients. """ - from events.models import Event, Town, UserProfile - - profile = ( - UserProfile.objects.select_related("user") - .prefetch_related("tags") - .filter(id=profile_id) - .first() - ) - if profile is None: - logger.info("send_one_digest: profile %s no longer exists — skipping.", profile_id) - return + from events.models import Event - email = profile.user.email - site_url = os.environ.get("SITE_URL", "https://www.thecommons.town") - subject = "The Commons — Your Weekly Digest" + tag_filter = set(tags) now = timezone.now() - cutoff = now + timedelta(days=7) - - town = Town.objects.filter(slug=profile.primary_city).first() - if town is None: - logger.info( - "send_one_digest: %s primary_city %r matches no Town — skipping.", - email, - profile.primary_city, - ) - return + cutoff, subject = digest_window(frequency) events = ( - Event.objects.filter(date__gte=now, date__lte=cutoff, town=town) + Event.objects.filter(date__gte=now, date__lte=cutoff) + .select_related("town") .prefetch_related("tags") .order_by("date") ) - - user_tags = set(profile.tags.values_list("name", flat=True)) - if user_tags: - events = [e for e in events if user_tags.intersection({t.name for t in e.tags.all()})] + if tag_filter: + events = [e for e in events if tag_filter.intersection({t.name for t in e.tags.all()})] else: events = list(events) if not events: - logger.info("send_one_digest: %s has no matching events this week — skipping.", email) + logger.info("send_one_digest: %s has no matching events — skipping.", email) return + site_url = os.environ.get("SITE_URL", "https://www.thecommons.town") html = render_to_string( - "email/weekly_digest.html", + "email/digest.html", { "events": events, - "site_url": site_url, + "frequency": frequency, "subject": subject, + "site_url": site_url, + "manage_url": manage_url_for(manage_token), }, ) @@ -85,16 +70,26 @@ def send_one_digest(self, profile_id): raise self.retry() +def _queue_digest_fan_out(frequency): + recipients = _build_recipients(frequency) + for recipient in recipients: + send_one_digest.delay( + recipient["email"], + list(recipient["tags"]), + str(recipient["manage_token"]), + frequency, + ) + logger.info("fan_out_%s_digest: queued %d digest subtasks.", frequency.lower(), len(recipients)) + return len(recipients) + + @shared_task def fan_out_weekly_digest(): """Queue one send_one_digest subtask per WEEKLY subscriber. Returns the count.""" - from events.models import UserProfile + return _queue_digest_fan_out("WEEKLY") - profile_ids = list( - UserProfile.objects.filter(email_preference="WEEKLY").values_list("id", flat=True) - ) - for profile_id in profile_ids: - send_one_digest.delay(profile_id) - logger.info("fan_out_weekly_digest: queued %d digest subtasks.", len(profile_ids)) - return len(profile_ids) +@shared_task +def fan_out_monthly_digest(): + """Queue one send_one_digest subtask per MONTHLY subscriber. Returns the count.""" + return _queue_digest_fan_out("MONTHLY") diff --git a/backendServer/events/tests/test_auth_bridge_db.py b/backendServer/events/tests/test_auth_bridge_db.py index dfe263c..7d5c690 100644 --- a/backendServer/events/tests/test_auth_bridge_db.py +++ b/backendServer/events/tests/test_auth_bridge_db.py @@ -18,7 +18,7 @@ from django.urls import reverse import backend.jwt_auth as jwt_auth -from events.models import BetterAuthAccount +from events.models import NewsletterSubscriber from .factories import make_user @@ -61,18 +61,6 @@ def _stub_jwks(self): rget.return_value.raise_for_status.return_value = None yield - def _add_credential(self, user): - now = datetime.now(UTC) - BetterAuthAccount.objects.create( - id=uuid.uuid4().hex, - account_id=uuid.uuid4().hex, - provider_id="credential", - user_id=str(user.id), - password="hashed-secret", - created_at=now, - updated_at=now, - ) - def test_valid_jwt_authenticates_and_returns_profile(self): token = self._token_for(self.user.id) with self._stub_jwks(): @@ -80,32 +68,36 @@ def test_valid_jwt_authenticates_and_returns_profile(self): self.assertEqual(resp.status_code, 200) self.assertEqual(resp.json()["email"], self.user.email) - def test_has_password_true_with_credential_account(self): - self._add_credential(self.user) - token = self._token_for(self.user.id) - with self._stub_jwks(): - resp = self.client.get(reverse("auth-me"), HTTP_AUTHORIZATION=f"Bearer {token}") - self.assertEqual(resp.status_code, 200) - self.assertTrue(resp.json()["has_password"]) - - def test_has_password_false_without_credential_account(self): + def test_patch_me_updates_through_bridge(self): token = self._token_for(self.user.id) with self._stub_jwks(): - resp = self.client.get(reverse("auth-me"), HTTP_AUTHORIZATION=f"Bearer {token}") + resp = self.client.patch( + reverse("auth-me"), + data={"primary_city": "carrboro"}, + content_type="application/json", + HTTP_AUTHORIZATION=f"Bearer {token}", + ) self.assertEqual(resp.status_code, 200) - self.assertFalse(resp.json()["has_password"]) + self.assertEqual(resp.json()["primary_city"], "carrboro") - def test_patch_me_updates_through_bridge(self): + def test_patch_me_never_deactivates_subscriber_without_deleting_row(self): + # A logged-in user and their anonymous-subscribe row share one manage_token, + # so turning off email must flip is_active rather than delete the row. + subscriber = NewsletterSubscriber.objects.create( + email=self.user.email, frequency="WEEKLY", is_active=True + ) token = self._token_for(self.user.id) with self._stub_jwks(): resp = self.client.patch( reverse("auth-me"), - data={"primary_city": "carrboro"}, + data={"email_preference": "NEVER"}, content_type="application/json", HTTP_AUTHORIZATION=f"Bearer {token}", ) self.assertEqual(resp.status_code, 200) - self.assertEqual(resp.json()["primary_city"], "carrboro") + subscriber.refresh_from_db() + self.assertFalse(subscriber.is_active) + self.assertEqual(NewsletterSubscriber.objects.count(), 1) def test_unknown_subject_is_rejected(self): token = self._token_for(uuid.uuid4()) diff --git a/backendServer/events/tests/test_digest.py b/backendServer/events/tests/test_digest.py index cb4cc8c..1ee9f05 100644 --- a/backendServer/events/tests/test_digest.py +++ b/backendServer/events/tests/test_digest.py @@ -5,21 +5,28 @@ from django.test import TestCase, tag from django.utils import timezone -from events.models import Event -from events.tasks import fan_out_weekly_digest, send_one_digest +from events.models import Event, NewsletterSubscriber +from events.tasks import fan_out_monthly_digest, fan_out_weekly_digest, send_one_digest -from .factories import make_town, make_user +from .factories import make_town @tag("db") -class WeeklyDigestTaskTests(TestCase): +class DigestTaskTests(TestCase): """Celery runs eagerly via settings.test (CELERY_TASK_ALWAYS_EAGER), and the neon_auth `user` table is built by NeonAuthTestRunner — no per-class setup. + + fan_out_* now resolves recipients from NewsletterSubscriber (the single + source of truth) rather than UserProfile, so subtasks take a resolved + email/tags/manage_token/frequency instead of a profile id. """ - def _make_profile(self, email_preference="WEEKLY", primary_city="carrboro"): - user = make_user("LOCAL", email_preference=email_preference, primary_city=primary_city) - return user.profile + def _make_subscriber(self, frequency="WEEKLY", is_active=True, email=None): + return NewsletterSubscriber.objects.create( + email=email or "reader@example.com", + frequency=frequency, + is_active=is_active, + ) def setUp(self): self.carrboro = make_town("carrboro", "Carrboro") @@ -31,11 +38,11 @@ def setUp(self): description="d", ) - def test_fan_out_queues_one_subtask_per_weekly_profile(self): - self._make_profile(email_preference="WEEKLY") - self._make_profile(email_preference="WEEKLY") - self._make_profile(email_preference="MONTHLY") # excluded - self._make_profile(email_preference="NEVER") # excluded + def test_fan_out_weekly_queues_one_subtask_per_weekly_subscriber(self): + self._make_subscriber("WEEKLY", email="a@example.com") + self._make_subscriber("WEEKLY", email="b@example.com") + self._make_subscriber("MONTHLY", email="c@example.com") # excluded + self._make_subscriber("WEEKLY", is_active=False, email="d@example.com") # excluded with mock.patch("events.tasks.send_one_digest.delay") as delay: count = fan_out_weekly_digest.delay().get() @@ -43,24 +50,32 @@ def test_fan_out_queues_one_subtask_per_weekly_profile(self): self.assertEqual(count, 2) self.assertEqual(delay.call_count, 2) + def test_fan_out_monthly_queues_one_subtask_per_monthly_subscriber(self): + self._make_subscriber("MONTHLY", email="a@example.com") + self._make_subscriber("WEEKLY", email="b@example.com") # excluded + + with mock.patch("events.tasks.send_one_digest.delay") as delay: + count = fan_out_monthly_digest.delay().get() + + self.assertEqual(count, 1) + delay.assert_called_once_with("a@example.com", [], mock.ANY, "MONTHLY") + def test_send_one_digest_sends_for_matching_events(self): - profile = self._make_profile() with mock.patch("events.tasks.send_email", return_value=True) as send: - send_one_digest.delay(profile.id) + send_one_digest.delay("reader@example.com", [], "some-token", "WEEKLY") send.assert_called_once() - self.assertEqual(send.call_args.args[0], profile.user.email) + self.assertEqual(send.call_args.args[0], "reader@example.com") - def test_send_one_digest_skips_unknown_town(self): - profile = self._make_profile(primary_city="nowhere") + def test_send_one_digest_skips_when_no_events_match_tags(self): + # The one seeded event has no tags, so a tag-filtered recipient gets nothing. with mock.patch("events.tasks.send_email") as send: - send_one_digest.delay(profile.id) + send_one_digest.delay("reader@example.com", ["music"], "some-token", "WEEKLY") send.assert_not_called() def test_send_one_digest_retries_on_brevo_failure(self): - profile = self._make_profile() with mock.patch("events.tasks.send_email", return_value=False) as send: # In eager mode self.retry() raises Retry; confirms a Brevo failure # requests a retry of this one subtask. with self.assertRaises(Retry): - send_one_digest.delay(profile.id) + send_one_digest.delay("reader@example.com", [], "some-token", "WEEKLY") send.assert_called_once() diff --git a/backendServer/events/tests/test_digest_db.py b/backendServer/events/tests/test_digest_db.py new file mode 100644 index 0000000..c00a845 --- /dev/null +++ b/backendServer/events/tests/test_digest_db.py @@ -0,0 +1,119 @@ +from datetime import timedelta +from unittest import mock + +from django.test import TestCase, tag +from django.utils import timezone +from django_celery_beat.models import PeriodicTask + +from events.email_service import _build_recipients, send_digest +from events.models import Event, NewsletterSubscriber, Tag + +from .factories import make_town, make_user + + +@tag("db") +class MonthlyBeatScheduleSeedTests(TestCase): + """Guards the beat schedule seeded by migration 0020 — mirrors the weekly + guard in test_beat_schedule_db.py so a future migration can't silently + drop or disable the monthly digest.""" + + def test_monthly_digest_schedule_seeded(self): + pt = PeriodicTask.objects.get(name="monthly-digest-first") + self.assertEqual(pt.task, "events.tasks.fan_out_monthly_digest") + self.assertTrue(pt.enabled) + self.assertEqual(pt.crontab.minute, "0") + self.assertEqual(pt.crontab.hour, "18") + self.assertEqual(pt.crontab.day_of_month, "1") + self.assertEqual(str(pt.crontab.timezone), "America/New_York") + + +@tag("db") +class BuildRecipientsTests(TestCase): + """_build_recipients is the single resolver behind both the Celery fan-out + and the send_digest command — NewsletterSubscriber is the source of truth, + so every recipient it produces must carry a manage_token, and an anonymous + subscriber (no matching UserProfile) must still come through with an empty + tag filter (i.e. gets every event).""" + + def test_every_recipient_carries_a_manage_token(self): + NewsletterSubscriber.objects.create(email="a@example.com", frequency="WEEKLY") + NewsletterSubscriber.objects.create(email="b@example.com", frequency="WEEKLY") + + recipients = _build_recipients("WEEKLY") + + self.assertEqual(len(recipients), 2) + for r in recipients: + self.assertIsNotNone(r["manage_token"]) + + def test_anonymous_subscriber_included_with_no_tag_filter(self): + sub = NewsletterSubscriber.objects.create(email="anon@example.com", frequency="WEEKLY") + + recipients = _build_recipients("WEEKLY") + + self.assertEqual(len(recipients), 1) + self.assertEqual(recipients[0]["email"], "anon@example.com") + self.assertEqual(recipients[0]["tags"], set()) + self.assertEqual(recipients[0]["manage_token"], sub.manage_token) + + def test_subscriber_matching_a_profile_is_tag_filtered(self): + music = Tag.objects.create(name="music") + user = make_user("LOCAL", email="matched@example.com") + user.profile.tags.add(music) + NewsletterSubscriber.objects.create(email="matched@example.com", frequency="WEEKLY") + + recipients = _build_recipients("WEEKLY") + + self.assertEqual(len(recipients), 1) + self.assertEqual(recipients[0]["tags"], {"music"}) + + def test_inactive_subscriber_excluded(self): + NewsletterSubscriber.objects.create( + email="gone@example.com", frequency="WEEKLY", is_active=False + ) + + self.assertEqual(_build_recipients("WEEKLY"), []) + + def test_wrong_frequency_excluded(self): + NewsletterSubscriber.objects.create(email="monthly@example.com", frequency="MONTHLY") + + self.assertEqual(_build_recipients("WEEKLY"), []) + + +@tag("db") +class SendDigestManageLinkTests(TestCase): + """The rendered digest email must include a working manage/unsubscribe + link keyed by the recipient's own manage_token, and an anonymous + subscriber (no account) must still receive a digest.""" + + def setUp(self): + town = make_town("carrboro", "Carrboro") + Event.objects.create( + title="Some Event", + town=town, + date=timezone.now() + timedelta(days=2), + venue="V", + description="d", + ) + + def test_anonymous_subscriber_receives_digest_with_manage_link(self): + sub = NewsletterSubscriber.objects.create(email="anon@example.com", frequency="WEEKLY") + + captured = {} + + def fake_send_email(to, subject, html, text=None): + captured["to"] = to + captured["html"] = html + return True + + with mock.patch("events.email_service.send_email", side_effect=fake_send_email): + result = send_digest("WEEKLY") + + self.assertEqual(result, {"sent": 1, "failed": 0}) + self.assertEqual(captured["to"], "anon@example.com") + self.assertIn(str(sub.manage_token), captured["html"]) + self.assertIn("/newsletter/manage?token=", captured["html"]) + self.assertIn("Manage preferences / Unsubscribe", captured["html"]) + + def test_no_subscribers_sends_nothing(self): + result = send_digest("WEEKLY") + self.assertEqual(result, {"sent": 0, "failed": 0}) diff --git a/backendServer/events/tests/test_newsletter_db.py b/backendServer/events/tests/test_newsletter_db.py index 51ca41d..7f8791f 100644 --- a/backendServer/events/tests/test_newsletter_db.py +++ b/backendServer/events/tests/test_newsletter_db.py @@ -1,3 +1,5 @@ +from unittest import mock + from django.test import TestCase, tag from django.urls import reverse @@ -6,6 +8,12 @@ @tag("db") class NewsletterSubscribeTests(TestCase): + def setUp(self): + # subscribe() now sends a welcome email; keep these tests off the network. + patcher = mock.patch("events.views.send_newsletter_welcome", return_value=True) + patcher.start() + self.addCleanup(patcher.stop) + def test_subscribe_creates_subscriber(self): resp = self.client.post( reverse("subscribe"), @@ -44,3 +52,88 @@ def test_missing_email_is_400(self): ) self.assertEqual(resp.status_code, 400) self.assertEqual(resp.json()["error"], "email is required") + + def test_subscribe_attempts_welcome_email_with_manage_link(self): + with mock.patch("events.views.send_newsletter_welcome") as welcome: + resp = self.client.post( + reverse("subscribe"), + {"email": "reader@example.com", "frequency": "WEEKLY"}, + content_type="application/json", + ) + self.assertEqual(resp.status_code, 201) + subscriber = NewsletterSubscriber.objects.get(email="reader@example.com") + welcome.assert_called_once_with(subscriber.email, subscriber.manage_token) + + def test_subscribe_survives_welcome_send_failure(self): + with mock.patch("events.views.send_newsletter_welcome", return_value=False): + resp = self.client.post( + reverse("subscribe"), + {"email": "reader@example.com", "frequency": "WEEKLY"}, + content_type="application/json", + ) + self.assertEqual(resp.status_code, 201) + + +@tag("db") +class NewsletterManageTests(TestCase): + def setUp(self): + self.subscriber = NewsletterSubscriber.objects.create( + email="reader@example.com", frequency="WEEKLY", is_active=True + ) + + def test_get_returns_prefs_for_valid_token(self): + resp = self.client.get( + reverse("newsletter-manage"), {"token": str(self.subscriber.manage_token)} + ) + self.assertEqual(resp.status_code, 200) + body = resp.json() + self.assertEqual(body["email"], "reader@example.com") + self.assertEqual(body["frequency"], "WEEKLY") + self.assertTrue(body["is_active"]) + + def test_patch_changes_frequency(self): + resp = self.client.patch( + f"{reverse('newsletter-manage')}?token={self.subscriber.manage_token}", + {"frequency": "MONTHLY"}, + content_type="application/json", + ) + self.assertEqual(resp.status_code, 200) + self.assertEqual(resp.json()["frequency"], "MONTHLY") + self.subscriber.refresh_from_db() + self.assertEqual(self.subscriber.frequency, "MONTHLY") + self.assertTrue(self.subscriber.is_active) + + def test_patch_never_deactivates_without_deleting_row(self): + resp = self.client.patch( + f"{reverse('newsletter-manage')}?token={self.subscriber.manage_token}", + {"frequency": "NEVER"}, + content_type="application/json", + ) + self.assertEqual(resp.status_code, 200) + self.assertFalse(resp.json()["is_active"]) + self.subscriber.refresh_from_db() + self.assertFalse(self.subscriber.is_active) + # Frequency is left as-is so a later re-subscribe can restore it. + self.assertEqual(self.subscriber.frequency, "WEEKLY") + self.assertEqual(NewsletterSubscriber.objects.count(), 1) + + def test_unknown_token_is_404(self): + unknown_token = "00000000-0000-0000-0000-000000000000" + resp = self.client.get(reverse("newsletter-manage"), {"token": unknown_token}) + self.assertEqual(resp.status_code, 404) + + def test_malformed_token_is_404(self): + resp = self.client.get(reverse("newsletter-manage"), {"token": "not-a-uuid"}) + self.assertEqual(resp.status_code, 404) + + def test_blank_token_is_404(self): + resp = self.client.get(reverse("newsletter-manage")) + self.assertEqual(resp.status_code, 404) + + def test_bad_frequency_is_400(self): + resp = self.client.patch( + f"{reverse('newsletter-manage')}?token={self.subscriber.manage_token}", + {"frequency": "DAILY"}, + content_type="application/json", + ) + self.assertEqual(resp.status_code, 400) diff --git a/backendServer/events/views.py b/backendServer/events/views.py index bf5dccc..c7f86e0 100644 --- a/backendServer/events/views.py +++ b/backendServer/events/views.py @@ -1,6 +1,7 @@ from datetime import timedelta from django.core.cache import cache +from django.core.exceptions import ValidationError from django.shortcuts import get_object_or_404 from django.utils import timezone from django.utils.dateparse import parse_datetime @@ -14,8 +15,8 @@ from ingestion.models import StagedEvent from . import cache as events_cache +from .email_service import send_newsletter_welcome from .models import ( - BetterAuthAccount, BusinessProfile, Category, Event, @@ -28,17 +29,6 @@ PAGE_SIZE = 30 -def user_has_password(user_id): - """True when the user has a Better Auth credential account with a password set. - - Lazy (passwordless) accounts are created via the internal adapter with no - credential row, so this distinguishes secured accounts from unsecured ones. - """ - return BetterAuthAccount.objects.filter( - user_id=str(user_id), provider_id="credential", password__isnull=False - ).exists() - - class EventsPagination(PageNumberPagination): page_size = PAGE_SIZE page_size_query_param = "page_size" @@ -395,12 +385,52 @@ def subscribe(request): defaults={"frequency": frequency, "is_active": True}, ) + send_newsletter_welcome(subscriber.email, subscriber.manage_token) + return Response( {"email": subscriber.email, "frequency": subscriber.frequency}, status=status.HTTP_201_CREATED if created else status.HTTP_200_OK, ) +@api_view(["GET", "PATCH"]) +def newsletter_manage(request): + token = request.query_params.get("token") + + subscriber = None + if token: + try: + subscriber = NewsletterSubscriber.objects.filter(manage_token=token).first() + except ValidationError: + subscriber = None + + if subscriber is None: + return Response({"error": "Unknown or invalid token."}, status=status.HTTP_404_NOT_FOUND) + + if request.method == "PATCH": + frequency = (request.data.get("frequency") or "").upper() + if frequency not in ("WEEKLY", "MONTHLY", "NEVER"): + return Response( + {"error": "frequency must be WEEKLY, MONTHLY, or NEVER"}, + status=status.HTTP_400_BAD_REQUEST, + ) + + if frequency == "NEVER": + subscriber.is_active = False + else: + subscriber.frequency = frequency + subscriber.is_active = True + subscriber.save() + + return Response( + { + "email": subscriber.email, + "frequency": subscriber.frequency, + "is_active": subscriber.is_active, + } + ) + + @api_view(["GET"]) @authentication_classes([BearerTokenAuthentication]) @permission_classes([IsAuthenticated]) @@ -417,7 +447,6 @@ def getMyProfile(request): "primary_city": profile.primary_city, "address": profile.address, "email_preference": profile.email_preference, - "has_password": user_has_password(profile.user.id), } ) @@ -487,7 +516,7 @@ def me(request): # noqa: C901 # multi-field profile PATCH; complexity is inher defaults={"frequency": profile.email_preference, "is_active": True}, ) else: - NewsletterSubscriber.objects.filter(email=email).delete() + NewsletterSubscriber.objects.filter(email=email).update(is_active=False) return Response( { @@ -499,6 +528,5 @@ def me(request): # noqa: C901 # multi-field profile PATCH; complexity is inher "address": profile.address, "email_preference": profile.email_preference, "tags": [t.name for t in profile.tags.all()], - "has_password": user_has_password(profile.user.id), } ) diff --git a/backendServer/templates/email/digest.html b/backendServer/templates/email/digest.html index f311a11..4c8d8f9 100644 --- a/backendServer/templates/email/digest.html +++ b/backendServer/templates/email/digest.html @@ -152,7 +152,11 @@

The Commons

thecommons.town
You're receiving this because you subscribed to the {{ frequency|lower }} digest.
- To unsubscribe, reply to this email with “unsubscribe” in the subject. + {% if manage_url %} + Manage preferences / Unsubscribe + {% else %} + To unsubscribe, reply to this email with “unsubscribe” in the subject. + {% endif %}
diff --git a/backendServer/templates/email/weekly_digest.html b/backendServer/templates/email/weekly_digest.html index 0ac9338..0b2bf6c 100644 --- a/backendServer/templates/email/weekly_digest.html +++ b/backendServer/templates/email/weekly_digest.html @@ -158,7 +158,11 @@

The Commons — Your Weekly Digest

thecommons.town
You're receiving this because you subscribed to the weekly digest.
- To unsubscribe, reply to this email with “unsubscribe” in the subject. + {% if manage_url %} + Manage preferences / Unsubscribe + {% else %} + To unsubscribe, reply to this email with “unsubscribe” in the subject. + {% endif %}
diff --git a/docs/redis-celery-handoff.md b/docs/redis-celery-handoff.md index f4f65c6..fe6f18c 100644 --- a/docs/redis-celery-handoff.md +++ b/docs/redis-celery-handoff.md @@ -61,6 +61,7 @@ the code and reproduces on a fresh DB. Current entries: |------|------|----------------------| | Ingestion pipeline | `ingestion.tasks.run_ingestion_pipeline` | 04:00 daily, `America/New_York` (`ingestion/migrations/0007_seed_ingest_beat.py`) | | Weekly digest fan-out | `events.tasks.fan_out_weekly_digest` | Sundays 18:00, `America/New_York` (`events/migrations/0015_seed_digest_beat.py`) | +| Monthly digest fan-out | `events.tasks.fan_out_monthly_digest` | 1st of month 18:00, `America/New_York` (`events/migrations/0020_seed_monthly_digest_beat.py`, task name `monthly-digest-first`) | The `CrontabSchedule.timezone` is set to `America/New_York` (not UTC) so beat tracks US-Eastern DST exactly like the OS cron these replaced. diff --git a/docs/suite-38-passwordless-rollover.md b/docs/suite-38-passwordless-rollover.md new file mode 100644 index 0000000..a7b1ae5 --- /dev/null +++ b/docs/suite-38-passwordless-rollover.md @@ -0,0 +1,231 @@ +# Suite 38.A4 — Rolling over passwordless accounts + +## Background + +Before suite 38 (38.A1/38.A2 removed the passwordless "lazy-auth" sign-in), +a user could get a `neon_auth.user` row with no password at all — sign-in +was just "enter your email." `theCommonsWeb/src/app/(portal)/signin/SignInForm.tsx` +now requires email **and** password (confirmed by reading the file — there is +no passwordless code path left). Anyone whose account never got a +credential-provider password is now locked out: there is no password for +them to type. + +**Decision (2026-07-31): send each affected user a password-reset email.** + +**How we got here (for context on why there's no shortcut).** The working +tree at the time of this writing has an in-progress, not-yet-committed diff +removing the passwordless flow entirely: `theCommonsWeb/src/lib/lazy-auth-plugin.ts` +(the `/enter` endpoint that logged a user in by email alone and set a +session cookie) is deleted, along with the old +`theCommonsWeb/src/app/(portal)/set-password/*` page and +`theCommonsWeb/src/app/api/auth/set-password/route.ts`. That old +`/set-password` page was the previous self-serve fix for exactly this +problem — but it only worked because a passwordless user already had a +valid session (from `/enter`) and could add a password to it. With +`lazy-auth-plugin.ts` gone, there is no way left to get a session without a +password, so that door is closed too. This is why a token-based, +no-session-required reset link (Better Auth's `sendResetPassword` flow) is +the only remaining path — it's the one mechanism that doesn't require the +user to already be signed in. + +## Status: blocked on frontend/auth wiring — do not `--send` yet + +This doc and the management command it describes are **prepared, not run**. +Before any real send, the reset link the email points to must actually work. +As of this writing it does not. Read "Prerequisite wiring" below before doing +anything else with this runbook. + +## 1. Identify affected users + +Affected = a `neon_auth.user` row with **no** `neon_auth.account` row where +`providerId = 'credential'` AND `password IS NOT NULL`. + +Dry run (identifies and prints only — sends nothing): + +```bash +cd backendServer +DJANGO_SETTINGS_MODULE=backend.settings.test uv run python manage.py rollover_passwordless_accounts +# or against a real DB: +DJANGO_SETTINGS_MODULE=backend.settings.prod uv run python manage.py rollover_passwordless_accounts +``` + +This is implemented in +`backendServer/events/management/commands/rollover_passwordless_accounts.py`. + +**Why raw SQL, not the ORM.** `events/models.py` declares +`BetterAuthAccount.user_id` as a Django `TextField`, but the live +`neon_auth.account."userId"` column is actually `uuid` — confirmed by +querying `information_schema.columns` against the test DB: + +``` +('neon_auth', 'account', 'userId', 'uuid') +('neon_auth', 'user', 'id', 'uuid') +``` + +An ORM anti-join (`BetterAuthUser.objects.exclude(id__in=BetterAuthAccount.objects.filter(...).values("user_id"))`) +binds the subquery's `userId` values as text (per the model's field type) +against `u.id` (uuid); Postgres has no `uuid = text` operator for that shape +and raises `operator does not exist: uuid = text` — reproduced while +building this command, before the fix. The command instead runs a plain +raw-SQL `LEFT JOIN ... WHERE a.id IS NULL` anti-join with no cast, since both +columns really are `uuid`. **This is a real model/schema drift** worth a +follow-up: `BetterAuthAccount.user_id` should probably be declared to match +the actual `uuid` column type (though since `managed = False`, nothing +enforces it and ORM reads still work — Python just gets the UUID's string +form back — the drift only bites when you build a cross-model join, as here). +Neither mirror model is migrated by Django in any case (`managed = False` +on all `neon_auth` mirrors — never run `makemigrations`/`migrate` against +them). + +Verified working: a dry run against the shared test DB found 2 real +leftover accounts matching the pattern (`diag4+…@example.com`, +`s37e2e-drive1@example.com`), and `--help` / `--limit` both work as expected. + +## 2. Prerequisite wiring — blocking, must land before `--send` + +The ticket's fallback plan was "email users a link to the existing +`/forgot-password` page where they self-serve." **Investigation shows this +does not currently work, for two independent reasons:** + +### 2a. Better Auth's native reset-password endpoint is disabled + +`theCommonsWeb/src/lib/auth.ts` configures: + +```ts +emailAndPassword: { enabled: true, autoSignIn: true }, +``` + +No `sendResetPassword` callback. Better Auth's own source +(`better-auth/dist/api/routes/password.mjs`, `/request-password-reset` +endpoint) checks this explicitly, **before** creating any reset token: + +```js +if (!ctx.context.options.emailAndPassword?.sendResetPassword) { + ctx.context.logger.error("Reset password isn't enabled..."); + throw APIError.from("BAD_REQUEST", { code: "RESET_PASSWORD_DISABLED" }); +} +``` + +So calling `authClient.requestPasswordReset({ email })` today returns a 400 +`RESET_PASSWORD_DISABLED` — it doesn't even generate a token, regardless of +what frontend page calls it. + +**Fix required:** add an `emailAndPassword.sendResetPassword` function to +`auth.ts` that sends the email Better Auth generates the token/url for (via +Brevo, or whatever transactional path the Next side uses). This also implies +adding `resetPasswordTokenExpiresIn` if the default 1-hour window isn't +wanted. + +### 2b. There is no frontend page to consume a reset token, and `/forgot-password` is a stale stub + +- No `/reset-password` or `/reset-password/[token]` page exists anywhere + under `theCommonsWeb/src/app` (searched — zero matches). Better Auth's + `/reset-password/:token` callback redirects to a `callbackURL` you supply + as `redirectTo`; there is currently nothing at that URL to receive the + token and call `authClient.resetPassword({ token, newPassword })`. +- The existing `/forgot-password` page + (`theCommonsWeb/src/app/(portal)/forgot-password/ForgotPasswordForm.tsx`) + is **not** a working self-serve reset flow. It's a static informational + stub left over from the passwordless era. Its copy literally says: + + > "The Commons doesn't send password reset emails. In most cases you don't + > need one — just enter your email on the Sign In page and continue + > without a password." + + That's now false (passwordless sign-in is gone), and the form's + `handleSubmit` just flips a `submitted` boolean to show a canned message — + it never calls any Better Auth API (`authClient.requestPasswordReset` or + anything else). Linking rolled-over users here today would land them on a + page that tells them to do something that no longer works. + +**Fix required:** +- Rewrite `ForgotPasswordForm.tsx` to actually call + `authClient.requestPasswordReset({ email, redirectTo: '/reset-password' })` + and show a real "check your email" state. +- Add a `/reset-password/[token]` (or matching `redirectTo`) page that reads + the token and calls `authClient.resetPassword({ token, newPassword })`. + +**None of the above is in scope for 38.A4** (scope was: management command + +this doc, no edits to `auth.ts` or frontend routes). It is called out here +as the blocking prerequisite. Track it as a follow-up ticket before running +`--send` for real. + +## 3. How the batch send works (once 2a/2b are wired and verified) + +```bash +cd backendServer + +# Dry run — always do this first, re-check the printed email list +DJANGO_SETTINGS_MODULE=backend.settings.prod uv run python manage.py rollover_passwordless_accounts + +# Canary: send to just the first affected account, confirm it lands and the +# link works end-to-end (see step 5) before doing everyone +DJANGO_SETTINGS_MODULE=backend.settings.prod uv run python manage.py rollover_passwordless_accounts --send --limit 1 + +# Full send +DJANGO_SETTINGS_MODULE=backend.settings.prod uv run python manage.py rollover_passwordless_accounts --send +``` + +The command sends via `events.email_service.send_email` (Brevo +transactional email — same primitive the digest emails use). It is +idempotent to re-run: if a user sets a password after being emailed, they +will have a `neon_auth.account` row with `providerId='credential'` and a +non-null password and will no longer show up in the affected-user query, so +re-running the dry run naturally shrinks the list and re-sends will not +re-email people who already rolled over. + +## 4. Email copy + +Subject: `Action needed: set a password for your Commons account` + +Body: + +> Hi {name}, +> +> We've changed how sign-in works on The Commons. Your account was created +> before we required a password, so the old "just enter your email" sign-in +> no longer works for you. +> +> [Set your password]({SITE_URL}/forgot-password) — it only takes a minute. +> +> If you don't recognize this account, you can safely ignore this email. +> +> — The Commons + +`SITE_URL` defaults to `https://www.thecommons.town` (same env var and +default `email_service.py` already uses for the newsletter manage link). +The link target (`/forgot-password`) is the page named in section 2b — do +not send until that page actually drives a working reset, or update this +copy/command to point at wherever the fixed flow lives. + +## 5. Verifying a user can sign in afterward + +Once 2a/2b are wired: + +1. Pick one affected email from the dry-run list (or use the `--limit 1` + canary send). +2. Confirm the email arrives (Brevo dashboard or the inbox) and the link + works: clicking it should reach a real "set your password" form, not the + stale copy described in 2b. +3. Submit a new password there. +4. Confirm a `neon_auth.account` row now exists for that user with + `providerId='credential'` and a non-null `password`: + ```sql + SELECT a.* FROM neon_auth."account" a + JOIN neon_auth."user" u ON u.id = a."userId" + WHERE u.email = '' AND a."providerId" = 'credential'; + ``` +5. Go to `/signin`, enter that email + the new password, confirm sign-in + succeeds and redirects normally (`SignInForm.tsx`'s `login()` / + `resolveRedirect` path). +6. Re-run the dry-run command — that user should have dropped out of the + affected list. + +## Repo stance reminders + +- `neon_auth` mirror models (`BetterAuthUser`, `BetterAuthAccount`, etc. in + `backendServer/events/models.py`) are `managed = False`. Never run + `makemigrations`/`migrate` against them; this command only reads + (`SELECT`, no writes) from that schema. +- No email verification for MVP — this rollover doesn't add any; it's + strictly about restoring the ability to sign in with a password. diff --git a/notion-sync/STATE.md b/notion-sync/STATE.md index 43df059..9ef3f4d 100644 --- a/notion-sync/STATE.md +++ b/notion-sync/STATE.md @@ -6,7 +6,7 @@ The ledger mirrors what *should* be on the Notion board so the desktop app can r --- -**Next suite number:** `38` +**Next suite number:** `39` ## Suite ledger @@ -33,6 +33,7 @@ per-ticket status lives on each ticket subpage (see OUTBOX preamble). | 35 | Prod scheduler outage (snap-uv user-slice teardown) + monitor correctness | In Prod | 35.1–35.11, 35.13, 35.14 (35.12 merged into 35.8) | _(pending)_ | | 36 | Ingestion funnel dead-ends (out-of-coverage limbo, town-less events, missed sends, beat bookkeeping) | Needs QA | 36.1–36.4, 36.6–36.7 (36.5 closed won't-fix; 36.8 investigation → benign, no code) | _(pending)_ | | 37 | Central auth reintegration — standalone portal + fix the live JWT bridge | Needs QA | 37.1–37.8, 37.10, 37.11 built+Needs QA (dev E2E 6/6); 37.9 Needs QA (prod cutover executed 2026-07-30, live JWT bridge verified); completes 29 + unbuilt half of 30 | _(pending)_ | +| 38 | Password-required accounts + decoupled newsletter | Open | 38.A1–38.A4, 38.B1–38.B4, 38.D1 (planned, not built) | _(pending)_ |