feat: PHIE Phase 1 — Personal Health Intelligence Engine (backend)

docs/architecture.md

@@ -108,6 +108,8 @@ src/
 | `conversations` | Chat conversation sessions (Phase 6A) | PK: UUID, FK: user_id |
 | `messages` | Individual chat messages (Phase 6A) | role CHECK: user/assistant/tool, JSONB tool_calls |
 | `action_items` | Persistent tracked action items from weekly reports (F3) | FK: weekly_reports(id) CASCADE; status CHECK with 7 states; 3 indexes |
+| `correlations` | PHIE: discovered Pearson correlations across nutrition/training/biometric data | Unique: `(user_id, factor_metric, factor_condition, outcome_metric)`; CHECK on `confidence_level`, `status`, `category`; `first_detected_at` preserved across re-runs |
+| `projections` | PHIE: 30-day trajectory projections with OLS confidence bands | Unique: `(user_id, metric, projection_date)`; CHECK on `method` |
 
 ### EAV Pattern (measurements table)
 
@@ -176,6 +178,8 @@ POST /api/reports/generate
         │
         ├── emit('completed') → completeReport()
         │   └── Report saved with full content
+        │   └── runCorrelationAnalysis() + runTrajectoryProjections()
+        │       └── non-blocking; failures logged under [intelligence]
         │
         └── on error: emit('failed') → updateReportStatus()
 
@@ -215,6 +219,8 @@ POST /api/reports/generate
 | GET | `/api/action-items/summary` | X-API-Key | F3 |
 | GET | `/api/action-items/:id` | X-API-Key | F3 |
 | PATCH | `/api/action-items/:id/status` | X-API-Key | F3 |
+| GET | `/api/correlations` | None | PHIE Phase 1 |
+| GET | `/api/projections/:metric` | None | PHIE Phase 1 |
 
 ## Authentication
 

docs/product-capabilities.md

@@ -454,6 +454,128 @@ The date range picker on other pages is irrelevant to report generation.
 
 ---
 
+## 4. Personal Health Intelligence Engine (PHIE Phase 1 — Backend)
+
+Backend-only intelligence layer that discovers Pearson correlations across
+nutrition/training/biometric data, projects metric trajectories 30 days
+forward with OLS confidence bands, and exposes results through REST +
+AI chat tools. No dedicated UI in Phase 1 — surfaced via the existing
+chat experience and consumed by the report generator.
+
+| ID | Use Case | Status |
+|----|----------|--------|
+| UC-INT-01 | Automatic correlation discovery across user health data | Implemented |
+| UC-INT-02 | 30-day trajectory projection with confidence bands | Implemented |
+| UC-INT-03 | REST API to list and filter correlations | Implemented |
+| UC-INT-04 | REST API to fetch metric projections | Implemented |
+| UC-INT-05 | AI chat tools for pattern discovery and what-if simulation | Implemented |
+
+### UC-INT-01: Automatic correlation discovery
+
+**As the system,** after a weekly report is generated, I want to run a
+correlation analysis across the user's nutrition, training, and biometric
+history, **so that** the chat and future reports can surface personalised
+patterns the user didn't explicitly ask about.
+
+**Behavior:**
+- Runs automatically at the end of both the sync (`?sync=true`) and async
+  report generation paths, inside a non-blocking `try/catch` — failures
+  are logged under `[intelligence]` and do not affect the returned report
+- Loads up to 90 days of daily averages for a fixed set of candidate
+  factor→outcome metric pairs (e.g., protein_g → muscle_mass_kg)
+- Skips pairs with fewer than `MIN_DATA_POINTS` (14) aligned dates
+- Computes Pearson r + p-value, classifies confidence (high / moderate /
+  suggestive) per `(|r|, n, p)` thresholds, persists via
+  `ON CONFLICT DO UPDATE` keyed on `(user_id, factor_metric, factor_condition, outcome_metric)`
+- `first_detected_at` is set only on the initial INSERT so historical
+  detection time is preserved across re-runs
+- Correlations not re-confirmed in a run are marked `weakening`
+
+**Test Coverage:** `packages/backend/src/services/intelligence/__tests__/correlation-engine.test.ts`
+(14-day positive case, <MIN_DATA_POINTS skip, zero-data graceful path)
+
+### UC-INT-02: 30-day trajectory projection
+
+**As the system,** I want to project each biometric metric forward 30
+days with confidence bands, **so that** the AI can answer "where is my
+weight heading?" questions and the user sees honest uncertainty.
+
+**Behavior:**
+- Queries distinct metrics for the user, intersects with the default
+  biometric set (`weight_kg`, `body_fat_pct`, `resting_hr`, sleep, steps,
+  etc.); returns early if no matching metrics exist
+- For each metric with ≥`MIN_DATA_POINTS` (7) daily values, fits a
+  linear regression via least squares and projects 30 days forward
+- Confidence bands use the proper OLS prediction interval formula
+  `σ_res · √(1 + 1/n + (x* − mean_x)² / Σ(xᵢ − mean_x)²)` — not the
+  earlier `σ · √d` heuristic. Bands widen with distance from the
+  observed mean
+- Persists to `projections` table with `method = 'linear_regression'`
+
+**Test Coverage:** `packages/backend/src/services/intelligence/__tests__/trajectory-projector.test.ts`
+(30-day projection with widening bands, min-data-points skip,
+empty-metric early return)
+
+### UC-INT-03: List and filter correlations
+
+**As a** consumer (chat, future UI), **I want to** query stored
+correlations by category, confidence level, or strength, **so that** I
+can show relevant patterns without fetching everything.
+
+**Behavior:**
+- `GET /api/correlations` — unauth'd (consistent with other read
+  endpoints), default user
+- Query filters: `category`, `confidenceLevel`, `status`, `metric`
+  (matches factor OR outcome), `minConfidence` (|r| ≥ threshold)
+- `?top=N` returns top-N by absolute coefficient, capped at 100 to
+  prevent DoS
+
+**Test Coverage:** `packages/backend/src/routes/__tests__/intelligence.test.ts`
+(15 tests covering each filter + `top` boundary cases)
+
+### UC-INT-04: Fetch trajectory projections for a metric
+
+**As a** consumer, **I want to** fetch projections for a named metric,
+**so that** I can render a forward-looking trajectory.
+
+**Behavior:**
+- `GET /api/projections/:metric`
+- Metric name bounded to 100 chars (400 on oversize); stored values
+  returned as-is with confidence bands
+
+**Test Coverage:** Same route test file (oversized-metric 400, happy
+path, user-not-found empty-list case)
+
+### UC-INT-05: AI chat intelligence tools
+
+**As a** user, **I want to** ask the AI "what patterns have you found?"
+or "what happens to my weight if I hit 2200 kcal every day?", **so that**
+I get personalised answers grounded in my own data.
+
+**Behavior:**
+- `query_correlations` — filter by `metric`, `category`, `minConfidence`;
+  filter values are length-bounded to defend against prompt-injected
+  giant strings
+- `predict_trajectory` — delegates to `getProjections` for a named metric
+- `simulate_change` — returns relevant stored correlations for a given
+  factor/delta pair (does NOT re-invoke the LLM, so no prompt-injection
+  re-entry path)
+- `parseDate` used by these tools throws a generic `'Invalid date'` to
+  avoid leaking raw user-supplied (potentially injected) values to logs
+
+**E2E Coverage:** End-to-end value emerges once correlations table is
+populated by the post-report intelligence run — verified indirectly by
+UC-INT-01 test coverage plus the existing chat route tests asserting
+tool dispatch.
+
+**Limitations (Phase 1):**
+- Backend-only — no dedicated correlations/projections UI
+- Trigger is post-report only; no scheduled cron yet. Users who don't
+  generate reports will never see correlations discovered. Follow-up
+  ticket should add a nightly scheduled run.
+
+---
+
 ## 5. Nutrition
 
 | ID | Use Case | Status |

packages/backend/src/app.ts

@@ -12,6 +12,7 @@ import { chatRoutes } from './routes/chat.js';
 import { wsChatRoutes } from './routes/ws-chat.js';
 import { uploadRoutes } from './routes/upload.js';
 import { actionItemRoutes } from './routes/action-items.js';
+import { intelligenceRoutes } from './routes/intelligence.js';
 import multipart from '@fastify/multipart';
 import rateLimit from '@fastify/rate-limit';
 import websocket from '@fastify/websocket';
@@ -52,6 +53,7 @@ export async function buildApp(env: EnvConfig) {
   await app.register(wsChatRoutes, { env });
   await app.register(uploadRoutes, { env });
   await app.register(actionItemRoutes, { env });
+  await app.register(intelligenceRoutes, { env });
 
   return app;
 }

packages/backend/src/db/migrations/010_intelligence.sql

@@ -0,0 +1,44 @@
+CREATE TABLE IF NOT EXISTS correlations (
+  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
+  user_id TEXT NOT NULL DEFAULT 'default',
+  factor_metric TEXT NOT NULL,
+  factor_condition TEXT NOT NULL,
+  factor_label TEXT NOT NULL,
+  outcome_metric TEXT NOT NULL,
+  outcome_effect TEXT NOT NULL,
+  outcome_label TEXT NOT NULL,
+  correlation_coefficient DOUBLE PRECISION NOT NULL,
+  confidence_level TEXT NOT NULL CHECK (confidence_level IN ('high', 'moderate', 'suggestive')),
+  data_points INTEGER NOT NULL,
+  p_value DOUBLE PRECISION,
+  first_detected_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+  last_confirmed_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+  times_confirmed INTEGER NOT NULL DEFAULT 1,
+  status TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active', 'weakening', 'disproven')),
+  summary TEXT NOT NULL,
+  category TEXT NOT NULL CHECK (category IN ('nutrition', 'training', 'recovery', 'cross-domain')),
+  created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+  updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+  UNIQUE (user_id, factor_metric, factor_condition, outcome_metric)
+);
+
+CREATE INDEX IF NOT EXISTS idx_correlations_user_status ON correlations (user_id, status);
+CREATE INDEX IF NOT EXISTS idx_correlations_user_category ON correlations (user_id, category);
+CREATE INDEX IF NOT EXISTS idx_correlations_user_factor ON correlations (user_id, factor_metric);
+CREATE INDEX IF NOT EXISTS idx_correlations_user_outcome ON correlations (user_id, outcome_metric);
+
+CREATE TABLE IF NOT EXISTS projections (
+  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
+  user_id TEXT NOT NULL DEFAULT 'default',
+  metric TEXT NOT NULL,
+  projection_date DATE NOT NULL,
+  projected_value DOUBLE PRECISION NOT NULL,
+  confidence_low DOUBLE PRECISION,
+  confidence_high DOUBLE PRECISION,
+  method TEXT NOT NULL CHECK (method IN ('linear_regression', 'rolling_average', 'exponential')),
+  data_points INTEGER NOT NULL,
+  generated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+  UNIQUE (user_id, metric, projection_date)
+);
+
+CREATE INDEX IF NOT EXISTS idx_projections_user_metric ON projections (user_id, metric);

packages/backend/src/db/queries/correlations.ts

@@ -0,0 +1,172 @@
+import type pg from 'pg';
+import type {
+  Correlation,
+  ConfidenceLevel,
+  CorrelationCategory,
+  CorrelationStatus,
+} from '@vitals/shared';
+
+function rowToCorrelation(r: Record<string, unknown>): Correlation {
+  return {
+    id: String(r['id']),
+    userId: String(r['user_id']),
+    factorMetric: String(r['factor_metric']),
+    factorCondition: String(r['factor_condition']),
+    factorLabel: String(r['factor_label']),
+    outcomeMetric: String(r['outcome_metric']),
+    outcomeEffect: String(r['outcome_effect']),
+    outcomeLabel: String(r['outcome_label']),
+    correlationCoefficient: Number(r['correlation_coefficient']),
+    confidenceLevel: String(r['confidence_level']) as ConfidenceLevel,
+    dataPoints: Number(r['data_points']),
+    pValue: r['p_value'] != null ? Number(r['p_value']) : null,
+    firstDetectedAt:
+      r['first_detected_at'] instanceof Date
+        ? r['first_detected_at'].toISOString()
+        : String(r['first_detected_at']),
+    lastConfirmedAt:
+      r['last_confirmed_at'] instanceof Date
+        ? r['last_confirmed_at'].toISOString()
+        : String(r['last_confirmed_at']),
+    timesConfirmed: Number(r['times_confirmed']),
+    status: String(r['status']) as CorrelationStatus,
+    summary: String(r['summary']),
+    category: String(r['category']) as CorrelationCategory,
+    createdAt:
+      r['created_at'] instanceof Date ? r['created_at'].toISOString() : String(r['created_at']),
+    updatedAt:
+      r['updated_at'] instanceof Date ? r['updated_at'].toISOString() : String(r['updated_at']),
+  };
+}
+
+export async function upsertCorrelation(
+  pool: pg.Pool,
+  correlation: Omit<Correlation, 'id' | 'createdAt' | 'updatedAt'>,
+): Promise<string> {
+  const { rows } = await pool.query(
+    `INSERT INTO correlations (
+       user_id, factor_metric, factor_condition, factor_label,
+       outcome_metric, outcome_effect, outcome_label,
+       correlation_coefficient, confidence_level, data_points, p_value,
+       first_detected_at, last_confirmed_at, times_confirmed,
+       status, summary, category
+     ) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17)
+     ON CONFLICT (user_id, factor_metric, factor_condition, outcome_metric) DO UPDATE SET
+       factor_label         = EXCLUDED.factor_label,
+       outcome_effect       = EXCLUDED.outcome_effect,
+       outcome_label        = EXCLUDED.outcome_label,
+       correlation_coefficient = EXCLUDED.correlation_coefficient,
+       confidence_level     = EXCLUDED.confidence_level,
+       data_points          = EXCLUDED.data_points,
+       p_value              = EXCLUDED.p_value,
+       last_confirmed_at    = EXCLUDED.last_confirmed_at,
+       times_confirmed      = correlations.times_confirmed + 1,
+       status               = EXCLUDED.status,
+       summary              = EXCLUDED.summary,
+       category             = EXCLUDED.category,
+       updated_at           = now()
+       -- first_detected_at is intentionally excluded: preserve the original detection timestamp
+     RETURNING id`,
+    [
+      correlation.userId,
+      correlation.factorMetric,
+      correlation.factorCondition,
+      correlation.factorLabel,
+      correlation.outcomeMetric,
+      correlation.outcomeEffect,
+      correlation.outcomeLabel,
+      correlation.correlationCoefficient,
+      correlation.confidenceLevel,
+      correlation.dataPoints,
+      correlation.pValue,
+      correlation.firstDetectedAt,
+      correlation.lastConfirmedAt,
+      correlation.timesConfirmed,
+      correlation.status,
+      correlation.summary,
+      correlation.category,
+    ],
+  );
+
+  return String(rows[0]['id']);
+}
+
+export async function listCorrelations(
+  pool: pg.Pool,
+  userId: string,
+  filters?: {
+    category?: CorrelationCategory | string;
+    confidenceLevel?: ConfidenceLevel | string;
+    status?: CorrelationStatus | string;
+    metric?: string;
+    minConfidence?: string;
+  },
+): Promise<Correlation[]> {
+  const conditions: string[] = ['user_id = $1'];
+  const params: unknown[] = [userId];
+  let idx = 2;
+
+  if (filters?.category) {
+    conditions.push(`category = $${idx++}`);
+    params.push(filters.category);
+  }
+  if (filters?.confidenceLevel) {
+    conditions.push(`confidence_level = $${idx++}`);
+    params.push(filters.confidenceLevel);
+  }
+  if (filters?.status) {
+    conditions.push(`status = $${idx++}`);
+    params.push(filters.status);
+  }
+  if (filters?.metric) {
+    conditions.push(`(factor_metric = $${idx} OR outcome_metric = $${idx})`);
+    params.push(filters.metric);
+    idx++;
+  }
+  if (filters?.minConfidence) {
+    conditions.push(`ABS(correlation_coefficient) >= $${idx++}`);
+    params.push(Number(filters.minConfidence));
+  }
+
+  const sql = `
+    SELECT id, user_id, factor_metric, factor_condition, factor_label,
+           outcome_metric, outcome_effect, outcome_label,
+           correlation_coefficient, confidence_level, data_points, p_value,
+           first_detected_at, last_confirmed_at, times_confirmed,
+           status, summary, category, created_at, updated_at
+    FROM correlations
+    WHERE ${conditions.join(' AND ')}
+    ORDER BY ABS(correlation_coefficient) DESC, last_confirmed_at DESC
+  `;
+
+  const { rows } = await pool.query(sql, params);
+  return rows.map(rowToCorrelation);
+}
+
+export async function getTopCorrelations(
+  pool: pg.Pool,
+  userId: string,
+  limit = 10,
+): Promise<Correlation[]> {
+  const { rows } = await pool.query(
+    `SELECT id, user_id, factor_metric, factor_condition, factor_label,
+            outcome_metric, outcome_effect, outcome_label,
+            correlation_coefficient, confidence_level, data_points, p_value,
+            first_detected_at, last_confirmed_at, times_confirmed,
+            status, summary, category, created_at, updated_at
+     FROM correlations
+     WHERE user_id = $1
+     ORDER BY ABS(correlation_coefficient) DESC, times_confirmed DESC
+     LIMIT $2`,
+    [userId, limit],
+  );
+
+  return rows.map(rowToCorrelation);
+}
+
+export async function markWeakening(pool: pg.Pool, id: string): Promise<void> {
+  await pool.query(
+    `UPDATE correlations SET status = 'weakening', updated_at = now() WHERE id = $1`,
+    [id],
+  );
+}