From 621b099ee6f4178bfcab7445e78708c5d4a04a94 Mon Sep 17 00:00:00 2001 From: vadimvlasenko Date: Tue, 23 Jun 2026 14:37:16 +0300 Subject: [PATCH 1/2] feat(analytics): add OTEL analytics source with session-detail report Add `analytics otel --file `: maps a flattened otel-events.jsonl (native Claude Code telemetry) into the existing analytics pipeline so it renders the same HTML report (cost, tokens, tools, timeline, skills, turns, project/branch) from OTEL data. - AnalyticsSource seam (sources/{types,sessions-source,otel-source}) so the CLI is agnostic to where the data comes from; the command is split into a shared runner plus the `otel` subcommand. - otel-loader: parse/filter events, authoritative cost index, per-model rollup, timeline dispatches, synthesized deltas, transcript cwd/branch join. Each agent api_request is attributed to a single dispatch so parallel subagents with overlapping windows do not double-count. - otel-source honors --session/--project/--agent/--branch so filtered runs return the right sessions and reports are never mislabeled. - usage-readers: keep the most complete row when Claude writes progressive JSONL records for one streaming response. - hook: on SessionStart re-entry, preserve accumulated startTime/activeDurationMs for a live session (compact); clear/exit start fresh instead of resurrecting a completed session. - report: prompt/title column, per-session JSON export, full-window timeline. - Test fixtures use anonymized user identifiers (no real emails). Generated with AI Co-Authored-By: codemie-ai --- .../plans/2026-06-19-analytics-source-seam.md | 553 ++++++++++++++++++ .../fixtures/otel-events.sample.jsonl | 18 + .../analytics/__tests__/otel-loader.test.ts | 200 +++++++ .../__tests__/otel-report.integration.test.ts | 69 +++ .../cost/__tests__/usage-readers.test.ts | 13 + .../commands/analytics/cost/usage-readers.ts | 38 +- src/cli/commands/analytics/data-loader.ts | 4 +- src/cli/commands/analytics/index.ts | 343 +++++------ src/cli/commands/analytics/otel-loader.ts | 531 +++++++++++++++++ .../commands/analytics/report/client/app.js | 77 ++- .../commands/analytics/report/template.html | 8 +- .../commands/analytics/sources/otel-source.ts | 47 ++ .../analytics/sources/sessions-source.ts | 30 + src/cli/commands/analytics/sources/types.ts | 37 ++ src/cli/commands/analytics/types.ts | 8 + src/cli/commands/hook.ts | 34 ++ src/utils/errors.ts | 7 + 17 files changed, 1818 insertions(+), 199 deletions(-) create mode 100644 docs/superpowers/plans/2026-06-19-analytics-source-seam.md create mode 100644 src/cli/commands/analytics/__tests__/fixtures/otel-events.sample.jsonl create mode 100644 src/cli/commands/analytics/__tests__/otel-loader.test.ts create mode 100644 src/cli/commands/analytics/__tests__/otel-report.integration.test.ts create mode 100644 src/cli/commands/analytics/otel-loader.ts create mode 100644 src/cli/commands/analytics/sources/otel-source.ts create mode 100644 src/cli/commands/analytics/sources/sessions-source.ts create mode 100644 src/cli/commands/analytics/sources/types.ts diff --git a/docs/superpowers/plans/2026-06-19-analytics-source-seam.md b/docs/superpowers/plans/2026-06-19-analytics-source-seam.md new file mode 100644 index 00000000..62d5db9c --- /dev/null +++ b/docs/superpowers/plans/2026-06-19-analytics-source-seam.md @@ -0,0 +1,553 @@ +# Analytics Source Seam Implementation Plan + +> **For agentic workers:** Steps use checkbox (`- [ ]`) syntax for tracking. +> +> **Project policy adaptations (AGENTS.md + user preference override the writing-plans defaults):** +> - **No TDD-first steps.** Tests are written/run only on explicit user request. Each task validates with `npx tsc --noEmit` (typecheck) and `npm run lint`. +> - **No per-task `git commit` steps.** Commit only when the user explicitly asks; deliver a single summary at the end. +> - Existing OTEL tests (`__tests__/otel-loader.test.ts`, `__tests__/otel-report.integration.test.ts`) call `loadOtelSessions` directly, which is **unchanged in signature**, so they keep passing. Run `npm test -- analytics` to confirm only if the user asks. + +**Goal:** Introduce an `AnalyticsSource` seam so the CLI is agnostic to where analytics data comes from, and move the (unreleased) OTEL surface from loose `--source`/`--otel-file` flags onto an `analytics otel` subcommand — making future Prometheus/Loki/Langfuse backends a one-file drop-in. + +**Architecture:** Define `AnalyticsSource.load() → { rawSessions, cost? }`. `SessionsSource` wraps the existing `MetricsDataLoader` + native-loader merge (cost omitted, enriched later only for reports). `OtelSource` wraps `loadOtelSessions` (cost authoritative, returned up-front). `index.ts` becomes a thin CLI layer: a default command (sessions) plus an `otel` subcommand, both sharing one `applyCommonOptions()` set, both delegating to a single `runAnalytics(options, source)`. + +**Tech Stack:** TypeScript (ESM, `.js` import extensions), commander, Vitest (existing only). + +**Back-compat anchor:** `--report` / `--report-format` / `--open` / filters are released → preserved on the default command unchanged. `--source` / `--otel-file` / `--user` are new on this branch and **unreleased** → replaced by the subcommand with no alias needed. + +--- + +## File Structure + +**Create:** +- `src/cli/commands/analytics/sources/types.ts` — `AnalyticsSource`, `SourceLoadOptions`, `SourceResult` + a header note reserving the future-backend shape. +- `src/cli/commands/analytics/sources/sessions-source.ts` — `SessionsSource` (local + native). +- `src/cli/commands/analytics/sources/otel-source.ts` — `OtelSource` (flattened OTEL file). + +**Modify:** +- `src/utils/errors.ts` — add `AnalyticsSourceError`. +- `src/cli/commands/analytics/data-loader.ts:16,33` — `export` `SessionStartEvent` / `SessionEndEvent`. +- `src/cli/commands/analytics/otel-loader.ts` — typed event factory (drop `as unknown as`), fix `lookupTranscriptCwd` `{}`→`undefined`. +- `src/cli/commands/analytics/types.ts` — drop `source`/`otelFile` from `AnalyticsOptions`; add `OtelCommandOptions`. +- `src/cli/commands/analytics/index.ts` — `applyCommonOptions`, `runAnalytics`, default + `otel` subcommand. + +--- + +## Task 1: Error class + export event types + +**Files:** +- Modify: `src/utils/errors.ts` +- Modify: `src/cli/commands/analytics/data-loader.ts:16`, `:33` + +- [ ] **Step 1: Add `AnalyticsSourceError`** to `src/utils/errors.ts` (after `PathSecurityError`, before the npm section): + +```ts +export class AnalyticsSourceError extends CodeMieError { + constructor(message: string) { + super(message); + this.name = 'AnalyticsSourceError'; + } +} +``` + +- [ ] **Step 2: Export the two event interfaces** so the OTEL loader can construct typed values. In `src/cli/commands/analytics/data-loader.ts` change: + +```ts +interface SessionStartEvent { +``` +to +```ts +export interface SessionStartEvent { +``` +and +```ts +interface SessionEndEvent { +``` +to +```ts +export interface SessionEndEvent { +``` + +- [ ] **Step 3: Typecheck.** Run `npx tsc --noEmit`. Expected: PASS (these are additive/visibility-only). + +--- + +## Task 2: Source seam types + +**Files:** +- Create: `src/cli/commands/analytics/sources/types.ts` + +- [ ] **Step 1: Write the seam.** + +```ts +/** + * Analytics source seam. + * + * Every data backend implements {@link AnalyticsSource}: it loads its data and returns the + * same {@link SourceResult} the aggregate → report pipeline consumes, so the CLI stays + * agnostic to where the data came from. + * + * Today: {@link SessionsSource} (local ~/.codemie + native logs) and OtelSource + * (flattened otel-events.jsonl). Future remote backends (Prometheus, Loki, Langfuse) become + * one new `AnalyticsSource` + one thin `analytics ` subcommand each, with connection + * URL + auth resolved from the `.codemie` config/profile (never CLI flags) — e.g. + * "analyticsSources": { "prod-langfuse": { "type": "langfuse", "url": "…", "authEnv": "LANGFUSE_TOKEN" } } + * where `authEnv` names the env var holding the token. No connector code exists yet. + */ +import type { RawSessionData } from '../data-loader.js'; +import type { AnalyticsFilter } from '../types.js'; +import type { SessionCostIndex, CostSummary } from '../cost/types.js'; + +export interface SourceLoadOptions { + filter: AnalyticsFilter; + /** Sessions source only: skip native agent-log discovery. Ignored by other sources. */ + scanNative?: boolean; +} + +export interface SourceResult { + rawSessions: RawSessionData[]; + /** + * Authoritative cost, present only when the source carries it in its own data + * (e.g. OTEL `cost_usd`). Omitted when cost must be derived later from correlated logs + * (local sessions → cost-enricher); the runner enriches in that case. + */ + cost?: { index: SessionCostIndex; summary: CostSummary }; +} + +export interface AnalyticsSource { + load(opts: SourceLoadOptions): Promise; +} +``` + +- [ ] **Step 2: Typecheck.** Run `npx tsc --noEmit`. Expected: PASS. + +--- + +## Task 3: SessionsSource + +**Files:** +- Create: `src/cli/commands/analytics/sources/sessions-source.ts` + +- [ ] **Step 1: Implement** (lifts the original sessions branch from `index.ts` verbatim; cost omitted — runner enriches when a report is wanted): + +```ts +import { MetricsDataLoader } from '../data-loader.js'; +import { logger } from '../../../../utils/logger.js'; +import type { AnalyticsSource, SourceLoadOptions, SourceResult } from './types.js'; + +/** + * Local-session source: ~/.codemie tracked sessions merged with discovered native agent + * logs (plain `claude` etc.). Cost is omitted here and derived by the cost-enricher only + * when a report is requested, matching the original analytics behavior. + */ +export class SessionsSource implements AnalyticsSource { + async load(opts: SourceLoadOptions): Promise { + const loader = new MetricsDataLoader(); + const rawSessions = loader.loadSessions(opts.filter); + + // Discover native agent logs (not tracked by CodeMie) and merge so analytics reflect + // ALL usage. Deduped against tracked logs inside the loader. + if (opts.scanNative !== false) { + try { + const { loadNativeSessions } = await import('../native-loader.js'); + const natives = (await loadNativeSessions(opts.filter)).filter((s) => + loader.sessionMatchesFilter(s, opts.filter) + ); + rawSessions.push(...natives); + } catch (error) { + logger.debug('Native session discovery failed (continuing with tracked sessions):', error); + } + } + return { rawSessions }; + } +} +``` + +- [ ] **Step 2: Typecheck.** Run `npx tsc --noEmit`. Expected: PASS. + +--- + +## Task 4: OtelSource + +**Files:** +- Create: `src/cli/commands/analytics/sources/otel-source.ts` + +- [ ] **Step 1: Implement** (adds the friendly missing-file error — Part-1 review fix — via `AnalyticsSourceError`): + +```ts +import { existsSync } from 'node:fs'; +import { AnalyticsSourceError } from '../../../../utils/errors.js'; +import type { AnalyticsSource, SourceLoadOptions, SourceResult } from './types.js'; + +/** + * OTEL source: a flattened otel-events.jsonl file. Cost is authoritative (native + * `cost_usd`), so it is returned up-front and bypasses the cost-enricher. + */ +export class OtelSource implements AnalyticsSource { + constructor(private readonly file: string, private readonly user?: string) {} + + async load(opts: SourceLoadOptions): Promise { + if (!existsSync(this.file)) { + throw new AnalyticsSourceError(`OTEL file not found: ${this.file}`); + } + const { loadOtelSessions } = await import('../otel-loader.js'); + const res = loadOtelSessions({ + file: this.file, + filter: { user: this.user, since: opts.filter.fromDate, until: opts.filter.toDate }, + }); + return { rawSessions: res.rawSessions, cost: { index: res.costIndex, summary: res.summary } }; + } +} +``` + +- [ ] **Step 2: Typecheck.** Run `npx tsc --noEmit`. Expected: PASS. + +--- + +## Task 5: OTEL loader fixes (Part-1 review items) + +**Files:** +- Modify: `src/cli/commands/analytics/otel-loader.ts:27` (import), `:296-314` (cwd), `:418-445` (buildRawSession) + +- [ ] **Step 1: Import the now-exported event types.** Change line 27: + +```ts +import type { RawSessionData } from './data-loader.js'; +``` +to +```ts +import type { RawSessionData, SessionStartEvent, SessionEndEvent } from './data-loader.js'; +``` + +- [ ] **Step 2: Drop the double cast** in `buildRawSession` — construct typed events, return a typed `RawSessionData`: + +```ts +export function buildRawSession(sessionId: string, sessionEvents: OtelEvent[], cwd: CwdInfo | undefined, projectLabel: string): RawSessionData { + const times = sessionEvents.map((e) => Date.parse(e.ts)).filter(Number.isFinite); + const startTime = times.length ? Math.min(...times) : 0; + const endTime = times.length ? Math.max(...times) : 0; + const workingDirectory = cwd?.cwd || projectLabel; + const startEvent: SessionStartEvent = { + recordId: sessionId, + type: 'session_start', + timestamp: startTime, + codeMieSessionId: sessionId, + agentName: 'claude', + syncStatus: 'synced', + data: { provider: 'claude', workingDirectory, startTime }, + }; + const endEvent: SessionEndEvent = { + recordId: `${sessionId}-end`, + type: 'session_end', + timestamp: endTime, + codeMieSessionId: sessionId, + agentName: 'claude', + syncStatus: 'synced', + data: { endTime, duration: Math.max(0, endTime - startTime), totalTurns: 0 }, + }; + return { sessionId, startEvent, endEvent, deltas: buildDeltas(sessionEvents, cwd) }; +} +``` + +- [ ] **Step 3: Fix the cwd fallback bug.** In `lookupTranscriptCwd`, the trailing `return {};` (transcript found but no cwd/branch in the head) defeats the `?? pluginCwdMap` fallback in `loadOtelSessions`. Change the final `return {};` to: + +```ts + return undefined; +``` + +(So a session whose transcript head lacks cwd/branch falls back to this plugin's own `session.start` event.) + +- [ ] **Step 4: Typecheck.** Run `npx tsc --noEmit`. Expected: PASS (typed events satisfy `RawSessionData` with no cast). + +--- + +## Task 6: Options types + +**Files:** +- Modify: `src/cli/commands/analytics/types.ts:248-258` + +- [ ] **Step 1: Remove the unreleased loose flags and add the subcommand options type.** In `AnalyticsOptions`, delete the three new fields (`source`, `otelFile`, `user`). Then add, after the `AnalyticsOptions` interface: + +```ts +/** Options for the `analytics otel` subcommand: the shared base plus OTEL-specific flags. */ +export interface OtelCommandOptions extends AnalyticsOptions { + /** Path to the flattened OTEL events file (required). */ + file: string; + /** Scope OTEL analytics to one user (matches native user.email or user.id). */ + user?: string; +} +``` + +> Keep the `scanNative?: boolean` field that already exists on `AnalyticsOptions` (backs `--no-scan-native`). + +- [ ] **Step 2: Typecheck.** Run `npx tsc --noEmit`. Expected: FAIL only in `index.ts` (still references removed fields) — fixed in Task 7. + +--- + +## Task 7: CLI restructure (default command + `otel` subcommand) + +**Files:** +- Modify: `src/cli/commands/analytics/index.ts:1-236` + +- [ ] **Step 1: Replace the imports** (top of file) with: + +```ts +/** + * Analytics command - display aggregated metrics from sessions + */ + +import { Command } from 'commander'; +import chalk from 'chalk'; +import { AnalyticsAggregator } from './aggregator.js'; +import { AnalyticsFormatter } from './formatter.js'; +import { AnalyticsExporter } from './exporter.js'; +import type { AnalyticsOptions, AnalyticsFilter, OtelCommandOptions } from './types.js'; +import { logger } from '../../../utils/logger.js'; +import { SessionsSource } from './sources/sessions-source.js'; +import { OtelSource } from './sources/otel-source.js'; +import type { AnalyticsSource } from './sources/types.js'; +``` + +- [ ] **Step 2: Replace `createAnalyticsCommand`** with a thin builder: shared options helper + default (sessions) action + `otel` subcommand. Commander runs the parent action when no known subcommand is given, and routes to `otel` when it is — so bare `codemie analytics --report` keeps working. + +```ts +export function createAnalyticsCommand(): Command { + const command = new Command('analytics') + .description('Display aggregated metrics and analytics from sessions'); + + // Default source: local CodeMie-tracked sessions + native agent logs. + applyCommonOptions(command) + .option('--no-scan-native', 'Skip native agent-log discovery (use only CodeMie-tracked sessions)') + .action((options: AnalyticsOptions) => runAnalytics(options, new SessionsSource())); + + // `codemie analytics otel --file ` — OTEL file source. + const otel = new Command('otel') + .description('Analytics from a flattened OTEL events file (otel-events.jsonl)'); + applyCommonOptions(otel) + .requiredOption('--file ', 'Path to the flattened OTEL events file') + .option('--user ', 'Scope to one user (native user.email or user.id)') + .action((options: OtelCommandOptions) => runAnalytics(options, new OtelSource(options.file, options.user))); + command.addCommand(otel); + + return command; +} + +/** Filter, report, export, and verbosity options shared by every analytics source. */ +function applyCommonOptions(command: Command): Command { + return command + .option('--session ', 'Filter by session ID') + .option('--project ', 'Filter by project path (basename, partial, or full path)') + .option('--agent ', 'Filter by agent name (claude, gemini, etc.)') + .option('--branch ', 'Filter by git branch') + .option('--from ', 'Filter sessions from date (YYYY-MM-DD)') + .option('--to ', 'Filter sessions to date (YYYY-MM-DD)') + .option('--last ', 'Filter sessions from last duration (e.g., 7d, 24h)') + .option('-v, --verbose', 'Show detailed session-level breakdown') + .option('--export ', 'Export to file (json or csv)') + .option('-o, --output ', 'Output file path (default: ./codemie-analytics-YYYY-MM-DD.{format})') + .option('--report', 'Generate a self-contained HTML dashboard') + .option('--open', 'Open the generated HTML report in the default browser') + .option('--report-output ', 'HTML report output path (default: ./codemie-analytics-YYYY-MM-DD.html)') + .option('--report-format ', 'Report serialization: html, json, or both (default: html)'); +} +``` + +- [ ] **Step 3: Add `runAnalytics`** — the former action body, now source-agnostic. The only structural change vs. the original: data + optional authoritative cost come from `source.load()`, and cost is enriched only when the source did not supply it. Everything from aggregation onward is unchanged. + +```ts +async function runAnalytics(options: AnalyticsOptions, source: AnalyticsSource): Promise { + try { + const filter = parseFilterOptions(options); + const { rawSessions, cost } = await source.load({ filter, scanNative: options.scanNative }); + + if (rawSessions.length === 0) { + console.log(chalk.yellow('\nNo sessions found matching the specified criteria.')); + console.log(chalk.dim('Run with different filters or check that metrics are being collected.\n')); + return; + } + + // A report needs cost computed BEFORE aggregation so zero-delta sessions that still carry + // real usage are retained instead of dropped as "empty". + const wantReport = Boolean(options.report || options.reportOutput || options.open || options.reportFormat); + const reportFormat = (options.reportFormat ?? 'html').toLowerCase(); + if (wantReport && reportFormat !== 'html' && reportFormat !== 'json' && reportFormat !== 'both') { + console.log(chalk.red('\n✗ Invalid report format. Use "html", "json", or "both".')); + return; + } + + // Cost: authoritative from the source (OTEL) when present; otherwise enrich from correlated + // logs, but only when a report needs it. Retain zero-delta sessions with real token usage. + let costResult = cost; + let keepSessionIds: Set | undefined; + if (cost) { + keepSessionIds = new Set( + [...cost.index.values()].filter((c) => c.tokens.total > 0).map((c) => c.sessionId) + ); + } else if (wantReport) { + const { enrichCosts, realDeps } = await import('./cost/cost-enricher.js'); + costResult = await enrichCosts(rawSessions, realDeps); + keepSessionIds = new Set( + [...costResult.index.values()].filter((c) => c.tokens.total > 0).map((c) => c.sessionId) + ); + } + + // Aggregate data (normalize models unless --verbose flag is set) + const analytics = AnalyticsAggregator.aggregate(rawSessions, !options.verbose, keepSessionIds); + + if (analytics.totalSessions === 0) { + console.log(chalk.yellow('\nNo analytics data available.')); + console.log(chalk.dim('Metrics collection may not have been enabled for these sessions.\n')); + return; + } + + // Display results + const formatter = new AnalyticsFormatter(options.verbose); + formatter.displayRoot(analytics); + formatter.displayProjects(analytics.projects); + + // Export if requested + if (options.export) { + const format = options.export.toLowerCase(); + if (format !== 'json' && format !== 'csv') { + console.log(chalk.red('\n✗ Invalid export format. Use "json" or "csv".')); + return; + } + const outputPath = options.output || AnalyticsExporter.getDefaultOutputPath(format, process.cwd()); + if (format === 'json') { + AnalyticsExporter.exportJSON(analytics, outputPath); + } else { + AnalyticsExporter.exportCSV(analytics, outputPath); + } + } + + // Generate the report if requested (--report-output and --open imply --report) + if (wantReport && costResult) { + const { buildPayload } = await import('./report/payload-builder.js'); + const { + generateReport, + generateReportJson, + getDefaultReportPath, + getDefaultReportJsonPath, + writeReportWithFallback + } = await import('./report/report-generator.js'); + + const { index: costIndex, summary } = costResult; + const payload = buildPayload(analytics, costIndex, summary, { + rangeLabel: options.last ?? (options.from || options.to ? 'custom' : 'all'), + projectFilter: options.project ?? 'all', + generatedAt: new Date().toISOString() + }); + + const cwd = process.cwd(); + let htmlPath: string | undefined; + let jsonPath: string | undefined; + let htmlIsDefault = false; + let jsonIsDefault = false; + + if (reportFormat === 'both') { + const base = options.reportOutput?.replace(/\.(html|json)$/i, ''); + htmlPath = base ? `${base}.html` : getDefaultReportPath(cwd); + jsonPath = base ? `${base}.json` : getDefaultReportJsonPath(cwd); + htmlIsDefault = jsonIsDefault = !base; + } else if (reportFormat === 'html') { + htmlPath = options.reportOutput || getDefaultReportPath(cwd); + htmlIsDefault = !options.reportOutput; + } else { + jsonPath = options.reportOutput || getDefaultReportJsonPath(cwd); + jsonIsDefault = !options.reportOutput; + } + + if (htmlPath) { + const result = writeReportWithFallback((p) => generateReport(payload, p), htmlPath, htmlIsDefault); + htmlPath = result.path; + if (result.relocatedFrom) { + console.log( + chalk.yellow(`\n! ${result.relocatedFrom} is not writable (drive root or read-only volume); using a writable location instead.`) + ); + } + console.log(chalk.green(`\n✓ HTML report written to: ${htmlPath}`)); + } + if (jsonPath) { + const result = writeReportWithFallback((p) => generateReportJson(payload, p), jsonPath, jsonIsDefault); + jsonPath = result.path; + if (result.relocatedFrom) { + console.log( + chalk.yellow(`\n! ${result.relocatedFrom} is not writable (drive root or read-only volume); using a writable location instead.`) + ); + } + console.log(chalk.green(`\n✓ JSON report written to: ${jsonPath}`)); + } + + const { sessions: totalReportSessions, pricedSessions } = payload.meta.totals; + if (pricedSessions < totalReportSessions) { + console.log( + chalk.dim( + ` Cost priced for ${pricedSessions}/${totalReportSessions} sessions (native agent logs required for the rest).` + ) + ); + } + + if (options.open) { + if (htmlPath) { + const { openUrlInBrowser } = await import('../../../utils/browser.js'); + await openUrlInBrowser(htmlPath); + } else { + console.log(chalk.dim(' --open ignored: no HTML produced (use --report-format html or both).')); + } + } + } + + console.log(''); + } catch (error) { + logger.error('Analytics command failed:', error); + console.error(chalk.red(`\n✗ Failed to generate analytics: ${error instanceof Error ? error.message : String(error)}\n`)); + process.exit(1); + } +} +``` + +- [ ] **Step 4: Leave `parseFilterOptions` unchanged** (it stays at the bottom of the file). + +- [ ] **Step 5: Typecheck.** Run `npx tsc --noEmit`. Expected: PASS (no more references to removed `source`/`otelFile` fields). + +--- + +## Task 8: Validate end to end + +**Files:** none (verification only) + +- [ ] **Step 1: Lint.** Run `npm run lint`. Expected: zero warnings (project policy). + +- [ ] **Step 2: Build.** Run `npm run build`. Expected: PASS. + +- [ ] **Step 3: Manual CLI check — default command unchanged.** Run: + +```bash +node bin/codemie.js analytics --help +``` +Expected: shows shared options AND an `otel` subcommand in the command list. + +- [ ] **Step 4: Manual CLI check — OTEL subcommand against the fixture.** Run: + +```bash +node bin/codemie.js analytics otel --file src/cli/commands/analytics/__tests__/fixtures/otel-events.sample.jsonl --report --report-output /tmp/otel-seam-check.html +``` +Expected: `✓ HTML report written to: /tmp/otel-seam-check.html`; two sessions aggregated. + +- [ ] **Step 5: Manual CLI check — missing-file error.** Run: + +```bash +node bin/codemie.js analytics otel --file /no/such/file.jsonl +``` +Expected: `✗ Failed to generate analytics: OTEL file not found: /no/such/file.jsonl` (no raw `ENOENT`). + +- [ ] **Step 6 (optional, only if user asks for tests):** `npm test -- analytics` — existing OTEL tests should still pass (`loadOtelSessions` signature is unchanged). + +--- + +## Self-Review + +- **Spec coverage:** seam interface (T2) ✓; sessions impl (T3) ✓; otel impl (T4) ✓; subcommand surface + default-command back-compat (T7) ✓; config shape reserved as doc note (T2 header) ✓; Part-1 fixes — typed events/no cast (T5), cwd fallback (T5), friendly missing-file error (T4) ✓. Out of scope by design: remote connectors, dedup, streaming, privacy redaction (documented as future constraints, not built). +- **Type consistency:** `SourceResult.cost` (`{ index, summary } | undefined`) is produced by `OtelSource` and consumed in `runAnalytics`; `enrichCosts` returns the same `{ index, summary }` shape → assignment to `costResult` type-checks. `OtelCommandOptions.file`/`user` match `OtelSource` constructor args. `applyCommonOptions` is applied to both the parent and `otel`, so both carry identical filter/report flags. +- **No placeholders:** every code step shows complete code; every run step shows the exact command + expected output. +- **Excluded from this work (commit hygiene):** `.codemie/codemie-cli.config.json` (profile switch) and `.claude/settings.json` (plugin toggle) are local env changes, not part of this feature. The `hook.ts` session re-entry change is adjacent (session tracking, not analytics) and untested — decide separately whether it ships on this branch. diff --git a/src/cli/commands/analytics/__tests__/fixtures/otel-events.sample.jsonl b/src/cli/commands/analytics/__tests__/fixtures/otel-events.sample.jsonl new file mode 100644 index 00000000..7a2fc8ba --- /dev/null +++ b/src/cli/commands/analytics/__tests__/fixtures/otel-events.sample.jsonl @@ -0,0 +1,18 @@ +{"_type":"log","ts":"2026-06-19T09:59:50.000Z","scope":"claude-code-otel-plugin","name":"claude-code-otel-plugin","attrs":{"event_type":"claude.session.start","session_id":"11111111-1111-1111-1111-111111111111","cwd":"/Users/dev/projects/codemie-code","git_branch":"feature/analytics-otel-source"},"resource":{"service.name":"claude-code-otel-plugin","user.id":"dev"}} +{"_type":"log","ts":"2026-06-19T10:00:00.000Z","scope":"com.anthropic.claude_code","name":"user_prompt","attrs":{"event.name":"user_prompt","session.id":"11111111-1111-1111-1111-111111111111","prompt.id":"P1","prompt":"Build the analytics loader","prompt_length":26,"user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:00:05.000Z","scope":"com.anthropic.claude_code","name":"api_request","attrs":{"event.name":"api_request","session.id":"11111111-1111-1111-1111-111111111111","prompt.id":"P1","model":"claude-opus-4-8","query_source":"repl_main_thread","input_tokens":1200,"output_tokens":800,"cache_read_tokens":15000,"cache_creation_tokens":2000,"cost_usd":0.0345,"user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:00:10.000Z","scope":"com.anthropic.claude_code","name":"skill_activated","attrs":{"event.name":"skill_activated","session.id":"11111111-1111-1111-1111-111111111111","prompt.id":"P1","skill.name":"superpowers:brainstorming","plugin.name":"superpowers","user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:00:20.000Z","scope":"com.anthropic.claude_code","name":"tool_result","attrs":{"event.name":"tool_result","session.id":"11111111-1111-1111-1111-111111111111","prompt.id":"P1","tool_name":"Edit","success":"true","duration_ms":42,"user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:00:25.000Z","scope":"com.anthropic.claude_code","name":"tool_result","attrs":{"event.name":"tool_result","session.id":"11111111-1111-1111-1111-111111111111","prompt.id":"P1","tool_name":"Edit","success":"true","duration_ms":38,"user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:00:30.000Z","scope":"com.anthropic.claude_code","name":"tool_result","attrs":{"event.name":"tool_result","session.id":"11111111-1111-1111-1111-111111111111","prompt.id":"P1","tool_name":"Bash","success":"true","duration_ms":275,"user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:00:30.500Z","scope":"com.anthropic.claude_code","name":"api_request","attrs":{"event.name":"api_request","session.id":"11111111-1111-1111-1111-111111111111","prompt.id":"P1","model":"claude-opus-4-8","query_source":"repl_main_thread","input_tokens":900,"output_tokens":600,"cache_read_tokens":18000,"cache_creation_tokens":0,"cost_usd":0.0120,"user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:01:00.000Z","scope":"com.anthropic.claude_code","name":"subagent_completed","attrs":{"event.name":"subagent_completed","session.id":"11111111-1111-1111-1111-111111111111","prompt.id":"P1","agent_type":"tech-analyst","duration_ms":120000,"total_tokens":5000,"total_tool_uses":7,"model":"claude-sonnet-4-6","user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"metric","ts":"2026-06-19T10:01:05.000Z","scope":"com.anthropic.claude_code","name":"claude_code.lines_of_code.count","kind":"sum","value":120,"attrs":{"session.id":"11111111-1111-1111-1111-111111111111","type":"added","user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"metric","ts":"2026-06-19T10:01:05.000Z","scope":"com.anthropic.claude_code","name":"claude_code.lines_of_code.count","kind":"sum","value":30,"attrs":{"session.id":"11111111-1111-1111-1111-111111111111","type":"removed","user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:03:00.000Z","scope":"com.anthropic.claude_code","name":"user_prompt","attrs":{"event.name":"user_prompt","session.id":"11111111-1111-1111-1111-111111111111","prompt.id":"P2","prompt":"fix the timeline","prompt_length":16,"user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:03:10.000Z","scope":"com.anthropic.claude_code","name":"tool_result","attrs":{"event.name":"tool_result","session.id":"11111111-1111-1111-1111-111111111111","prompt.id":"P2","tool_name":"Read","success":"false","duration_ms":12,"user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:03:30.000Z","scope":"com.anthropic.claude_code","name":"api_request","attrs":{"event.name":"api_request","session.id":"11111111-1111-1111-1111-111111111111","prompt.id":"P2","model":"claude-opus-4-8","query_source":"repl_main_thread","input_tokens":700,"output_tokens":400,"cache_read_tokens":21000,"cache_creation_tokens":0,"cost_usd":0.0080,"user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:02:00.000Z","scope":"com.anthropic.claude_code","name":"user_prompt","attrs":{"event.name":"user_prompt","session.id":"22222222-2222-2222-2222-222222222222","prompt.id":"P3","prompt":"research the indexer","prompt_length":20,"user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:02:05.000Z","scope":"com.anthropic.claude_code","name":"api_request","attrs":{"event.name":"api_request","session.id":"22222222-2222-2222-2222-222222222222","prompt.id":"P3","model":"claude-sonnet-4-6","query_source":"agent:tech-analyst","input_tokens":2200,"output_tokens":1500,"cache_read_tokens":9000,"cache_creation_tokens":3000,"cost_usd":0.0200,"user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:02:45.000Z","scope":"com.anthropic.claude_code","name":"api_request","attrs":{"event.name":"api_request","session.id":"22222222-2222-2222-2222-222222222222","prompt.id":"P3","model":"converse/eu.anthropic.claude-sonnet-4-6-v1:0","query_source":"agent:tech-analyst","input_tokens":1800,"output_tokens":1100,"cache_read_tokens":11000,"cache_creation_tokens":0,"cost_usd":0.0150,"user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} +{"_type":"log","ts":"2026-06-19T10:02:10.000Z","scope":"com.anthropic.claude_code","name":"tool_result","attrs":{"event.name":"tool_result","session.id":"22222222-2222-2222-2222-222222222222","prompt.id":"P3","tool_name":"Grep","success":"true","duration_ms":55,"user.email":"dev@example.com"},"resource":{"service.name":"claude-code"}} diff --git a/src/cli/commands/analytics/__tests__/otel-loader.test.ts b/src/cli/commands/analytics/__tests__/otel-loader.test.ts new file mode 100644 index 00000000..a4764746 --- /dev/null +++ b/src/cli/commands/analytics/__tests__/otel-loader.test.ts @@ -0,0 +1,200 @@ +/** + * OTEL analytics source tests. + * + * The OTEL loader reads a flattened otel-events.jsonl (the contract produced by the + * codemie-claude-otel file sink) and maps native Claude Code events into the same + * RawSessionData + SessionCostIndex the analytics pipeline consumes — so + * `codemie analytics --source otel --report` produces the existing HTML (incl. session + * detail: tools, timeline, skills, turns, title, net lines) from OTEL data. + */ + +import { describe, it, expect } from 'vitest'; +import { readFileSync, mkdtempSync, mkdirSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { + parseOtelJsonl, + filterApiRequests, + buildCostIndex, + buildDispatches, + lookupTranscriptCwd, + loadOtelSessions, + type OtelEvent, +} from '../otel-loader.js'; + +const FIXTURE = join(__dirname, 'fixtures', 'otel-events.sample.jsonl'); +const TEXT = readFileSync(FIXTURE, 'utf-8'); + +const MAIN = '11111111-1111-1111-1111-111111111111'; +const SUB = '22222222-2222-2222-2222-222222222222'; + +describe('parseOtelJsonl', () => { + it('parses every JSONL line and skips malformed ones', () => { + const events = parseOtelJsonl(TEXT + '\n{ not json\n'); + expect(events.length).toBe(18); + expect(events.filter((e) => e.name === 'api_request')).toHaveLength(5); + }); +}); + +describe('filterApiRequests', () => { + const events = parseOtelJsonl(TEXT); + it('returns only api_request log events', () => { + expect(filterApiRequests(events)).toHaveLength(5); + }); + it('filters by user (matches user.email)', () => { + expect(filterApiRequests(events, { user: 'dev@example.com' })).toHaveLength(5); + expect(filterApiRequests(events, { user: 'nobody@x.io' })).toHaveLength(0); + }); + it('filters by time window', () => { + const since = new Date('2026-06-19T10:02:30.000Z'); + expect(filterApiRequests(events, { since })).toHaveLength(2); + }); +}); + +describe('buildCostIndex', () => { + const { index, summary } = buildCostIndex(filterApiRequests(parseOtelJsonl(TEXT))); + it('splits cost per session and totals correctly', () => { + expect(index.size).toBe(2); + expect(index.get(MAIN)!.costUSD).toBeCloseTo(0.0545, 6); + expect(index.get(SUB)!.costUSD).toBeCloseTo(0.035, 6); + expect(summary.totalCostUSD).toBeCloseTo(0.0895, 6); + expect(summary.totalSessions).toBe(2); + }); + it('sums token usage per session', () => { + const m = index.get(MAIN)!.tokens; + expect(m.input).toBe(2800); + expect(m.output).toBe(1800); + expect(m.cacheRead).toBe(54000); + expect(m.cacheCreation).toBe(2000); + expect(m.total).toBe(60600); + }); + it('merges bedrock + native model spellings (normalization)', () => { + // SUB used claude-sonnet-4-6 AND converse/eu.anthropic.claude-sonnet-4-6-v1:0 + const models = index.get(SUB)!.perModel.map((p) => p.model); + expect(models).toEqual(['claude-sonnet-4-6']); + }); +}); + +describe('buildDispatches', () => { + it('emits agent + skill dispatches (agent spans completion − duration)', () => { + const mainEvents = parseOtelJsonl(TEXT).filter((e) => (e.attrs['session.id'] as string) === MAIN); + const d = buildDispatches(mainEvents); + // agent completes @10:01:00 with 120s duration → starts @09:59:00, before the skill @10:00:10 + expect(d.map((x) => x.kind)).toEqual(['agent', 'skill']); + const agent = d.find((x) => x.kind === 'agent')!; + expect(agent.name).toBe('tech-analyst'); + expect(agent.durationMs).toBe(120000); + // no agent:* api_requests in MAIN's window → falls back to subagent_completed total_tokens + expect(agent.tokens?.total).toBe(5000); + }); + + it('recovers dispatch cost + tokens from windowed agent:* api_requests', () => { + const ev = (name: string, ts: string, extra: Record): OtelEvent => ({ + _type: 'log', + ts, + name, + attrs: { 'session.id': 'S', 'event.name': name, ...extra }, + resource: {}, + }); + const dispatches = buildDispatches([ + ev('subagent_completed', '2026-06-19T10:05:00.000Z', { agent_type: 'Explore', duration_ms: 200000, total_tokens: 999 }), + // two subagent calls inside [10:01:40, 10:05:00] + ev('api_request', '2026-06-19T10:02:00.000Z', { query_source: 'agent:builtin:Explore', cost_usd: 0.5, input_tokens: 100, output_tokens: 50 }), + ev('api_request', '2026-06-19T10:04:00.000Z', { query_source: 'agent:builtin:Explore', cost_usd: 0.5, input_tokens: 200, output_tokens: 80 }), + // a main call outside the agent attribution + ev('api_request', '2026-06-19T10:03:00.000Z', { query_source: 'repl_main_thread', cost_usd: 9.9, input_tokens: 1 }), + ]); + const agent = dispatches.find((x) => x.kind === 'agent')!; + expect(agent.costUSD).toBeCloseTo(1.0, 6); // windowed agent calls only, not the $9.9 main call + expect(agent.tokens?.total).toBe(430); // 100+50+200+80 + }); + + it('attributes real cost + duration to skills via api_request.skill.name', () => { + const ev = (name: string, ts: string, extra: Record): OtelEvent => ({ + _type: 'log', + ts, + name, + attrs: { 'session.id': 'S', 'event.name': name, ...extra }, + resource: {}, + }); + const dispatches = buildDispatches([ + ev('skill_activated', '2026-06-19T10:00:00.000Z', { 'skill.name': 'code-review' }), + ev('api_request', '2026-06-19T10:00:05.000Z', { 'skill.name': 'code-review', cost_usd: 0.20, input_tokens: 100, output_tokens: 50, duration_ms: 3000 }), + ev('api_request', '2026-06-19T10:00:10.000Z', { 'skill.name': 'code-review', cost_usd: 0.30, input_tokens: 200, output_tokens: 80, duration_ms: 2000 }), + // unrelated request (different skill) must not be attributed: + ev('api_request', '2026-06-19T10:00:07.000Z', { 'skill.name': 'other', cost_usd: 9.9, input_tokens: 1 }), + ]); + const skill = dispatches.find((x) => x.kind === 'skill')!; + expect(skill.name).toBe('code-review'); + expect(skill.costUSD).toBeCloseTo(0.5, 6); // 0.20 + 0.30, not the $9.9 "other" + expect(skill.tokens?.total).toBe(430); // 100+50+200+80 + expect(skill.durationMs).toBe(12000); // last(10:00:10 + 2000ms) − start(10:00:00) + }); +}); + +describe('lookupTranscriptCwd', () => { + it('recovers cwd + branch from a Claude Code transcript', () => { + const projects = mkdtempSync(join(tmpdir(), 'cc-projects-')); + const dir = join(projects, '-Users-dev-my-repo'); + mkdirSync(dir); + const sid = 'aaaa1111-bbbb-2222-cccc-3333dddd4444'; + writeFileSync( + join(dir, `${sid}.jsonl`), + [JSON.stringify({ type: 'summary', sessionId: sid }), JSON.stringify({ type: 'user', cwd: '/Users/dev/my-repo', gitBranch: 'develop', sessionId: sid })].join('\n') + ); + const info = lookupTranscriptCwd(sid, projects); + expect(info).toEqual({ cwd: '/Users/dev/my-repo', branch: 'develop' }); + expect(lookupTranscriptCwd('no-such-session', projects)).toBeUndefined(); + }); +}); + +describe('loadOtelSessions (full session detail)', () => { + // Inject a no-op transcript lookup so the test exercises the plugin session.start + // fallback deterministically (no dependency on the real ~/.claude/projects). + const { rawSessions, costIndex } = loadOtelSessions({ file: FIXTURE, cwdLookup: () => undefined }); + const main = rawSessions.find((r) => r.sessionId === MAIN)!; + const sub = rawSessions.find((r) => r.sessionId === SUB)!; + + it('produces one RawSessionData per owned session', () => { + expect(rawSessions).toHaveLength(2); + }); + + it('derives turns from user_prompt events (one delta per prompt)', () => { + expect(main.deltas).toHaveLength(2); // P1, P2 + expect(sub.deltas).toHaveLength(1); // P3 + }); + + it('extracts the session title from the first prompt', () => { + expect(main.deltas[0].userPrompts?.[0].text).toBe('Build the analytics loader'); + }); + + it('aggregates tool calls with success/failure', () => { + const t = main.deltas[0]; + expect(t.tools).toEqual({ Edit: 2, Bash: 1, Read: 1 }); + expect(t.toolStatus!.Edit).toEqual({ success: 2, failure: 0 }); + expect(t.toolStatus!.Read).toEqual({ success: 0, failure: 1 }); + }); + + it('captures agent + skill invocations', () => { + expect(main.deltas[0].agentInvocations).toEqual({ 'tech-analyst': 1 }); + expect(main.deltas[0].skillInvocations).toEqual({ 'superpowers:brainstorming': 1 }); + }); + + it('recovers net lines from lines_of_code events', () => { + expect(main.deltas[0].fileOperations).toEqual([{ type: 'edit', path: '', linesAdded: 120, linesRemoved: 30 }]); + }); + + it('recovers project (cwd) and git branch from the plugin session.start join', () => { + expect(main.startEvent!.data.workingDirectory).toBe('/Users/dev/projects/codemie-code'); + expect(main.deltas[0].gitBranch).toBe('feature/analytics-otel-source'); + }); + + it('falls back to the project label when no plugin cwd is present', () => { + expect(sub.startEvent!.data.workingDirectory).toBe('Unknown'); // SUB has no session.start event + }); + + it('attaches Timeline dispatches to the cost index', () => { + expect(costIndex.get(MAIN)!.dispatches).toHaveLength(2); + expect(costIndex.get(SUB)!.dispatches).toBeUndefined(); // SUB has no agent/skill events + }); +}); diff --git a/src/cli/commands/analytics/__tests__/otel-report.integration.test.ts b/src/cli/commands/analytics/__tests__/otel-report.integration.test.ts new file mode 100644 index 00000000..2771e49a --- /dev/null +++ b/src/cli/commands/analytics/__tests__/otel-report.integration.test.ts @@ -0,0 +1,69 @@ +/** + * OTEL source → existing report pipeline, end to end (no CLI/commander): + * loadOtelSessions → aggregate → buildPayload → generateReport. Proves the OTEL + * data produces the same HTML report with correct cost/tokens. + */ + +import { describe, it, expect } from 'vitest'; +import { readFileSync, mkdtempSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { loadOtelSessions } from '../otel-loader.js'; +import { AnalyticsAggregator } from '../aggregator.js'; +import { buildPayload } from '../report/payload-builder.js'; +import { generateReport } from '../report/report-generator.js'; + +const FIXTURE = join(__dirname, 'fixtures', 'otel-events.sample.jsonl'); + +describe('OTEL → report pipeline', () => { + const { rawSessions, costIndex, summary } = loadOtelSessions({ file: FIXTURE }); + const keep = new Set( + [...costIndex.values()].filter((c) => c.tokens.total > 0).map((c) => c.sessionId) + ); + const analytics = AnalyticsAggregator.aggregate(rawSessions, true, keep); + + it('aggregates both OTEL sessions', () => { + expect(analytics.totalSessions).toBe(2); + }); + + it('builds a payload whose totals carry the authoritative OTEL cost', () => { + const payload = buildPayload(analytics, costIndex, summary, { + rangeLabel: 'all', + projectFilter: 'all', + generatedAt: '2026-06-19T00:00:00.000Z', + }); + expect(payload.sessions).toHaveLength(2); + expect(payload.meta.totals.totalCostUSD).toBeCloseTo(0.0895, 6); + const summed = payload.sessions.reduce((s, r) => s + r.costUSD, 0); + expect(summed).toBeCloseTo(0.0895, 6); + }); + + it('carries session detail (turns, tools, dispatches, project) into the payload', () => { + const payload = buildPayload(analytics, costIndex, summary, { + rangeLabel: 'all', + projectFilter: 'all', + generatedAt: '2026-06-19T00:00:00.000Z', + }); + const main = payload.sessions.find((s) => s.sessionId.startsWith('1111'))!; + expect(main.turns).toBe(2); + expect(main.toolCallsTotal).toBe(4); // Edit x2, Bash, Read + expect(main.tools.length).toBeGreaterThan(0); + expect(main.dispatches?.length).toBe(2); // tech-analyst agent + brainstorming skill + expect(main.project).toBe('/Users/dev/projects/codemie-code'); + expect(main.branch).toBe('feature/analytics-otel-source'); + expect(main.netLines).toBe(90); // 120 - 30 + }); + + it('generates a self-contained HTML report file', () => { + const payload = buildPayload(analytics, costIndex, summary, { + rangeLabel: 'all', + projectFilter: 'all', + generatedAt: '2026-06-19T00:00:00.000Z', + }); + const out = join(mkdtempSync(join(tmpdir(), 'otel-report-')), 'report.html'); + generateReport(payload, out); + const html = readFileSync(out, 'utf-8'); + expect(html.length).toBeGreaterThan(5000); + expect(html.toLowerCase()).toContain(' { const map = sumUsageRecords(recs); expect(map.get('claude-sonnet-4-6')!.input).toBe(30); }); + + it('keeps the most complete row when Claude writes progressive records for one response', () => { + const recs = gatherDedupedUsageRecords( + 'claude', + claude([ + { timestamp: '2026-06-08T10:00:00Z', requestId: 'req-1', message: { id: 'msg-1', model: 'claude-sonnet-4-6', usage: { input_tokens: 3, output_tokens: 1, cache_read_input_tokens: 20_679, cache_creation_input_tokens: 2_682 } } }, + { timestamp: '2026-06-08T10:00:01Z', requestId: 'req-1', message: { id: 'msg-1', model: 'claude-sonnet-4-6', usage: { input_tokens: 3, output_tokens: 121, cache_read_input_tokens: 20_679, cache_creation_input_tokens: 2_682 } } }, + ]), + new Set() + ); + expect(recs).toHaveLength(1); + expect(recs[0].usage.output).toBe(121); + }); }); describe('extractClaudeUsageRecords — sub-agent transcripts', () => { diff --git a/src/cli/commands/analytics/cost/usage-readers.ts b/src/cli/commands/analytics/cost/usage-readers.ts index f43e44db..b938d527 100644 --- a/src/cli/commands/analytics/cost/usage-readers.ts +++ b/src/cli/commands/analytics/cost/usage-readers.ts @@ -73,16 +73,35 @@ export interface UsageRecord { usage: TokenUsage; } +function usageWeight(usage: TokenUsage): number { + return usage.input + usage.output + usage.cacheRead + usage.cacheCreation; +} + +function isMoreCompleteUsageRecord(candidate: UsageRecord, current: UsageRecord): boolean { + const candidateWeight = usageWeight(candidate.usage); + const currentWeight = usageWeight(current.usage); + if (candidateWeight !== currentWeight) { + return candidateWeight > currentWeight; + } + if (candidate.ts !== null && current.ts !== null && candidate.ts !== current.ts) { + return candidate.ts > current.ts; + } + return false; +} + /** * Extract one {@link UsageRecord} per Claude assistant message (skipping ``), * across the main transcript AND every sub-agent transcript in `parsed.subagents`. * Claude Code replays prior turns into resumed/forked session files, so the SAME API * response (same message.id + requestId) appears in multiple logs — callers dedupe by `key`. + * Claude Code can also write progressive JSONL rows for a streaming response before the + * final usage arrives; keep the most complete same-key row so partial chunks do not win. * Records are returned in chronological order when every record is timed; otherwise in * concatenation order (main transcript first, then sub-agent files in discovery order). */ export function extractClaudeUsageRecords(parsed: ParsedSession): UsageRecord[] { const records: UsageRecord[] = []; + const keyedRecords = new Map(); for (const messages of allMessageArrays(parsed)) { for (const raw of messages as ClaudeRawMessage[]) { const usage = raw.message?.usage; @@ -103,12 +122,27 @@ export function extractClaudeUsageRecords(parsed: ParsedSession): UsageRecord[] const key = id || reqId ? `${id ?? ''}::${reqId ?? ''}` : null; const parsedTs = raw.timestamp ? Date.parse(raw.timestamp) : NaN; const ts = Number.isFinite(parsedTs) ? parsedTs : null; - records.push({ + const record: UsageRecord = { key, ts, model, usage: { input, output, cacheRead, cacheCreation, cacheCreation1h, total: input + output + cacheRead + cacheCreation }, - }); + }; + if (key !== null) { + const current = keyedRecords.get(key); + if (!current) { + keyedRecords.set(key, record); + records.push(record); + } else if (isMoreCompleteUsageRecord(record, current)) { + const index = records.indexOf(current); + if (index !== -1) { + records[index] = record; + } + keyedRecords.set(key, record); + } + } else { + records.push(record); + } } } // Main and sub-agent records interleave in real time. Sort chronologically when every diff --git a/src/cli/commands/analytics/data-loader.ts b/src/cli/commands/analytics/data-loader.ts index 4f0d07cd..5f1edcb2 100644 --- a/src/cli/commands/analytics/data-loader.ts +++ b/src/cli/commands/analytics/data-loader.ts @@ -13,7 +13,7 @@ import { getCodemiePath } from '../../../utils/paths.js'; * Session start event (special record type) * Not part of MetricDelta, but stored in same JSONL file */ -interface SessionStartEvent { +export interface SessionStartEvent { recordId: string; type: 'session_start'; timestamp: number; @@ -30,7 +30,7 @@ interface SessionStartEvent { /** * Session end event (special record type) */ -interface SessionEndEvent { +export interface SessionEndEvent { recordId: string; type: 'session_end'; timestamp: number; diff --git a/src/cli/commands/analytics/index.ts b/src/cli/commands/analytics/index.ts index dad332f2..16a44ba7 100644 --- a/src/cli/commands/analytics/index.ts +++ b/src/cli/commands/analytics/index.ts @@ -4,19 +4,45 @@ import { Command } from 'commander'; import chalk from 'chalk'; -import { MetricsDataLoader } from './data-loader.js'; import { AnalyticsAggregator } from './aggregator.js'; import { AnalyticsFormatter } from './formatter.js'; import { AnalyticsExporter } from './exporter.js'; -import type { AnalyticsOptions, AnalyticsFilter } from './types.js'; -import type { SessionCostIndex, CostSummary } from './cost/types.js'; +import type { AnalyticsOptions, AnalyticsFilter, OtelCommandOptions } from './types.js'; import { logger } from '../../../utils/logger.js'; +import { SessionsSource } from './sources/sessions-source.js'; +import { OtelSource } from './sources/otel-source.js'; +import type { AnalyticsSource } from './sources/types.js'; export function createAnalyticsCommand(): Command { - const command = new Command('analytics'); + const command = new Command('analytics') + .description('Display aggregated metrics and analytics from sessions'); - command - .description('Display aggregated metrics and analytics from sessions') + // Default source: local CodeMie-tracked sessions + native agent logs. + applyCommonOptions(command) + .option('--no-scan-native', 'Skip native agent-log discovery (use only CodeMie-tracked sessions)') + .action((options: AnalyticsOptions) => runAnalytics(options, new SessionsSource())); + + // `codemie analytics otel --file ` — OTEL file source. + const otel = new Command('otel') + .description('Analytics from a flattened OTEL events file (otel-events.jsonl)'); + applyCommonOptions(otel) + .requiredOption('--file ', 'Path to the flattened OTEL events file') + .option('--user ', 'Scope to one user (native user.email or user.id)') + .action((_options: OtelCommandOptions, command: Command) => { + // The shared options (--report, --from, …) are registered on BOTH the parent and this + // subcommand, so commander binds them to the PARENT when they appear after `otel`. + // optsWithGlobals() merges parent + subcommand options into the full set the runner needs. + const opts = command.optsWithGlobals() as OtelCommandOptions; + return runAnalytics(opts, new OtelSource(opts.file, opts.user)); + }); + command.addCommand(otel); + + return command; +} + +/** Filter, report, export, and verbosity options shared by every analytics source. */ +function applyCommonOptions(command: Command): Command { + return command .option('--session ', 'Filter by session ID') .option('--project ', 'Filter by project path (basename, partial, or full path)') .option('--agent ', 'Filter by agent name (claude, gemini, etc.)') @@ -30,180 +56,157 @@ export function createAnalyticsCommand(): Command { .option('--report', 'Generate a self-contained HTML dashboard') .option('--open', 'Open the generated HTML report in the default browser') .option('--report-output ', 'HTML report output path (default: ./codemie-analytics-YYYY-MM-DD.html)') - .option('--report-format ', 'Report serialization: html, json, or both (default: html)') - .option('--no-scan-native', 'Skip native agent-log discovery (use only CodeMie-tracked sessions)') - .action(async (options: AnalyticsOptions) => { - try { - // Parse filter options - const filter = parseFilterOptions(options); - - // Load CodeMie-tracked sessions - const loader = new MetricsDataLoader(); - const rawSessions = loader.loadSessions(filter); - - // Discover native agent logs (plain `claude` etc. — not tracked by CodeMie) and merge, - // so the analytics reflect ALL usage. Deduped against tracked logs inside the loader. - if (options.scanNative !== false) { - try { - const { loadNativeSessions } = await import('./native-loader.js'); - const natives = (await loadNativeSessions(filter)).filter((s) => - loader.sessionMatchesFilter(s, filter) - ); - rawSessions.push(...natives); - } catch (error) { - logger.debug('Native session discovery failed (continuing with tracked sessions):', error); - } - } + .option('--report-format ', 'Report serialization: html, json, or both (default: html)'); +} - if (rawSessions.length === 0) { - console.log(chalk.yellow('\nNo sessions found matching the specified criteria.')); - console.log(chalk.dim('Run with different filters or check that metrics are being collected.\n')); - return; - } +async function runAnalytics(options: AnalyticsOptions, source: AnalyticsSource): Promise { + try { + const filter = parseFilterOptions(options); + const { rawSessions, cost } = await source.load({ filter, scanNative: options.scanNative }); - // A report needs cost, and cost must be computed BEFORE aggregation so that zero-delta - // sessions which still carry cost (empty metrics file but real usage in a correlated - // agent log) are retained instead of dropped as "empty". Price first, then aggregate. - const wantReport = Boolean(options.report || options.reportOutput || options.open || options.reportFormat); - const reportFormat = (options.reportFormat ?? 'html').toLowerCase(); - if (wantReport && reportFormat !== 'html' && reportFormat !== 'json' && reportFormat !== 'both') { - console.log(chalk.red('\n✗ Invalid report format. Use "html", "json", or "both".')); - return; - } - let costResult: { index: SessionCostIndex; summary: CostSummary } | undefined; - let keepSessionIds: Set | undefined; - if (wantReport) { - const { enrichCosts, realDeps } = await import('./cost/cost-enricher.js'); - costResult = await enrichCosts(rawSessions, realDeps); - // Retain zero-delta sessions that still carry real recoverable token usage — even - // when the model is unpriced (costUSD === 0) — so they are not silently dropped and - // can surface in the unpriced-models coverage. Filtering on cost would lose them. - keepSessionIds = new Set( - [...costResult.index.values()].filter((c) => c.tokens.total > 0).map((c) => c.sessionId) - ); - } + if (rawSessions.length === 0) { + console.log(chalk.yellow('\nNo sessions found matching the specified criteria.')); + console.log(chalk.dim('Run with different filters or check that metrics are being collected.\n')); + return; + } - // Aggregate data (normalize models unless --verbose flag is set) - const analytics = AnalyticsAggregator.aggregate(rawSessions, !options.verbose, keepSessionIds); + // A report needs cost computed BEFORE aggregation so zero-delta sessions that still carry + // real usage are retained instead of dropped as "empty". + const wantReport = Boolean(options.report || options.reportOutput || options.open || options.reportFormat); + const reportFormat = (options.reportFormat ?? 'html').toLowerCase(); + if (wantReport && reportFormat !== 'html' && reportFormat !== 'json' && reportFormat !== 'both') { + console.log(chalk.red('\n✗ Invalid report format. Use "html", "json", or "both".')); + return; + } - if (analytics.totalSessions === 0) { - console.log(chalk.yellow('\nNo analytics data available.')); - console.log(chalk.dim('Metrics collection may not have been enabled for these sessions.\n')); - return; - } + // Cost: authoritative from the source (OTEL) when present; otherwise enrich from correlated + // logs, but only when a report needs it. Retain zero-delta sessions with real token usage. + let costResult = cost; + let keepSessionIds: Set | undefined; + if (cost) { + keepSessionIds = new Set( + [...cost.index.values()].filter((c) => c.tokens.total > 0).map((c) => c.sessionId) + ); + } else if (wantReport) { + const { enrichCosts, realDeps } = await import('./cost/cost-enricher.js'); + costResult = await enrichCosts(rawSessions, realDeps); + keepSessionIds = new Set( + [...costResult.index.values()].filter((c) => c.tokens.total > 0).map((c) => c.sessionId) + ); + } - // Display results - const formatter = new AnalyticsFormatter(options.verbose); - formatter.displayRoot(analytics); - formatter.displayProjects(analytics.projects); - - // Export if requested - if (options.export) { - const format = options.export.toLowerCase(); - if (format !== 'json' && format !== 'csv') { - console.log(chalk.red('\n✗ Invalid export format. Use "json" or "csv".')); - return; - } - - const outputPath = options.output || AnalyticsExporter.getDefaultOutputPath(format, process.cwd()); - - if (format === 'json') { - AnalyticsExporter.exportJSON(analytics, outputPath); - } else { - AnalyticsExporter.exportCSV(analytics, outputPath); - } - } + // Aggregate data (normalize models unless --verbose flag is set) + const analytics = AnalyticsAggregator.aggregate(rawSessions, !options.verbose, keepSessionIds); + + if (analytics.totalSessions === 0) { + console.log(chalk.yellow('\nNo analytics data available.')); + console.log(chalk.dim('Metrics collection may not have been enabled for these sessions.\n')); + return; + } + + // Display results + const formatter = new AnalyticsFormatter(options.verbose); + formatter.displayRoot(analytics); + formatter.displayProjects(analytics.projects); + + // Export if requested + if (options.export) { + const format = options.export.toLowerCase(); + if (format !== 'json' && format !== 'csv') { + console.log(chalk.red('\n✗ Invalid export format. Use "json" or "csv".')); + return; + } + const outputPath = options.output || AnalyticsExporter.getDefaultOutputPath(format, process.cwd()); + if (format === 'json') { + AnalyticsExporter.exportJSON(analytics, outputPath); + } else { + AnalyticsExporter.exportCSV(analytics, outputPath); + } + } - // Generate the report if requested (--report-output and --open imply --report) - if (wantReport && costResult) { - const { buildPayload } = await import('./report/payload-builder.js'); - const { - generateReport, - generateReportJson, - getDefaultReportPath, - getDefaultReportJsonPath, - writeReportWithFallback - } = await import('./report/report-generator.js'); - - const { index: costIndex, summary } = costResult; - const payload = buildPayload(analytics, costIndex, summary, { - rangeLabel: options.last ?? (options.from || options.to ? 'custom' : 'all'), - projectFilter: options.project ?? 'all', - generatedAt: new Date().toISOString() - }); - - const cwd = process.cwd(); - let htmlPath: string | undefined; - let jsonPath: string | undefined; - // Default (cwd-derived) paths may relocate when cwd is unwritable (drive root, - // read-only volume); an explicit --report-output is honored as-is, errors included. - let htmlIsDefault = false; - let jsonIsDefault = false; - - if (reportFormat === 'both') { - // Derive a shared base (strip a trailing .html/.json from --report-output, if any) - // so the two artifacts are siblings and never collide, whatever extension was passed. - const base = options.reportOutput?.replace(/\.(html|json)$/i, ''); - htmlPath = base ? `${base}.html` : getDefaultReportPath(cwd); - jsonPath = base ? `${base}.json` : getDefaultReportJsonPath(cwd); - htmlIsDefault = jsonIsDefault = !base; - } else if (reportFormat === 'html') { - htmlPath = options.reportOutput || getDefaultReportPath(cwd); - htmlIsDefault = !options.reportOutput; - } else { - jsonPath = options.reportOutput || getDefaultReportJsonPath(cwd); - jsonIsDefault = !options.reportOutput; - } - - if (htmlPath) { - const result = writeReportWithFallback((p) => generateReport(payload, p), htmlPath, htmlIsDefault); - htmlPath = result.path; // keep --open and the success line pointed at the real file - if (result.relocatedFrom) { - console.log( - chalk.yellow(`\n! ${result.relocatedFrom} is not writable (drive root or read-only volume); using a writable location instead.`) - ); - } - console.log(chalk.green(`\n✓ HTML report written to: ${htmlPath}`)); - } - if (jsonPath) { - const result = writeReportWithFallback((p) => generateReportJson(payload, p), jsonPath, jsonIsDefault); - jsonPath = result.path; - if (result.relocatedFrom) { - console.log( - chalk.yellow(`\n! ${result.relocatedFrom} is not writable (drive root or read-only volume); using a writable location instead.`) - ); - } - console.log(chalk.green(`\n✓ JSON report written to: ${jsonPath}`)); - } - - const { sessions: totalReportSessions, pricedSessions } = payload.meta.totals; - if (pricedSessions < totalReportSessions) { - console.log( - chalk.dim( - ` Cost priced for ${pricedSessions}/${totalReportSessions} sessions (native agent logs required for the rest).` - ) - ); - } - - if (options.open) { - if (htmlPath) { - const { openUrlInBrowser } = await import('../../../utils/browser.js'); - await openUrlInBrowser(htmlPath); - } else { - console.log(chalk.dim(' --open ignored: no HTML produced (use --report-format html or both).')); - } - } + // Generate the report if requested (--report-output and --open imply --report) + if (wantReport && costResult) { + const { buildPayload } = await import('./report/payload-builder.js'); + const { + generateReport, + generateReportJson, + getDefaultReportPath, + getDefaultReportJsonPath, + writeReportWithFallback + } = await import('./report/report-generator.js'); + + const { index: costIndex, summary } = costResult; + const payload = buildPayload(analytics, costIndex, summary, { + rangeLabel: options.last ?? (options.from || options.to ? 'custom' : 'all'), + projectFilter: options.project ?? 'all', + generatedAt: new Date().toISOString() + }); + + const cwd = process.cwd(); + let htmlPath: string | undefined; + let jsonPath: string | undefined; + let htmlIsDefault = false; + let jsonIsDefault = false; + + if (reportFormat === 'both') { + const base = options.reportOutput?.replace(/\.(html|json)$/i, ''); + htmlPath = base ? `${base}.html` : getDefaultReportPath(cwd); + jsonPath = base ? `${base}.json` : getDefaultReportJsonPath(cwd); + htmlIsDefault = jsonIsDefault = !base; + } else if (reportFormat === 'html') { + htmlPath = options.reportOutput || getDefaultReportPath(cwd); + htmlIsDefault = !options.reportOutput; + } else { + jsonPath = options.reportOutput || getDefaultReportJsonPath(cwd); + jsonIsDefault = !options.reportOutput; + } + + if (htmlPath) { + const result = writeReportWithFallback((p) => generateReport(payload, p), htmlPath, htmlIsDefault); + htmlPath = result.path; + if (result.relocatedFrom) { + console.log( + chalk.yellow(`\n! ${result.relocatedFrom} is not writable (drive root or read-only volume); using a writable location instead.`) + ); } + console.log(chalk.green(`\n✓ HTML report written to: ${htmlPath}`)); + } + if (jsonPath) { + const result = writeReportWithFallback((p) => generateReportJson(payload, p), jsonPath, jsonIsDefault); + jsonPath = result.path; + if (result.relocatedFrom) { + console.log( + chalk.yellow(`\n! ${result.relocatedFrom} is not writable (drive root or read-only volume); using a writable location instead.`) + ); + } + console.log(chalk.green(`\n✓ JSON report written to: ${jsonPath}`)); + } - console.log(''); - } catch (error) { - logger.error('Analytics command failed:', error); - console.error(chalk.red(`\n✗ Failed to generate analytics: ${error instanceof Error ? error.message : String(error)}\n`)); - process.exit(1); + const { sessions: totalReportSessions, pricedSessions } = payload.meta.totals; + if (pricedSessions < totalReportSessions) { + console.log( + chalk.dim( + ` Cost priced for ${pricedSessions}/${totalReportSessions} sessions (native agent logs required for the rest).` + ) + ); } - }); - return command; + if (options.open) { + if (htmlPath) { + const { openUrlInBrowser } = await import('../../../utils/browser.js'); + await openUrlInBrowser(htmlPath); + } else { + console.log(chalk.dim(' --open ignored: no HTML produced (use --report-format html or both).')); + } + } + } + + console.log(''); + } catch (error) { + logger.error('Analytics command failed:', error); + console.error(chalk.red(`\n✗ Failed to generate analytics: ${error instanceof Error ? error.message : String(error)}\n`)); + process.exit(1); + } } /** diff --git a/src/cli/commands/analytics/otel-loader.ts b/src/cli/commands/analytics/otel-loader.ts new file mode 100644 index 00000000..726fdb9d --- /dev/null +++ b/src/cli/commands/analytics/otel-loader.ts @@ -0,0 +1,531 @@ +/** + * OTEL analytics source. + * + * Reads a flattened `otel-events.jsonl` (the contract produced by the + * codemie-claude-otel file sink / collector fan-out) and maps native Claude Code + * telemetry into the same {@link RawSessionData} + {@link SessionCostIndex} the rest + * of the analytics pipeline consumes, so `codemie analytics --source otel` renders the + * existing HTML report (incl. session detail) from OTEL data instead of local sessions. + * + * What each OTEL event contributes: + * - api_request → tokens + cost (authoritative `cost_usd`) + per-model rollup + * - tool_result → per-tool calls / success / failure + * - subagent_completed → agent dispatches (Timeline) + agentInvocations + * - skill_activated → skill dispatches (Timeline) + skillInvocations + * - user_prompt → turn count + session title (first prompt) + * - lines_of_code.count → net lines added/removed + * - claude.session.start (this plugin's own event) → cwd + git branch (Project/Branch) + * + * NOTE: native Claude Code emits NO cwd or git branch. Project/Branch are only available + * for sessions recorded with the codemie-claude-otel plugin active (joined by session id); + * native-only sessions fall back to "Unknown". + */ + +import { readFileSync, readdirSync, openSync, readSync, closeSync } from 'node:fs'; +import { join } from 'node:path'; +import { homedir } from 'node:os'; +import type { RawSessionData, SessionStartEvent, SessionEndEvent } from './data-loader.js'; +import type { MetricDelta } from '../../../agents/core/metrics/types.js'; +import type { + SessionCost, + SessionCostIndex, + CostSummary, + TokenUsage, + ModelCost, + CostSeriesPoint, + DispatchEvent, +} from './cost/types.js'; +import { MAX_SERIES_POINTS, MAX_DISPATCHES } from './cost/types.js'; +import { normalizeModelName } from './model-normalizer.js'; + +/** One flattened OTEL event line (see the contract in codemie-claude-otel/plugin/README.md). */ +export interface OtelEvent { + _type: 'log' | 'metric'; + ts: string; + scope?: string; + name: string; + attrs: Record; + resource: Record; + value?: unknown; + kind?: string; +} + +export interface OtelFilter { + /** Match against native user.email or user.id (goal #3: a particular user). */ + user?: string; + since?: Date; + until?: Date; +} + +interface CwdInfo { + cwd?: string; + branch?: string; +} + +function num(v: unknown): number { + const n = Number(v); + return Number.isFinite(n) ? n : 0; +} + +function attr(e: OtelEvent, k: string): unknown { + return (e.attrs || {})[k]; +} + +function sessionOf(e: OtelEvent): string { + return String(attr(e, 'session.id') || attr(e, 'session_id') || ''); +} + +function userKey(e: OtelEvent): string { + return String(attr(e, 'user.email') || attr(e, 'user.id') || ''); +} + +function isApiRequest(e: OtelEvent): boolean { + return e._type === 'log' && (e.name === 'api_request' || attr(e, 'event.name') === 'api_request'); +} + +/** Parse flattened JSONL text into events, skipping malformed lines. */ +export function parseOtelJsonl(text: string): OtelEvent[] { + const out: OtelEvent[] = []; + for (const line of text.split('\n')) { + const t = line.trim(); + if (!t) continue; + try { + const e = JSON.parse(t) as OtelEvent; + if (e && typeof e === 'object' && e.name) out.push(e); + } catch { + /* skip malformed line */ + } + } + return out; +} + +/** Select `api_request` log events, optionally filtered by user and time window. */ +export function filterApiRequests(events: OtelEvent[], filter: OtelFilter = {}): OtelEvent[] { + const sinceMs = filter.since ? filter.since.getTime() : -Infinity; + const untilMs = filter.until ? filter.until.getTime() : Infinity; + return events.filter((e) => { + if (!isApiRequest(e)) return false; + if (filter.user && userKey(e) !== filter.user) return false; + const ms = Date.parse(e.ts); + if (Number.isFinite(ms) && (ms < sinceMs || ms > untilMs)) return false; + return true; + }); +} + +function emptyTokens(): TokenUsage { + return { input: 0, output: 0, cacheRead: 0, cacheCreation: 0, cacheCreation1h: 0, total: 0 }; +} + +function addTokens(t: TokenUsage, e: OtelEvent): void { + t.input += num(attr(e, 'input_tokens')); + t.output += num(attr(e, 'output_tokens')); + t.cacheRead += num(attr(e, 'cache_read_tokens')); + t.cacheCreation += num(attr(e, 'cache_creation_tokens')); + t.total = t.input + t.output + t.cacheRead + t.cacheCreation; +} + +/** Build the cost index + run summary directly from native api_request events. */ +export function buildCostIndex(apiRequests: OtelEvent[]): { + index: SessionCostIndex; + summary: CostSummary; +} { + const index: SessionCostIndex = new Map(); + const perModelMap = new Map>(); + const seriesPts = new Map>(); + + for (const e of apiRequests) { + const sid = sessionOf(e); + if (!sid) continue; + if (!index.has(sid)) { + index.set(sid, { sessionId: sid, tokens: emptyTokens(), costUSD: 0, perModel: [], priced: true, hadLog: true }); + perModelMap.set(sid, new Map()); + seriesPts.set(sid, []); + } + const sc = index.get(sid)!; + addTokens(sc.tokens, e); + const cost = num(attr(e, 'cost_usd')); + sc.costUSD += cost; + + // Normalize so bedrock/converse spellings collapse onto the canonical model. + const model = normalizeModelName(String(attr(e, 'model') || '(unknown)')); + const mm = perModelMap.get(sid)!; + if (!mm.has(model)) mm.set(model, { tokens: emptyTokens(), costUSD: 0 }); + const mc = mm.get(model)!; + addTokens(mc.tokens, e); + mc.costUSD += cost; + + const ms = Date.parse(e.ts); + seriesPts.get(sid)!.push({ + t: Number.isFinite(ms) ? ms : 0, + cost, + tokens: num(attr(e, 'input_tokens')) + num(attr(e, 'output_tokens')) + num(attr(e, 'cache_read_tokens')) + num(attr(e, 'cache_creation_tokens')), + }); + } + + for (const [sid, sc] of index) { + const mm = perModelMap.get(sid)!; + sc.perModel = [...mm.entries()] + .map(([model, r]): ModelCost => ({ model, tokens: r.tokens, costUSD: r.costUSD, unpriced: false })) + .sort((a, b) => b.costUSD - a.costUSD); + sc.costSeries = downsampleSeries(seriesPts.get(sid)!); + } + + const summary: CostSummary = { + totalCostUSD: [...index.values()].reduce((s, c) => s + c.costUSD, 0), + pricedSessions: [...index.values()].filter((c) => c.priced).length, + totalSessions: index.size, + unpricedModels: [], + }; + return { index, summary }; +} + +function downsampleSeries(pts: Array<{ t: number; cost: number; tokens: number }>): CostSeriesPoint[] { + const sorted = [...pts].sort((a, b) => a.t - b.t); + let cost = 0; + let tokens = 0; + const series: CostSeriesPoint[] = sorted.map((p) => { + cost += p.cost; + tokens += p.tokens; + return { t: p.t, cost, tokens }; + }); + if (series.length <= MAX_SERIES_POINTS) return series; + const step = series.length / MAX_SERIES_POINTS; + const ds: CostSeriesPoint[] = []; + for (let i = 0; i < MAX_SERIES_POINTS; i++) ds.push(series[Math.floor(i * step)]); + ds.push(series[series.length - 1]); + return ds; +} + +/** + * Timed agent/skill invocations for the session-detail Timeline. + * + * A subagent_completed event fires at completion and carries duration_ms, so the dispatch + * spans [completion − duration, completion]. Its cost/tokens are recovered by attributing + * the subagent's own api_requests — those tagged `query_source = agent:*` — that fall inside + * that window (the subagent shares the parent session.id). Each api_request is attributed to at + * most one dispatch, so parallel subagents whose windows overlap do not both claim a shared + * request (which would double-count a dispatch's cost/tokens). Falls back to the event's + * total_tokens when no windowed agent request matches. + */ +export function buildDispatches(sessionEvents: OtelEvent[]): DispatchEvent[] { + const agentReqs = sessionEvents + .filter((e) => isApiRequest(e) && String(attr(e, 'query_source') || '').startsWith('agent:')) + .map((e) => ({ ms: Date.parse(e.ts), e, consumed: false })) + .filter((x) => Number.isFinite(x.ms)); + + const skillReqs = sessionEvents + .filter((e) => isApiRequest(e) && attr(e, 'skill.name')) + .map((e) => ({ ms: Date.parse(e.ts), name: String(attr(e, 'skill.name')), e })) + .filter((x) => Number.isFinite(x.ms)); + + const skillStarts = sessionEvents + .filter((e) => e.name === 'skill_activated') + .map((e) => ({ name: String(attr(e, 'skill.name') || '(skill)'), ms: Date.parse(e.ts) })) + .filter((x) => Number.isFinite(x.ms)); + + const out: DispatchEvent[] = []; + for (const e of sessionEvents) { + if (e.name === 'subagent_completed') { + const end = Date.parse(e.ts); + if (!Number.isFinite(end)) continue; + const durationMs = num(attr(e, 'duration_ms')); + const start = durationMs > 0 ? end - durationMs : end; + const tokens = emptyTokens(); + let costUSD = 0; + let matched = 0; + for (const ar of agentReqs) { + // Attribute each subagent api_request to at most ONE dispatch. Parallel subagents complete + // with overlapping [end − duration, end] windows; without consuming a matched request, the + // same agent:* api_request would be counted by every overlapping window — double-counting a + // dispatch's cost/tokens. Earlier-processed (≈ earlier-completing) dispatches claim first. + if (ar.consumed) continue; + if (ar.ms >= start && ar.ms <= end) { + addTokens(tokens, ar.e); + costUSD += num(attr(ar.e, 'cost_usd')); + ar.consumed = true; + matched += 1; + } + } + const d: DispatchEvent = { kind: 'agent', name: String(attr(e, 'agent_type') || '(agent)'), start, durationMs }; + if (matched > 0) { + d.tokens = tokens; + d.costUSD = costUSD; + } else { + const tt = num(attr(e, 'total_tokens')); + if (tt > 0) { + const t = emptyTokens(); + t.total = tt; + d.tokens = t; + } + } + out.push(d); + } else if (e.name === 'skill_activated') { + const start = Date.parse(e.ts); + if (!Number.isFinite(start)) continue; + const name = String(attr(e, 'skill.name') || '(skill)'); + // Window ends at the next activation of the SAME skill (else open-ended). + const nextSame = skillStarts + .filter((s) => s.name === name && s.ms > start) + .reduce((min, s) => Math.min(min, s.ms), Infinity); + const matched = skillReqs.filter((r) => r.name === name && r.ms >= start && r.ms < nextSame); + const d: DispatchEvent = { kind: 'skill', name, start, durationMs: 0 }; + if (matched.length) { + const tokens = emptyTokens(); + let costUSD = 0; + let lastEnd = start; + for (const { ms, e: req } of matched) { + addTokens(tokens, req); + costUSD += num(attr(req, 'cost_usd')); + lastEnd = Math.max(lastEnd, ms + num(attr(req, 'duration_ms'))); + } + d.durationMs = Math.max(0, lastEnd - start); + d.tokens = tokens; + d.costUSD = costUSD; + } + out.push(d); + } + } + out.sort((a, b) => a.start - b.start); + return out.slice(0, MAX_DISPATCHES); +} + +/** Read the first `bytes` of a file without loading the whole (transcripts can be large). */ +function readHead(file: string, bytes = 65536): string { + const fd = openSync(file, 'r'); + try { + const buf = Buffer.alloc(bytes); + const n = readSync(fd, buf, 0, bytes, 0); + return buf.toString('utf8', 0, n); + } finally { + closeSync(fd); + } +} + +function defaultProjectsDir(): string { + return join(homedir(), '.claude', 'projects'); +} + +let _transcriptIndex: { dir: string; map: Map } | null = null; +function transcriptIndex(projectsDir: string): Map { + if (_transcriptIndex && _transcriptIndex.dir === projectsDir) return _transcriptIndex.map; + const map = new Map(); + try { + for (const dir of readdirSync(projectsDir)) { + let files: string[]; + try { + files = readdirSync(join(projectsDir, dir)); + } catch { + continue; + } + for (const f of files) if (f.endsWith('.jsonl')) map.set(f.slice(0, -6), join(projectsDir, dir, f)); + } + } catch { + /* no projects dir */ + } + _transcriptIndex = { dir: projectsDir, map }; + return map; +} + +/** + * Recover the real project (cwd) and git branch for a session from its Claude Code + * transcript (~/.claude/projects//.jsonl) — the authoritative record of + * where Claude Code ran. Native OTEL carries neither. Returns undefined if no transcript. + */ +export function lookupTranscriptCwd(sessionId: string, projectsDir: string = defaultProjectsDir()): CwdInfo | undefined { + const file = transcriptIndex(projectsDir).get(sessionId); + if (!file) return undefined; + try { + for (const line of readHead(file).split('\n')) { + if (!line.trim()) continue; + let e: { cwd?: string; gitBranch?: string }; + try { + e = JSON.parse(line); + } catch { + continue; + } + if (e.cwd || e.gitBranch) return { cwd: e.cwd, branch: e.gitBranch }; + } + } catch { + /* unreadable */ + } + return undefined; +} + +/** + * Synthesize MetricDelta records for a session so the aggregator computes tools, skills, + * agents, turns, title, and net lines. Turns are derived from user_prompt events (one delta + * each); all session activity is aggregated onto the first delta (the report shows session + * totals, not per-turn splits). A session with activity but no captured prompts still gets + * one delta so its tools/agents surface. + */ +export function buildDeltas(sessionEvents: OtelEvent[], cwd?: CwdInfo): MetricDelta[] { + const sid = sessionEvents.length ? sessionOf(sessionEvents[0]) : ''; + const prompts = sessionEvents + .filter((e) => e.name === 'user_prompt') + .sort((a, b) => Date.parse(a.ts) - Date.parse(b.ts)); + + // Aggregate session activity. + const tools: Record = {}; + const toolStatus: Record = {}; + const agentInvocations: Record = {}; + const skillInvocations: Record = {}; + const models = new Set(); + let linesAdded = 0; + let linesRemoved = 0; + + for (const e of sessionEvents) { + if (e.name === 'tool_result') { + const tn = String(attr(e, 'tool_name') || '(unknown)'); + tools[tn] = (tools[tn] || 0) + 1; + if (!toolStatus[tn]) toolStatus[tn] = { success: 0, failure: 0 }; + if (String(attr(e, 'success')) === 'true') toolStatus[tn].success += 1; + else toolStatus[tn].failure += 1; + } else if (e.name === 'subagent_completed') { + const a = String(attr(e, 'agent_type') || '(agent)'); + agentInvocations[a] = (agentInvocations[a] || 0) + 1; + } else if (e.name === 'skill_activated') { + const s = String(attr(e, 'skill.name') || '(skill)'); + skillInvocations[s] = (skillInvocations[s] || 0) + 1; + } else if (isApiRequest(e)) { + if (attr(e, 'model')) models.add(String(attr(e, 'model'))); + } else if (e._type === 'metric' && e.name === 'claude_code.lines_of_code.count') { + const v = num(e.value); + if (attr(e, 'type') === 'added') linesAdded += v; + else if (attr(e, 'type') === 'removed') linesRemoved += v; + } + } + + const hasActivity = + Object.keys(tools).length > 0 || + Object.keys(agentInvocations).length > 0 || + Object.keys(skillInvocations).length > 0 || + linesAdded > 0 || + linesRemoved > 0 || + models.size > 0; + + const turnCount = prompts.length || (hasActivity ? 1 : 0); + if (turnCount === 0) return []; + + const branch = cwd?.branch; + const deltas: MetricDelta[] = []; + for (let i = 0; i < turnCount; i++) { + const p = prompts[i]; + deltas.push({ + recordId: `${sid}-otel-${i}`, + sessionId: sid, + agentSessionId: sid, + timestamp: p ? Date.parse(p.ts) : (sessionEvents[0] ? Date.parse(sessionEvents[0].ts) : 0), + gitBranch: branch, + userPrompts: p ? [{ count: 1, text: String(attr(p, 'prompt') || '') }] : undefined, + syncStatus: 'synced', + syncAttempts: 0, + }); + } + + // Attach all aggregated activity to the first delta — totals are what the report shows. + const first = deltas[0]; + if (Object.keys(tools).length) first.tools = tools; + if (Object.keys(toolStatus).length) first.toolStatus = toolStatus; + if (Object.keys(agentInvocations).length) first.agentInvocations = agentInvocations; + if (Object.keys(skillInvocations).length) first.skillInvocations = skillInvocations; + if (models.size) first.models = [...models]; + if (linesAdded > 0 || linesRemoved > 0) { + // OTEL's lines_of_code metric has no per-file path, so net lines are attributed to one + // synthetic aggregate path (the aggregator skips path-less ops). Consequence: a session's + // "files changed" reads 1 — net lines are accurate, per-file breakdown is not available. + first.fileOperations = [{ type: 'edit', path: '', linesAdded, linesRemoved }]; + } + return deltas; +} + +/** Map of session id → cwd/branch, from this plugin's own session.start events. */ +function buildCwdMap(events: OtelEvent[]): Map { + const m = new Map(); + for (const e of events) { + if (attr(e, 'event_type') === 'claude.session.start' || attr(e, 'cwd')) { + const sid = sessionOf(e); + if (sid && !m.has(sid)) { + m.set(sid, { cwd: attr(e, 'cwd') ? String(attr(e, 'cwd')) : undefined, branch: attr(e, 'git_branch') ? String(attr(e, 'git_branch')) : undefined }); + } + } + } + return m; +} + +/** Build one RawSessionData (with synthesized deltas) for a session's events. */ +export function buildRawSession(sessionId: string, sessionEvents: OtelEvent[], cwd: CwdInfo | undefined, projectLabel: string): RawSessionData { + const times = sessionEvents.map((e) => Date.parse(e.ts)).filter(Number.isFinite); + const startTime = times.length ? Math.min(...times) : 0; + const endTime = times.length ? Math.max(...times) : 0; + const workingDirectory = cwd?.cwd || projectLabel; + const startEvent: SessionStartEvent = { + recordId: sessionId, + type: 'session_start', + timestamp: startTime, + codeMieSessionId: sessionId, + agentName: 'claude', + syncStatus: 'synced', + data: { provider: 'claude', workingDirectory, startTime }, + }; + const endEvent: SessionEndEvent = { + recordId: `${sessionId}-end`, + type: 'session_end', + timestamp: endTime, + codeMieSessionId: sessionId, + agentName: 'claude', + syncStatus: 'synced', + data: { endTime, duration: Math.max(0, endTime - startTime), totalTurns: 0 }, + }; + return { sessionId, startEvent, endEvent, deltas: buildDeltas(sessionEvents, cwd) }; +} + +/** Load + map an OTEL events file into the analytics pipeline inputs. */ +export function loadOtelSessions(opts: { + file: string; + filter?: OtelFilter; + projectLabel?: string; + /** Override the cwd/branch source (defaults to the Claude Code transcript join). */ + cwdLookup?: (sessionId: string) => CwdInfo | undefined; +}): { + rawSessions: RawSessionData[]; + costIndex: SessionCostIndex; + summary: CostSummary; +} { + const projectLabel = opts.projectLabel ?? 'Unknown'; + const cwdLookup = opts.cwdLookup ?? ((sid: string) => lookupTranscriptCwd(sid)); + const events = parseOtelJsonl(readFileSync(opts.file, 'utf-8')); + + // Owned sessions = those with an api_request matching the user/time filter. + const apiRequests = filterApiRequests(events, opts.filter); + const ownedSids = new Set(apiRequests.map(sessionOf).filter(Boolean)); + + // Group ALL events for owned sessions (deltas/dispatches need the full event variety). + const bySession = new Map(); + for (const e of events) { + const sid = sessionOf(e); + if (sid && ownedSids.has(sid)) { + if (!bySession.has(sid)) bySession.set(sid, []); + bySession.get(sid)!.push(e); + } + } + + const pluginCwdMap = buildCwdMap(events); // fallback for sessions recorded with this plugin + const { index, summary } = buildCostIndex(apiRequests); + + // Attach per-session dispatches (Timeline) onto the cost index. + for (const [sid, evs] of bySession) { + const sc: SessionCost | undefined = index.get(sid); + if (sc) { + const d = buildDispatches(evs); + if (d.length) sc.dispatches = d; + } + } + + const rawSessions = [...bySession.entries()].map(([sid, evs]) => { + // Project/branch: transcript join first (authoritative, works for all sessions), + // then this plugin's session.start event as a fallback. + const cwd = cwdLookup(sid) ?? pluginCwdMap.get(sid); + return buildRawSession(sid, evs, cwd, projectLabel); + }); + return { rawSessions, costIndex: index, summary }; +} diff --git a/src/cli/commands/analytics/report/client/app.js b/src/cli/commands/analytics/report/client/app.js index 6486e1ef..713a98f0 100644 --- a/src/cli/commands/analytics/report/client/app.js +++ b/src/cli/commands/analytics/report/client/app.js @@ -802,18 +802,21 @@ var list = fs.slice().sort(function (a, b) { return b.startTime - a.startTime; }); if (q) { var ql = q.toLowerCase(); - list = list.filter(function (s) { return (s.sessionId + ' ' + s.agentName + ' ' + s.project + ' ' + s.branch).toLowerCase().indexOf(ql) >= 0; }); + list = list.filter(function (s) { return (s.sessionId + ' ' + s.agentName + ' ' + s.project + ' ' + s.branch + ' ' + (s.title || '')).toLowerCase().indexOf(ql) >= 0; }); } var shown = list.slice(0, 300); holder.innerHTML = tableHTML( - ['Date', 'Agent', 'Project', 'Branch', 'Turns', 'Net lines', 'Input', 'Output', 'Cached', 'Cost'], + ['Date', 'Prompt', 'Agent', 'Project', 'Branch', 'Turns', 'Net lines', 'Input', 'Output', 'Cached', 'Cost'], shown.map(function (s) { + var branchCell = s.branch ? '' + esc(s.branch) + '' : '—'; + var promptCell = '' + esc(truncStr(s.title || '—', 80)) + ''; return [new Date(s.startTime).toISOString().slice(0, 16).replace('T', ' '), + promptCell, '' + esc(s.agentName) + '', - '' + esc(shortPath(s.project)) + '', esc(s.branch || '—'), + '' + esc(shortPath(s.project)) + '', branchCell, fmtNum(s.turns), fmtNum(s.netLines), fmtTokens(tkIn(s)), fmtTokens(tkOut(s)), fmtTokens(tkCached(s)), fmtUSD(s.costUSD)]; }), - [false, false, false, false, true, true, true, true, true, true], + [false, false, false, false, false, true, true, true, true, true, true], shown.map(function (s) { return 'class="clickable" data-session="' + esc(s.sessionId) + '"'; })); if (list.length > 300) holder.appendChild(el('p', 'text-muted', 'Showing first 300 of ' + list.length + '.')); } @@ -900,7 +903,7 @@ return Math.floor(ms / 3600000) + 'h ' + Math.round((ms % 3600000) / 60000) + 'm'; } // Side panel content for a selected timeline step. - function tlDetailEl(d) { + function tlDetailEl(d, sessionStart) { var panel = el('div', ''); if (!d) { panel.appendChild(el('div', 'tl-side-empty', 'Click a timeline step to see details.')); @@ -914,10 +917,12 @@ panel.appendChild(nameEl); var hasCost = d.costUSD != null; var hasTok = d.tokens != null; + var startOffset = sessionStart != null && d.start != null ? Math.max(0, d.start - sessionStart) : null; panel.appendChild(statsEl([ ['Cost', hasCost ? fmtUSD(d.costUSD) : '—', ''], ['Tokens', hasTok ? fmtTokens(d.tokens.total) : '—', ''], ['Duration', fmtTimelineDuration(d.durationMs), ''], + ['Started', startOffset != null ? fmtTimelineDuration(startOffset) : '—', 'into session'], ])); if (hasTok) { var tok = d.tokens; @@ -958,35 +963,45 @@ } return panel; } - // Gantt of top-level agent dispatches: session span on top, each agent bar positioned - // by wall-clock start time relative to the session start (proportional scaling). - // Skills and commands are excluded — they have no separate transcript or cost to show. - // min-width 28px (CSS) keeps short agents visible and clickable without distorting the scale. + // Gantt of top-level dispatches: session span on top, then every agent/skill/command + // dispatch positioned by wall-clock start time across the activity window, so each bar + // sits where it actually ran. Short skills and 0ms commands fall back to the CSS + // min-width (28px) so they stay visible as markers without distorting the time scale. function timelineEl(s) { - var dispatches = (s.dispatches || []).filter(function (d) { return d.kind === 'agent'; }).sort(function (a, b) { return a.start - b.start; }); - // Scale bars to the agent activity window, not the full session duration. - // Agent bars fill the track proportionally relative to each other; absolute - // start offset (vs. session start) is shown in the side panel detail view. - var actStart = dispatches.length ? dispatches.reduce(function (m, d) { return Math.min(m, d.start); }, Infinity) : s.startTime; - var actEnd = dispatches.length ? dispatches.reduce(function (m, d) { return Math.max(m, d.start + (d.durationMs || 0)); }, -Infinity) : s.startTime + s.durationMs; - var ganttSpan = Math.max(1, actEnd - actStart); + var dispatches = (s.dispatches || []).slice().sort(function (a, b) { return a.start - b.start; }); + // Window = union of the tracked session span and the dispatch activity span. Resumed or + // compacted sessions can carry transcript dispatches that fall OUTSIDE the tracked + // start→end window; scaling to the session window alone would clamp them all to left:0% + // (single-column pile-up). The union positions every bar correctly and collapses back to + // exactly the session window for the normal case (dispatches fully inside it). + var winStart = s.startTime; + var winEnd = s.startTime + (s.durationMs || 0); + dispatches.forEach(function (d) { + if (d.start < winStart) winStart = d.start; + var end = d.start + (d.durationMs || 0); + if (end > winEnd) winEnd = end; + }); + var ganttSpan = Math.max(1, winEnd - winStart); var container = el('div', 'tl-container'); var gantt = el('div', 'tl-gantt timeline'); var sidePane = el('div', 'tl-side'); - var selected = dispatches[0] || null; - sidePane.appendChild(tlDetailEl(selected)); + var selected = dispatches.filter(function (d) { return d.kind === 'agent'; })[0] || dispatches[0] || null; + sidePane.appendChild(tlDetailEl(selected, winStart)); function selectDispatch(d, barEl) { gantt.querySelectorAll('.tl-bar[data-dispatch]').forEach(function(b) { b.style.outline = b === barEl ? '2px solid rgba(255,255,255,0.55)' : 'none'; }); sidePane.innerHTML = ''; - sidePane.appendChild(tlDetailEl(d)); + sidePane.appendChild(tlDetailEl(d, winStart)); } - gantt.appendChild(tlRow('session', '#259F4C', 0, 100, fmtTimelineDuration(s.durationMs), fmtUSD(s.costUSD), true)); + // Session bar spans the full window (the activity envelope). Its label shows the envelope + // span — equal to the tracked duration in the normal case, but revealing the true span when + // the tracked duration under-counts (e.g. dispatches predating a post-compaction window). + gantt.appendChild(tlRow('session', '#259F4C', 0, 100, fmtTimelineDuration(ganttSpan), fmtUSD(s.costUSD), true)); var occ = {}; dispatches.forEach(function (d) { @@ -994,7 +1009,7 @@ var total = dispatches.filter(function (x) { return x.kind === d.kind && x.name === d.name; }).length; occ[key] = (occ[key] || 0) + 1; var label = d.kind + ' · ' + d.name + (total > 1 ? ' #' + occ[key] : ''); - var leftPct = Math.max(0, (d.start - actStart) / ganttSpan * 100); + var leftPct = Math.max(0, (d.start - winStart) / ganttSpan * 100); var wPct = (d.durationMs || 0) / ganttSpan * 100; if (leftPct + wPct > 100) wPct = Math.max(0, 100 - leftPct); var barText = d.costUSD != null && wPct >= 8 ? fmtUSD(d.costUSD) : ''; @@ -1035,8 +1050,24 @@ var metaBits = [s.agentName, (s.models && s.models[0]) || null, shortPath(s.project), s.branch].filter(Boolean); htxt.appendChild(el('div', 'modal-meta', metaBits.map(function (b) { return esc(b); }).join(' · '))); head.appendChild(htxt); + var headBtns = el('div'); headBtns.style.cssText = 'display:flex;gap:6px;align-items:center;flex-shrink:0;'; + var exportBtn = el('button', 'modal-export', '↓ JSON'); + exportBtn.setAttribute('aria-label', 'Export session as JSON'); + exportBtn.setAttribute('title', 'Export session details as JSON'); + exportBtn.addEventListener('click', function () { + var json = JSON.stringify(s, null, 2); + var blob = new Blob([json], { type: 'application/json' }); + var url = URL.createObjectURL(blob); + var a = document.createElement('a'); + var date = new Date(s.startTime).toISOString().slice(0, 10); + a.href = url; a.download = 'session-' + date + '-' + String(s.sessionId).slice(0, 8) + '.json'; + document.body.appendChild(a); a.click(); document.body.removeChild(a); + URL.revokeObjectURL(url); + }); + headBtns.appendChild(exportBtn); var close = el('button', 'modal-close', '✕'); close.setAttribute('aria-label', 'Close'); close.addEventListener('click', closeSessionModal); - head.appendChild(close); + headBtns.appendChild(close); + head.appendChild(headBtns); modal.appendChild(head); var body = el('div', 'modal-body'); @@ -1101,7 +1132,7 @@ // Timeline — Gantt of all top-level agent, skill, and command dispatches. var hasDispatches = (s.dispatches || []).length > 0; var hasCostData = (s.dispatches || []).some(function (d) { return d.kind === 'agent' && d.costUSD != null; }); - var tlSubtitle = hasDispatches ? (hasCostData ? 'click a step for cost, token & timing details · long idle gaps compressed' : 'click a step for timing details · long idle gaps compressed') : ''; + var tlSubtitle = hasDispatches ? (hasCostData ? 'click a step for cost, token & timing details · positioned across the session' : 'click a step for timing details · positioned across the session') : ''; var tlCard = card('Timeline', tlSubtitle); if (hasDispatches) { tlCard._body.appendChild(timelineEl(s)); diff --git a/src/cli/commands/analytics/report/template.html b/src/cli/commands/analytics/report/template.html index 9bb9e25b..9aa4c6b7 100644 --- a/src/cli/commands/analytics/report/template.html +++ b/src/cli/commands/analytics/report/template.html @@ -136,13 +136,15 @@ .proj-sessions .table td:first-child { white-space: normal; word-break: break-word; } /* session-detail modal */ - .modal-overlay { position: fixed; inset: 0; background: rgba(0,0,0,0.62); backdrop-filter: blur(4px); display: flex; align-items: center; justify-content: center; z-index: var(--z-modal, 50); padding: 24px; } + .modal-overlay { position: fixed; inset: 0; background: rgba(0,0,0,0.72); display: flex; align-items: center; justify-content: center; z-index: var(--z-modal, 50); padding: 24px; } .modal { background: var(--color-bg-card); border: 1px solid var(--color-border-structural); border-radius: var(--radius-xl, 12px); box-shadow: var(--shadow-lg); width: 100%; max-width: 880px; max-height: 88vh; display: flex; flex-direction: column; overflow: hidden; } .modal-head { display: flex; align-items: flex-start; justify-content: space-between; gap: 12px; padding: 16px 20px; border-bottom: 1px solid var(--color-border-structural); } .modal-title { font-size: 15px; font-weight: 700; color: var(--color-text-primary); margin: 0 0 3px; } .modal-meta { font-size: 12px; color: var(--color-text-muted); text-transform: capitalize; } .modal-close { background: transparent; border: 1px solid var(--color-border-structural); color: var(--color-text-muted); border-radius: 8px; height: 30px; width: 30px; cursor: pointer; font-size: 12px; flex-shrink: 0; } .modal-close:hover { color: var(--color-text-primary); background: var(--color-bg-tertiary); } + .modal-export { background: transparent; border: 1px solid var(--color-border-structural); color: var(--color-text-muted); border-radius: 8px; height: 30px; padding: 0 10px; cursor: pointer; font-size: 12px; flex-shrink: 0; white-space: nowrap; font-family: inherit; } + .modal-export:hover { color: var(--color-primary, #7C5CFC); background: var(--color-bg-tertiary); border-color: var(--color-primary, #7C5CFC); } .modal-body { padding: 18px 20px; overflow-y: auto; } .modal-body .card { margin-bottom: 14px; } /* light, borderless stat grid (replaces the heavy box-in-box KPIs) */ @@ -180,13 +182,15 @@ .tl-tok-seg { border-radius: 2px; } /* session-detail modal */ - .modal-overlay { position: fixed; inset: 0; background: rgba(0,0,0,0.62); backdrop-filter: blur(4px); display: flex; align-items: center; justify-content: center; z-index: var(--z-modal, 50); padding: 24px; } + .modal-overlay { position: fixed; inset: 0; background: rgba(0,0,0,0.72); display: flex; align-items: center; justify-content: center; z-index: var(--z-modal, 50); padding: 24px; } .modal { background: var(--color-bg-card); border: 1px solid var(--color-border-structural); border-radius: var(--radius-xl, 12px); box-shadow: var(--shadow-lg); width: 100%; max-width: 880px; max-height: 88vh; display: flex; flex-direction: column; overflow: hidden; } .modal-head { display: flex; align-items: flex-start; justify-content: space-between; gap: 12px; padding: 16px 20px; border-bottom: 1px solid var(--color-border-structural); } .modal-title { font-size: 15px; font-weight: 700; color: var(--color-text-primary); margin: 0 0 3px; } .modal-meta { font-size: 12px; color: var(--color-text-muted); text-transform: capitalize; } .modal-close { background: transparent; border: 1px solid var(--color-border-structural); color: var(--color-text-muted); border-radius: 8px; height: 30px; width: 30px; cursor: pointer; font-size: 12px; flex-shrink: 0; } .modal-close:hover { color: var(--color-text-primary); background: var(--color-bg-tertiary); } + .modal-export { background: transparent; border: 1px solid var(--color-border-structural); color: var(--color-text-muted); border-radius: 8px; height: 30px; padding: 0 10px; cursor: pointer; font-size: 12px; flex-shrink: 0; white-space: nowrap; font-family: inherit; } + .modal-export:hover { color: var(--color-primary, #7C5CFC); background: var(--color-bg-tertiary); border-color: var(--color-primary, #7C5CFC); } .modal-body { padding: 18px 20px; overflow-y: auto; } .modal-body .card { margin-bottom: 14px; } /* light, borderless stat grid (replaces the heavy box-in-box KPIs) */ diff --git a/src/cli/commands/analytics/sources/otel-source.ts b/src/cli/commands/analytics/sources/otel-source.ts new file mode 100644 index 00000000..3b048196 --- /dev/null +++ b/src/cli/commands/analytics/sources/otel-source.ts @@ -0,0 +1,47 @@ +import { existsSync } from 'node:fs'; +import { AnalyticsSourceError } from '../../../../utils/errors.js'; +import { MetricsDataLoader } from '../data-loader.js'; +import type { AnalyticsFilter } from '../types.js'; +import type { AnalyticsSource, SourceLoadOptions, SourceResult } from './types.js'; + +/** + * OTEL source: a flattened otel-events.jsonl file. Cost is authoritative (native + * `cost_usd`), so it is returned up-front and bypasses the cost-enricher. + */ +export class OtelSource implements AnalyticsSource { + constructor(private readonly file: string, private readonly user?: string) {} + + async load(opts: SourceLoadOptions): Promise { + if (!existsSync(this.file)) { + throw new AnalyticsSourceError(`OTEL file not found: ${this.file}`); + } + const { loadOtelSessions } = await import('../otel-loader.js'); + const res = loadOtelSessions({ + file: this.file, + filter: { user: this.user, since: opts.filter.fromDate, until: opts.filter.toDate }, + }); + + // Honor the structural CLI filters OTEL can evaluate from session metadata. `--user` and the + // time window are already applied inside loadOtelSessions (on api_request user/ts); apply the + // rest here so they are not silently ignored — and so a report is never mislabeled (e.g. + // `--project foo` must not produce a "project: foo" report that still contains every project). + // The structural filter is date-free: re-applying the window would conflict with the + // api_request-time filter already done. `--session` is matched explicitly because + // sessionMatchesFilter does not cover session id. The cost index is left intact — entries for + // dropped sessions are inert (downstream only reads cost for sessions present in rawSessions, + // and keepSessionIds cannot resurrect a session the aggregator never sees). + const structural: AnalyticsFilter = { + ...(opts.filter.agentName && { agentName: opts.filter.agentName }), + ...(opts.filter.projectPattern && { projectPattern: opts.filter.projectPattern }), + ...(opts.filter.branch && { branch: opts.filter.branch }), + }; + const loader = new MetricsDataLoader(); + const rawSessions = res.rawSessions.filter( + (s) => + (!opts.filter.sessionId || s.sessionId === opts.filter.sessionId) && + loader.sessionMatchesFilter(s, structural) + ); + + return { rawSessions, cost: { index: res.costIndex, summary: res.summary } }; + } +} diff --git a/src/cli/commands/analytics/sources/sessions-source.ts b/src/cli/commands/analytics/sources/sessions-source.ts new file mode 100644 index 00000000..96e56b5a --- /dev/null +++ b/src/cli/commands/analytics/sources/sessions-source.ts @@ -0,0 +1,30 @@ +import { MetricsDataLoader } from '../data-loader.js'; +import { logger } from '../../../../utils/logger.js'; +import type { AnalyticsSource, SourceLoadOptions, SourceResult } from './types.js'; + +/** + * Local-session source: ~/.codemie tracked sessions merged with discovered native agent + * logs (plain `claude` etc.). Cost is omitted here and derived by the cost-enricher only + * when a report is requested, matching the original analytics behavior. + */ +export class SessionsSource implements AnalyticsSource { + async load(opts: SourceLoadOptions): Promise { + const loader = new MetricsDataLoader(); + const rawSessions = loader.loadSessions(opts.filter); + + // Discover native agent logs (not tracked by CodeMie) and merge so analytics reflect + // ALL usage. Deduped against tracked logs inside the loader. + if (opts.scanNative !== false) { + try { + const { loadNativeSessions } = await import('../native-loader.js'); + const natives = (await loadNativeSessions(opts.filter)).filter((s) => + loader.sessionMatchesFilter(s, opts.filter) + ); + rawSessions.push(...natives); + } catch (error) { + logger.debug('Native session discovery failed (continuing with tracked sessions):', error); + } + } + return { rawSessions }; + } +} diff --git a/src/cli/commands/analytics/sources/types.ts b/src/cli/commands/analytics/sources/types.ts new file mode 100644 index 00000000..47e4c5cc --- /dev/null +++ b/src/cli/commands/analytics/sources/types.ts @@ -0,0 +1,37 @@ +/** + * Analytics source seam. + * + * Every data backend implements {@link AnalyticsSource}: it loads its data and returns the + * same {@link SourceResult} the aggregate → report pipeline consumes, so the CLI stays + * agnostic to where the data came from. + * + * Today: {@link SessionsSource} (local ~/.codemie + native logs) and OtelSource + * (flattened otel-events.jsonl). Future remote backends (Prometheus, Loki, Langfuse) become + * one new `AnalyticsSource` + one thin `analytics ` subcommand each, with connection + * URL + auth resolved from the `.codemie` config/profile (never CLI flags) — e.g. + * "analyticsSources": { "prod-langfuse": { "type": "langfuse", "url": "…", "authEnv": "LANGFUSE_TOKEN" } } + * where `authEnv` names the env var holding the token. No connector code exists yet. + */ +import type { RawSessionData } from '../data-loader.js'; +import type { AnalyticsFilter } from '../types.js'; +import type { SessionCostIndex, CostSummary } from '../cost/types.js'; + +export interface SourceLoadOptions { + filter: AnalyticsFilter; + /** Sessions source only: skip native agent-log discovery. Ignored by other sources. */ + scanNative?: boolean; +} + +export interface SourceResult { + rawSessions: RawSessionData[]; + /** + * Authoritative cost, present only when the source carries it in its own data + * (e.g. OTEL `cost_usd`). Omitted when cost must be derived later from correlated logs + * (local sessions → cost-enricher); the runner enriches in that case. + */ + cost?: { index: SessionCostIndex; summary: CostSummary }; +} + +export interface AnalyticsSource { + load(opts: SourceLoadOptions): Promise; +} diff --git a/src/cli/commands/analytics/types.ts b/src/cli/commands/analytics/types.ts index 0492b252..ad81c704 100644 --- a/src/cli/commands/analytics/types.ts +++ b/src/cli/commands/analytics/types.ts @@ -249,3 +249,11 @@ export interface AnalyticsOptions { /** When false (via --no-scan-native), skip native-log discovery and use tracked sessions only. */ scanNative?: boolean; } + +/** Options for the `analytics otel` subcommand: the shared base plus OTEL-specific flags. */ +export interface OtelCommandOptions extends AnalyticsOptions { + /** Path to the flattened OTEL events file (required). */ + file: string; + /** Scope OTEL analytics to one user (matches native user.email or user.id). */ + user?: string; +} diff --git a/src/cli/commands/hook.ts b/src/cli/commands/hook.ts index 89b05300..0c3207ad 100644 --- a/src/cli/commands/hook.ts +++ b/src/cli/commands/hook.ts @@ -676,6 +676,40 @@ async function createSessionRecord(event: SessionStartEvent, sessionId: string, const { SessionStore } = await import('../../agents/core/session/SessionStore.js'); const sessionStore = new SessionStore(); + // A SessionStart can RE-ENTER an already-tracked LIVE session: `compact` fires SessionStart for + // the SAME CODEMIE_SESSION_ID without ending the session, so the primary {id}.json is still on + // disk and loadSession returns the live record. Rebuilding it from scratch would reset startTime + // to "now" and zero the accumulated activeDurationMs — under-reporting the session's true span + // and active time (startTime is also read by the metrics aggregator and sent to the backend). + // Preserve the existing record's accumulated state in place; only refresh status and correlation + // to the current transcript from fields the event actually provides. + // + // Guard on a LIVE record (active + no endTime): `clear` (and exit/logout) first fire SessionEnd, + // which marks the record completed (endTime set) and renames {id}.json → completed_{id}.json. + // Because loadSession transparently falls back to the completed_ file, an unguarded re-entry + // would RESURRECT a finished session — inheriting its stale startTime/activeDurationMs and + // leaving an `active` record that still carries a past endTime. A post-clear session must start + // fresh, so completed records fall through to the fresh-build below. (resume runs in a new CLI + // process with a fresh CODEMIE_SESSION_ID, so it never reaches this branch.) + const existing = await sessionStore.loadSession(sessionId); + if (existing && existing.status === 'active' && existing.endTime === undefined) { + existing.status = 'active'; + if (gitBranch) existing.gitBranch = gitBranch; + if (remoteRepository) existing.repository = remoteRepository; + existing.correlation = { + ...existing.correlation, + status: 'matched', + ...(event.session_id && { agentSessionId: event.session_id }), + ...(event.transcript_path && { agentSessionFile: event.transcript_path }), + }; + await sessionStore.saveSession(existing); + logger.info( + `[hook:SessionStart] Session re-entered (source=${event.source}): preserved ` + + `startTime=${existing.startTime} activeDurationMs=${existing.activeDurationMs}` + ); + return; + } + // Create session record with correlation already matched const session = { sessionId, diff --git a/src/utils/errors.ts b/src/utils/errors.ts index 8958aa15..44e5e55d 100644 --- a/src/utils/errors.ts +++ b/src/utils/errors.ts @@ -40,6 +40,13 @@ export class PathSecurityError extends CodeMieError { } } +export class AnalyticsSourceError extends CodeMieError { + constructor(message: string) { + super(message); + this.name = 'AnalyticsSourceError'; + } +} + /** * npm error codes for categorizing failures */ From 0a50a0d61e1ab852937bc233b61cb2a1a753091f Mon Sep 17 00:00:00 2001 From: vadimvlasenko Date: Tue, 30 Jun 2026 12:08:12 +0300 Subject: [PATCH 2/2] docs(analytics): document OTEL source subcommand and update timeline description Generated with AI Co-Authored-By: codemie-ai --- docs/ANALYTICS-REPORT.md | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/docs/ANALYTICS-REPORT.md b/docs/ANALYTICS-REPORT.md index 0c0e8d21..0fd77813 100644 --- a/docs/ANALYTICS-REPORT.md +++ b/docs/ANALYTICS-REPORT.md @@ -145,7 +145,7 @@ Clicking a session anywhere in the report opens a detail overlay with: - **Activity** — turns, tool calls and success rate, agent/skill/command invocation counts - **Code changes** — files changed, lines added/removed, net - **Token & cost growth chart** — cumulative cost and token usage per turn (from the native log; shown when data is available) -- **Dispatch timeline** — Interactive Gantt of agent dispatches. Bars are proportionally scaled to the agent activity window; click any bar to open a detail panel on the right showing that agent's estimated cost, token breakdown (input / output / cache read / cache write), wall-clock duration, start offset from session start, and top tool call counts. Skills and slash commands are shown in the chip lists below the Gantt, not as bars. +- **Dispatch timeline** — Interactive Gantt of every top-level dispatch. A session bar spans the full activity window across the top, then each agent, skill, and slash-command dispatch is rendered as its own bar positioned by wall-clock start time, so each sits where it actually ran. The window is the union of the tracked session span and the dispatch span, so dispatches from a resumed or compacted session still place correctly. Short skills and zero-duration commands fall back to a minimum bar width so they stay visible as markers. Click any bar to open a detail panel on the right showing that dispatch's estimated cost, token breakdown (input / output / cache read / cache write), wall-clock duration, start offset from session start, and top tool call counts. - **Skills / Agent subtypes / Slash commands** — chip lists of what was invoked and how many times --- @@ -187,6 +187,20 @@ Pass `--no-scan-native` to disable native-log discovery and use only CodeMie-tra Cost enrichment requires the native log to read per-turn token data. Sessions where the log has already been rotated or deleted will appear with `—` cost; the **Coverage** section in the Cost view shows exactly which sessions are priced. +### OTEL events file (`analytics otel`) + +As an alternative to the local-session sources above, the `analytics otel` subcommand builds the same report from a **flattened OTEL events file** (`otel-events.jsonl`) — for example, telemetry exported from a fleet or CI environment rather than the current machine's history. + +```bash +# Report from an OTEL events file +codemie analytics otel --file ./otel-events.jsonl --report --open + +# Scope to a single user (matches native user.email or user.id) +codemie analytics otel --file ./otel-events.jsonl --user jane@example.com --report +``` + +With this source, **cost is authoritative**: it is read directly from each event's native `cost_usd`, so no native-log enrichment is needed and every session with token data is priced. All the same filter and report flags apply (`--from`/`--to`, `--project`, `--agent`, `--branch`, `--session`, `--report-format`, etc.); the time window and `--user` are applied to the raw events, and the remaining structural filters narrow the session set so a filtered report is never mislabeled. + --- ## CLI Reference @@ -217,3 +231,14 @@ Other flags: ``` All filter flags work for both the terminal output and the HTML report. The date filters control which sessions are **embedded** in the report; the client-side range presets (Today / 7d / 30d / 90d) then let the report viewer narrow further within that data. + +### OTEL source subcommand + +``` +codemie analytics otel --file [options] + + --file Path to the flattened OTEL events file (required) + --user Scope to one user (native user.email or user.id) +``` + +All filter, report, and export flags from the base command also apply to `analytics otel`. Cost is read from each event's native `cost_usd`, so `--no-scan-native` is not used by this source.