feat(data_explorer): add no-code visual query builder (alpha)

[albertogrande]

Data explorer is a no-code builder for tables that span PostHog and warehouse data — joins, rollups, and filters by clicking, saved as a queryable warehouse view. No SQL.

▶ Try it live: https://posthog-demo.fly.dev/ — demo@posthog-demo.fly.dev / PostHogDemo

▶ Demo video

1080-30.mp4

Problem

"Which accounts stopped paying but still use the product?" is a standard growth-review question. Answering it means joining group data, a warehouse invoices table, and usage events, per account. Today that's hand-written HogQL — out of reach for the people who actually run those reviews, who think in tables and filters, not SQL.

Knowing SQL doesn't make it safe either. Join two one-to-many sources in one query and the rows fan out: every aggregate comes back inflated, and nothing tells you it's wrong.

Changes

A new alpha product in products/data_explorer/, behind FEATURE_FLAGS.DATA_EXPLORER. You pick a primary source, add columns from it and from related sources, connect warehouse tables on a join key, add filters and per-column time windows, preview live, and save the result as a view.

There's no new model. A saved view is a plain DataWarehouseSavedQuery with the builder state stored in builder_metadata, so it inherits the warehouse's materialization, permissions, and tenancy, and is queryable anywhere HogQL works.

Walkthrough — tracking monthly churn

Built entirely by clicking — the demo video above is this exact flow.

1. Pick what each row is. Churn is an account-level question, so pick groups: one row per account.

2. Connect the data. Join invoices (many rows per account → rollups) and salesforce (one row per account → details) on key = account_key. Events are already a related source for groups.

3. Add columns. Detail columns join 1:1. Rollups carry a math chip (Sum, Count, Avg, …) and a time window.

Source	Field	Added as
Account (primary)	name (company)	detail
Salesforce (1:1)	owner, icp_score	details
Invoices (1:many)	amount_usd	Sum · this month → mrr_this_month
Invoices (1:many)	amount_usd	Sum · last month → mrr_last_month
Events (1:many)	exceptions_captured_in_period	Sum · this month → exceptions_this_month
Events (1:many)	days_with_exceptions	Sum · this month → days_this_month
Events (1:many)	timestamp	Latest · all time → last_active

4. Filter to the churn segment. Two rules, ANDed: mrr_this_month = 0 and mrr_last_month > 0 — paid last month, nothing this month. Sort by active days descending so still-active accounts float to the top.

5. Read the result. Every row churned on billing; the usage columns say why. Active days near zero: they disengaged, then left. Active days near 30 with a high exception count: still using it daily without paying — the win-back list billing data alone can't show you.

6. Save it. The result is a queryable DataWarehouseSavedQuery — build a cohort from it, chart it in an insight, materialize it. Reopening it lands you back in the builder.

…or just ask PostHog AI

The explore_data tool lets Max drive the same builder. Describe the table in plain English and it picks fields, aggregations, filters, and sort from your connected sources, then hands back the same editable config you'd build by hand — not SQL dropped into chat. For example:

"Add each account's CS owner and ICP score, plus the total exceptions they've captured, and sort by total exceptions — highest first."

Max adds owner and icp_score as details and a Sum rollup of exceptions_captured_in_period, sorted descending. Hand-built and AI-built columns go through the same column-assembly module, so you can keep editing where it left off. In the flag-gated builder mode, Max is restricted to this tool: analytical prompts land in the builder, not as SQL in chat.

v1 scope: the AI works over sources already connected in the rail. It doesn't author new joins or time windows yet — you set those up once by hand (steps 2–3), then iterate with the AI on top.

Under the hood

Compiler. Pure and ORM-free: builder_metadata in, HogQL SelectQuery out. Each related source compiles to its own subquery reduced to one row per join key — any() for details, the chosen aggregate for rollups — then LEFT JOINs back onto the entity. Reducing before joining is what prevents the fan-out: two rollups over two sources can't multiply each other's rows. Time windows compile to conditional aggregation (sumIf/countIf), so "MRR this month" and "MRR last month" share one subquery.

Preview and persistence. POST /api/environments/:id/data_explorer_preview/ compiles and executes the config for the capped live grid; with compile_only it returns the uncapped HogQL that's persisted on the view. Connected joins live in the view's own builder_metadata, validated and size-bounded on write — building never touches the global warehouse schema.

PostHog AI. The tool (products/data_explorer/backend/max_tools.py) receives the current builder state plus a catalog of pickable fields, each with a ready-made column config. The model selects by reference; the backend resolves the selection into builder_metadata and compile-validates it before the frontend applies it. The model never writes HogQL — that's what keeps every AI-built query valid and editable.

Security. All user and AI input is lowered to typed AST constants and identifiers, resolved against the team's own schema, and escaped by the HogQL printer. No string interpolation into SQL anywhere in the path.

How did you test this code?

Manually. Built the walkthrough above end-to-end on the live build (posthog-demo.fly.dev), sanity-checked the numbers, saved the view, and reopened it in the builder. An agent browser QA pass over the same build found no P0/P1 defects.

The claims a reviewer will care about, and how each is verified:

• No fan-out. Compiler tests assert each one-to-many source reduces to one row per key before joining (including a live ClickHouse execution); confirmed on the live build's compiled HogQL, where the two MRR windows share one sumIf subquery.
• Injection neutralized. id) OR 1=1 -- resolves as a schema identifier and returns a 400, never executed — covered in the preview API tests and reproduced against the live endpoint. The AI tool likewise refuses injection and declines nonexistent fields.
• Tenancy and gating. Preview API tests cover cross-tenant isolation, feature flag gating, row caps, and malformed-config rejection.

In CI: backend tests for the compiler, validation, preview API, and AI tool; Jest for the builder logics and grid; a Playwright end-to-end click-through.

Publish to changelog?

No — alpha, behind a flag.

Docs update

None in this PR (alpha). Add the skip-inkeep-docs label.

🤖 Agent context

Built with Claude Code; I drove the design and own the code, the compiler most of all. Decisions along the way:

• No new model. Builder state rides on DataWarehouseSavedQuery.builder_metadata instead of a parallel model, so views get materialization, tenancy, and permissions for free.
• Pre-aggregate, then join. Rejected naive multi-joins early: two one-to-many joins in one query double-count. Each related source reduces to one row per key before joining.
• AI selects, never writes SQL. The explore_data tool picks from a catalog of real fields, the backend compile-validates the result, and one shared module assembles hand-built and AI-built columns alike.

Human review required — do not self-merge.

[albertogrande]

Note for reviewers: the red checks are fork-infra, not test failures — the paths-filter gate jobs need PostHog-internal GitHub App secrets that forks don't have, so their dependent "… Tests Pass" checks fail without running any tests. The backend, Jest, Playwright, tach, and OpenAPI suites all pass locally.

[github-actions]

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the stale label – otherwise this will be closed in another week. If you want to permanently keep it open, use the waiting label.