AI analyst: natural-language questions to a planned, sandboxed, provenance-cited analysis

[dcondrey]

Why this is powerful

The Ask tab answers simple questions. This makes it an agentic analyst: ask "why did signups drop last Tuesday and which channel caused it?" and Facet decomposes it into a plan, runs a series of safe, read-only cube queries, cross-checks the numbers against the transparency log (#14), and returns a narrative answer with every claim cited to a verifiable metric — then offers to build the boxes that show it. Trustworthy AI analytics: no hallucinated numbers, because each figure is provable.

Architecture sketch

• Planner: an LLM turns the question into a typed query plan over the cube schema (dimensions/metrics/date math) — never free-form SQL. A validator rejects anything outside a read-only allowlist and bounds fan-out/cost.
• Executor: runs the plan against the existing aggregation layer; results are attached to ProofRefs (Verifiable analytics: cryptographic inclusion proofs for every published metric #14) so the answer is auditable.
• Grounded synthesis: the model may only reference numbers returned by the executor; a verifier step re-checks each cited figure and strips/flags anything unsupported (no hallucinated metrics).
• Actionable output: answers can spawn boxes on the board (reuse the box library / TileDef) and save as a shareable, re-runnable "analysis".
• Guardrails: per-site token/cost budget, PII-free prompts (aggregates only, honoring the DP mode from Differential privacy for arbitrary segmentation (per-site ε budget) #15), and full local-verifiability of every cited number.

Subtasks

• Typed query-plan schema + validator (read-only, cost-bounded)
• LLM planner + deterministic executor over the cube
• Grounded-citation synthesis + a verifier that rejects unsupported figures
• "Turn this answer into boxes" + saved/shareable analyses
• Cost/PII guardrails; eval harness for answer faithfulness

Hard parts: a query-plan IR expressive enough to be useful yet safe, guaranteeing zero fabricated numbers (grounding + verification), and multi-step planning that stays within edge cost/latency budgets.