Skip to content

feat(provenance): kb.graph_export gains format=json for a webapp graph view - #705

Closed
Tet-9 wants to merge 1 commit into
vouchdev:testfrom
Tet-9:feat/604-graph-export-json
Closed

feat(provenance): kb.graph_export gains format=json for a webapp graph view#705
Tet-9 wants to merge 1 commit into
vouchdev:testfrom
Tet-9:feat/604-graph-export-json

Conversation

@Tet-9

@Tet-9 Tet-9 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

what

kb.graph_export (the DAG behind vouch graph) supported dot and mermaid text formats only. Added json: {nodes: [{id, kind, label, status}], edges: [{src, dst, kind}]}, using the exact same node/edge walk as the existing formats.

why

A webapp graph view (pan/zoom visualization of the KB's provenance DAG) needs structured data, not Graphviz text. status is populated from each node's own durable ClaimStatus/PageStatus where the node kind carries one — null for source/evidence/session/event nodes, which have no status concept, and for a dangling id.

Scope note, left directly in the code: pending proposals are not graph nodes at all today — build_graph reads only durable (approved) artifacts, never proposed/. Fully supporting "pending vs. approved" coloring the way the originating issue describes would mean teaching build_graph itself to walk pending proposals — a materially bigger change than a serializer addition. Flagged as a follow-up rather than silently narrowing what status means.

Also out of scope for this PR: the React MemoryNetworkView.tsx frontend component (pan/zoom rendering, Shell/ArtifactDrawer wiring) the originating issue also asks for. This PR is the backend half only.

invariants held

  • dot and mermaid output are byte-identical to before — verified with an explicit regression test that takes a dot export, requests a json export, then takes another dot export and asserts they match exactly.
  • Every node object has a consistent shape (id, kind, label, status) regardless of kind; status is null, never omitted, when a kind has no status concept.

tests

  • test_graph_export_json: shape correctness, real status values for claim/page nodes, null for other kinds.
  • test_graph_export_json_does_not_affect_dot_or_mermaid: the byte-identical regression check above.
  • Full test_provenance.py + test_capabilities.py + test_trust.py: passing.

Closes #604 (backend half — see scope notes above for what's intentionally not included)

…h view

graph_export previously supported dot and mermaid text formats only.
Added json: {nodes: [{id, kind, label, status}], edges: [{src, dst,
kind}]}, same node/edge walk as the existing formats -- dot and mermaid
are byte-identical to before (verified: a dot export taken before and
after a json export match exactly, and a committed regression test
enforces this).

status is the artifact's own durable ClaimStatus/PageStatus for claim/page
nodes, null for source/evidence/session/event nodes (no status concept)
and for a dangling id. Scope note left in the code: pending proposals are
not graph nodes at all currently (build_graph reads only durable
artifacts), so full pending-vs-approved coloring per the issue's
description would need a bigger change to build_graph itself -- not
attempted here, flagged as a follow-up rather than silently narrowed.

- provenance/query.py: new _to_json(), fmt validation extended to accept
  'json', graph_export docstring updated.
- cli.py: --format Choice extended to include json.
- New tests/test_provenance.py cases: json shape/status correctness, and
  an explicit dot-output-unchanged-by-json regression test.

Not done in this pass (backend-only, out of scope for a fast turnaround):
the React MemoryNetworkView.tsx frontend component the issue also asks
for -- pan/zoom graph visualization, wired into Shell + ArtifactDrawer.

Closes vouchdev#604 (backend half)
@Tet-9
Tet-9 requested a review from plind-junior as a code owner July 31, 2026 05:43
@Tet-9

Tet-9 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

@plind-junior

@github-actions github-actions Bot added cli command line interface tests tests and fixtures size: S 50-199 changed non-doc lines labels Jul 31, 2026
@plind-junior

Copy link
Copy Markdown
Member

the byte-identical regression test on dot and mermaid is the right instinct for a serializer addition, and the scope notes left in the code rather than only in the PR description are the correct call — a future reader hitting status needs to know it means "durable artifact status", not "pending vs approved".

but those scope notes are also why i would want a decision from a maintainer before this merges. #604 asks for a graph view where the pending frontier is visible; this ships the serializer with pending explicitly out of scope, and the React component explicitly out of scope. what lands is a third output format for vouch graph with no consumer in the repo and a status field that does not yet answer the question the issue was filed to answer. that may well be the right first slice — but it is worth being explicit that it is a slice, and that the follow-up (teaching build_graph to walk proposed/) is the part carrying the actual value. otherwise this sits as a format nobody calls.

one concrete implementation note. _to_json re-fetches every node from disk:

status = store.get_claim(n).status.value
...
status = store.get_page(n).status.value

_to_dot and _to_mermaid do zero I/O — they render from the in-memory ProvGraph. so fmt="json" turns a pure formatting call into one get_claim/get_page per node across the whole graph, on a code path that already has a cache (load_graph(use_cache=...)) specifically to avoid re-reading. on a KB of any size that is the expensive format by an order of magnitude, and it is avoidable: build_graph already walks every claim and page to build node_kinds, so status could be captured there and carried on the graph instead of re-read here.

the signature asymmetry is the tell — _to_json needs store and its two siblings do not. worth fixing before this becomes the format a webapp calls on every render.

@plind-junior

Copy link
Copy Markdown
Member

closing this — it ships a format with no consumer, and the part of #604 that carries the value is explicitly deferred.

#604 asks for a graph view where the pending frontier is visible. this PR's own scope note says pending proposals are not graph nodes at all, because build_graph reads only durable artifacts and never proposed/ — so status here means "durable ClaimStatus/PageStatus", not "pending vs approved". the React component is also out of scope. what actually lands is a third output format for vouch graph that nothing in the repo calls, whose one distinguishing field does not yet answer the question the issue was filed to answer.

that is a reasonable slice to propose, but merging it means carrying a public output format on kb.graph_export — one we then owe compatibility to — before anything reads it and before we know what shape the consumer actually wants. if the webapp ends up needing the pending frontier, status has to change meaning or grow a sibling field, and we will be changing a shipped format rather than designing an unshipped one.

there is also a concrete regression in it. _to_json re-fetches every node from disk:

status = store.get_claim(n).status.value
...
status = store.get_page(n).status.value

_to_dot and _to_mermaid do zero I/O — they render from the in-memory ProvGraph. so fmt="json" turns a pure formatting call into one get_claim/get_page per node across the whole graph, on a path that already has load_graph(use_cache=...) specifically to avoid re-reading. the signature asymmetry is the tell: _to_json needs store and its two siblings do not. build_graph already walks every claim and page to build node_kinds, so status belongs on the graph, captured once.

the byte-identical dot/mermaid regression test was the right instinct and i would keep it.

what would get this merged: teach build_graph to carry status (fixing the N-fetch), and land the serializer together with the consumer that needs it — either the React view or a concrete spec for it — so the format is designed against a real caller. happy to review that as one piece.

@Tet-9

Tet-9 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor Author

closing this — it ships a format with no consumer, and the part of #604 that carries the value is explicitly deferred.

#604 asks for a graph view where the pending frontier is visible. this PR's own scope note says pending proposals are not graph nodes at all, because build_graph reads only durable artifacts and never proposed/ — so status here means "durable ClaimStatus/PageStatus", not "pending vs approved". the React component is also out of scope. what actually lands is a third output format for vouch graph that nothing in the repo calls, whose one distinguishing field does not yet answer the question the issue was filed to answer.

that is a reasonable slice to propose, but merging it means carrying a public output format on kb.graph_export — one we then owe compatibility to — before anything reads it and before we know what shape the consumer actually wants. if the webapp ends up needing the pending frontier, status has to change meaning or grow a sibling field, and we will be changing a shipped format rather than designing an unshipped one.

there is also a concrete regression in it. _to_json re-fetches every node from disk:

status = store.get_claim(n).status.value
...
status = store.get_page(n).status.value

_to_dot and _to_mermaid do zero I/O — they render from the in-memory ProvGraph. so fmt="json" turns a pure formatting call into one get_claim/get_page per node across the whole graph, on a path that already has load_graph(use_cache=...) specifically to avoid re-reading. the signature asymmetry is the tell: _to_json needs store and its two siblings do not. build_graph already walks every claim and page to build node_kinds, so status belongs on the graph, captured once.

the byte-identical dot/mermaid regression test was the right instinct and i would keep it.

what would get this merged: teach build_graph to carry status (fixing the N-fetch), and land the serializer together with the consumer that needs it — either the React view or a concrete spec for it — so the format is designed against a real caller. happy to review that as one piece.

If you may reopen this, I'd then make all changes required on it ASAP

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli command line interface size: S 50-199 changed non-doc lines tests tests and fixtures

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(web): memory-network graph view — render the kb as a pannable graph, pending included

2 participants