Skip to content

feat(graph): add scalable snapshot structure graph explorer (web + server)#90

Merged
saurabhsharma2u merged 9 commits into
mainfrom
codex/add-interactive-structural-graph-page
Mar 4, 2026
Merged

feat(graph): add scalable snapshot structure graph explorer (web + server)#90
saurabhsharma2u merged 9 commits into
mainfrom
codex/add-interactive-structural-graph-page

Conversation

@saurabhsharma2u

Copy link
Copy Markdown
Contributor

Motivation

  • Provide an interactive structure graph view for a snapshot that lets users pan/zoom, inspect URLs, and apply filters while supporting very large graphs (target: 10k nodes) without UI lag.
  • Push heavy filtering/limits server-side to keep browser rendering responsive and avoid transferring huge payloads.

Description

  • Add a server endpoint GET /api/graph/snapshot that returns a lightweight graph projection for a snapshot with server-side filters (maxNodes, maxEdges, minPageRank, minInlinks, minOutlinks, role, search) and truncation metadata; implemented in packages/server/src/index.ts.
  • Introduce typed API contracts and a client helper fetchSnapshotGraph in packages/web/src/api.ts (SnapshotGraphNode, SnapshotGraphEdge, SnapshotGraphResponse, SnapshotGraphQuery).
  • Implement a new high-performance canvas-based UI GraphPage in packages/web/src/pages/GraphPage.tsx with pan/zoom, hover tooltips, URL search and numeric filters, maxNodes control, and an edge-truncation notice for performance.
  • Wire the page into the application router at /graph by updating packages/web/src/App.tsx and expose server-side SQL that aggregates in/out counts and limits node/edge results for predictable client-side rendering.

Testing

  • Ran a production build for the web package with pnpm --filter @crawlith/web build, which completed successfully. ✅
  • Type-check for the server with pnpm exec tsc -p packages/server/tsconfig.json --noEmit failed due to workspace type resolution in this environment (@crawlith/core types not resolved), not related to the introduced endpoint logic. ⚠️
  • Attempted an automated browser capture via Playwright to validate the page render, but the Chromium instance crashed in this environment (TargetClosedError / SIGSEGV), so no screenshot artifact was produced. ⚠️

Codex Task

@saurabhsharma2u
saurabhsharma2u merged commit fe3a23a into main Mar 4, 2026
1 of 7 checks passed
@saurabhsharma2u
saurabhsharma2u deleted the codex/add-interactive-structural-graph-page branch March 4, 2026 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant