View for community discovery/connected components

[sclfnc]

Here I propose a panel covering LCC/WCC and centrality metrics for the dashboard. Community detection could be added naturally (see Future extensions), but I left it out since we haven't aligned on which algorithms to use yet. The metric tables below are the real core of the proposal — the data model is scoped to the panel's requirements and can obviously be changed or replaced entirely.

The model has two blocks (Global block and Node block) that enrich the input data without ever modifying the original files:

• Original files provide the keys (node_id, edge_id) that act as referential integrity constraints keeping everything consistent.

• The two blocks communicate via graph_id and a shared positional index i, where index i always refers to the same node across all arrays.

• Metrics are split into two tiers: default (computed at upload, fast) and on demand (computed on explicit user request, to keep memory footprint low).

• The Node block uses a columnar format — one array per metric — because filters and aggregations are the dominant use case: filtering by node type, comparing metrics, or computing distributions become linear vectorized operations with no joins and no additional API calls.

• Edge metrics are out of scope for now — node-level metrics seem more useful for initial exploratory analysis.

  The plots below were produced during exploratory analysis in Python — they are far from production quality, just meant to illustrate the point concretely.

Global Block

Contains:

• graph's structural properties
• aggregate metrics
• connected components with their members.

Note:

• null fields indicate on-demand metrics not yet computed
• Components are sorted by size descending, so the LCC is always wcc_0.
• self_loop_nodes is stored explicitly to enable client-side filtering without API calls.
• by_type provides an inverse index for fast node-type lookups without scanning the full Node block.

Metrics

Metric	Undir.	Dir.	Availability	Notes
n_nodes	✅	✅	Default	—
n_edges	✅	✅	Default	Self-loops ⚠️
n_wcc	—	✅	Default	—
n_scc	—	✅	Default	—
Density	✅	✅	Default	Self-loops excluded
Avg. Path Length (harmonic)	✅	✅	On demand	Global measure, includes disconnected nodes
Diameter (per component)	✅	✅	On demand	Columnar in components
Avg. Path Length (per comp.)	✅	✅	On demand	Columnar in components

Data structure example

{
  "graph_id": "g1",
  "properties": {
    "directed": true,
    "weighted": true,
    "weight_semantics": "distance",
    "has_self_loops": false,
    "node_types": ["person", "org"],
    "edge_types": ["follows", "owns"]
  },
  "metrics": {
    "n_nodes": 1200,
    "n_edges": 4300,
    "n_wcc": 3,
    "n_scc": 12,
    "density": 0.006,
    "avg_path_length_harmonic": null
  },
  "node_sets": {
    "self_loop_nodes": [0, 4, 17],
    "by_type": {
      "person": [0, 1, 3, 5],
      "org": [2, 4]
    }
  },
  "components": {
    "wcc": {
      "lcc_ref": "wcc_0",
      "count": 3,
      "sizes": [578, 412, 23],
      "refs": ["wcc_0", "wcc_1", "wcc_2"],
      "members": [[0, 1, 3], [2, 5], [4]],
      "diameter": null,
      "avg_path_length": null
    },
    "scc": {
      "lscc_ref": "scc_0",
      "count": 12,
      "sizes": [312, 89, 45],
      "refs": ["scc_0", "scc_1", "scc_2"],
      "members": [[0, 3], [1, 5], [2]],
      "diameter": null,
      "avg_path_length": null
    }
  }
}

---

Node Block

• One array per metric, all aligned by index.
• Nodes outside the LCC receive null on connectivity-dependent metrics.

Note:

• Having node_type as a columnar array makes type-based filtering straightforward: selecting nodes of a given type is a mask operation over the array, and the resulting index set can be applied directly to restrict the adjacency matrix.
• Cross-metric comparisons — betweenness vs closeness colored by node type, for example — are equally simple since all arrays are index-aligned. The pipeline is the same regardless of which subset of nodes is selected.

Metrics — undirected graph

Metric	Availability	Notes
Degree	Default	Self-loops ⚠️
Strength	Default	Sum of incident edge weights
Eigenvector	Default	—
Betweenness	On demand	LCC only
Closeness	On demand	LCC only

Metrics — directed graph

Metric	Availability	Notes
In/Out-degree	Default	Self-loops ⚠️
In/Out-strength	Default	Sum of incoming/outgoing edge weights
Eigenvector	Default	Convergence not guaranteed on all directed graphs
PageRank	Default	Distance vs attractiveness semantics affects computation
Betweenness	On demand	LCC only
Closeness	On demand	LCC only; harmonic mean for directed graphs

Data structure

{
  "graph_id": "g1",
  "nodes": {
    "ids": ["n0", "n1", "n2"],
    "node_type": ["person", "person", "org"],
    "in_degree": [5, 2, 3],
    "out_degree": [7, 2, 4],
    "in_strength": [1.8, 0.5, 1.1],
    "out_strength": [1.6, 0.6, 1.7],
    "eigenvector": [0.88, 0.31, 0.59],
    "pagerank": [0.043, 0.011, 0.027],
    "betweenness": null,
    "closeness": null
  }
}

---

Still open

I still did not managed it, but we can easily work on:

1. Edge Block — same columnar structure, arrays of length n_edges. Left out for now to keep scope manageable.
2. Community detection — the structure extends naturally: Global block exposes n_leiden and community arrays, Node block carries a community column.
3. Sorted variants — pre-sorted versions of columnar arrays to speed up ranking and distribution operations like CCDF computation.
4. Subgraphs — if filtering is implemented as adjacency matrix restriction, the structure holds without changes and referential integrity is maintained by construction. Local metrics (degree, strength, eigenvector) can be recomputed on the restricted matrix; betweenness and closeness must be recomputed from scratch on the LCC of the subgraph. The pipeline is reusable; most values are not.

---

I don't know if GitHub notifies repository owners on new issues, so tagging you explicitly — @rinziv. Sorry if this causes a double notification.