This directory holds the processed analysis artefacts that the figure
and text scripts in R/ consume. Raw connectome data (per-synapse
tables, per-neuron meshes, per-neuron metadata, edgelists, …) is not
checked in — those are large products that live on the Harvard Dataverse
deposit and the lab-managed Google Cloud Storage bucket. The pointers
are documented in manuscript/print/banc_data_locations.md;
the most current connectome state is browsable on
FlyWire Codex.
Most of the files here are regenerated by upstream pipelines
(primarily bancpipeline)
and read by the BANC-project R code in this repository. Wherever a
script in this repository produced or consumes a file, that's noted
below; wherever bancpipeline (or another external repo or paper) is the
producer, that's noted too.
| File | What it is |
|---|---|
README.md |
This file. |
specimen.md |
Detailed provenance of the Drosophila specimen used for the BANC: genotype, age, sex, behavioural assay, fixation/staining protocol. Read this if you want to know which fly was reconstructed. |
The table below is the top-level map. Each row says what's inside, who
produced it, and which scripts in this repository read it. Subdirs
preserved across versions (v626/, v850/, v888/) follow the project
convention: the BANC dataset is materialised every few months, and we
keep prior snapshots so older analyses remain re-runnable.
| Subdir | What it contains | Provenance | Consumed by |
|---|---|---|---|
banc_annotations/ |
Per-version snapshots of the cluster + UMAP assignment tables (banc_neck_functional_classes.csv for ANs/DNs, banc_efferent_functional_classes.csv for effectors), the annotation-terms spreadsheet (annotation_terms_list*.xlsx), and the seed-group definitions (seeds.md). Subdirs v626/, v850/, v888/ are the canonical per-materialisation snapshots. |
Written by R/figures/panels_an_dn_umap.R and panels_efferent_umap.R for the current version; older versions are frozen. Annotation-terms xlsx generated by R/annotations/make_annotation_terms_spreadsheet.R. |
Almost every panel script (cluster colour mapping, UMAP coordinates, super_cluster assignments). |
behavior/ |
Raw behavioural-assay data for the BANC fly (Fly T2F73) before dissection — high-speed video traces and per-fly rBIas ("relative behavioural index across strains") indices used to confirm that the specimen behaved within Canton-S × w1118 norms. |
Generated by the behavioural-screening rig at the time of the specimen's behavioural characterisation (see specimen.md). Self-contained — not regenerated by any script in this repo. |
Not currently read by paper-bound code; archival evidence for specimen.md. |
betweenness/ |
Per-neuron betweenness centrality scores (Methods §"Betweenness centrality", Eqs. 11–12). Two variants per version: all-to-all (every neuron a source and a target) and afferent-to-efferent (restricted to the sensory→effector path set). Per-version subdir (betweenness/<version>/). |
Computed by bancpipeline/banc/betweenness/banc-betweenness.py (PETSc-backed igraph Brandes algorithm) → exported as CSV → uploaded to GCS → checked in here. |
R/figures/panels_betweenness_layers.R (Fig. 3a, ED Fig. 5a); helper paths in R/startup/banc-startup.R (.banc_betweenness_*_csv). |
braun_et_al/ |
Supplementary spreadsheet from Braun et al., 2024, Nature (DOI 10.1038/s41586-024-07523) — predicted DN behavioural functions from direct DN-DN connectivity in FAFB. Used as an external reference for the ED Fig. 6c "predicted functions" overlay on the PCA-UMAP. |
External — downloaded from the Nature article's supplementary materials. | R/figures/panels_an_dn_umap.R (ED Fig. 6c). |
cache/ |
Gitignored. Auto-managed local cache of feathers pulled from GCS (BANC edgelist + meta, FAFB v783 edgelist + meta, MANC v1.2.1 edgelist + meta, franken-meta, NBLAST similarity matrices). Regenerated on first access by R/startup/banc-edgelist.R and banc-meta-live.R. Safe to delete; will repopulate. |
Pulled from GCS by R startup scripts. | Every script that touches connectivity or cross-dataset metadata. |
cascade/ |
Pre-computed signal-cascade outputs (Winding et al. 2023 algorithm) used to validate the adjusted-influence metric (Fig. 2b cross-check, ED Fig. 4a). One pickle per seed group under cascade/frankenbrain_v1.6/; the manifest is cascade_modality_batch.json. |
Computed by bancpipeline/analysis/python/cascade_model.py + batch_cascade.py → exported pickles. Frankenbrain v1.6 is the combined BANC+FAFB+MANC graph used for cascade validation. |
R/startup/banc-distances.R (joins cascade output with influence); R/figures/panels_influence_validation.R (Fig. 2b–c, ED Fig. 4a). |
cns_network/ |
Spectral-clustering outputs that partition the central brain + VNC intrinsic + AN/DN + visual-projection + visual-centrifugal neurons into the 13 CNS networks of Fig. 6 / ED Fig. 10 (Methods §"Spectral clustering", Eqs. 13–14). File naming encodes the parameter set: spectral_clustering_min_connection_strength_1_banc_version_<NNN>_cluster_count_13_cluster_seed_10_embedding_seed_3_<v>.csv. |
Computed by bancpipeline/banc/clustering/banc-spectral-clustering.py. |
R/startup/banc-startup.R (.banc_spectral_csv helper); R/figures/panels_cns_networks.R, panels_cns_network_analyses.R, panels_cns_network_diagram.R. |
completion/ |
Per-neuron synapse-capture (proofreading completeness) rates, broken down by gross category, in/out direction, region, and neuropil. Per-version (banc_888_v1_* etc.) and per-size-threshold (size_thresh_5, size_thresh_10) variants. |
Computed by bancpipeline/banc/metrics/banc-calculate-completion.R. |
R/startup/banc-startup.R (.banc_completion_capture_csv() helper); R/figures/panels_inventory.R (Fig. 1c proofreading-fraction panel) and panels_proofread_matching.R. |
determined_thresholds/ |
Two CSV one-liners that record threshold values chosen elsewhere in the pipeline: influence_norm_log_elbow_threshold.csv (the 17.28 "high-influence" cutoff fitted in panels_body_parts.R) and pairwise_modal_influence.csv (the modal cluster–cluster influence used for Fig. 3 normalisation). |
Written by figure scripts (see file headers in R/figures/panels_body_parts.R). Read at startup so the cutoff is available to any panel. |
R/startup/banc-startup.R; downstream panels (panels_an_dn_connectivity.R, panels_mbx_cx_control.R, etc.). |
feedforward/ |
Forward / backward graph-traversal layer assignments for v626 (Schlegel et al., 2024 method, applied to BANC). Used by the early-version influence-validation panel and retained here for reference. | Externally produced (collaborator's run); v888-era replacement is the schlegel_2024_fafb_neuron_ranks/ directory and the on-the-fly query_influence() chain. |
R/figures/panels_influence_validation.R (legacy validation tail). |
influence/ |
Per-seed adjusted-influence CSVs (Methods §"Influence", Eqs. 1–10) — one file per source cell sub class, columns id, influence, influence_norm, influence_norm_log. Currently only the frankenbrain v1.6 cross-dataset outputs are checked in; per-BANC-version outputs are pulled live from GCS by query_influence() on demand. |
Computed by the influencer R package + PETSc backend, dispatched from bancpipeline/banc/influence/. |
query_influence() in R/startup/banc-functions.R; every figure-2-onwards panel that does cross-seed influence. |
meta/ |
The committed BANC metadata snapshot (banc_888_meta_<YYYYMMDD>.parquet — only the latest is retained), the neck-connective inclusion list (banc_neck_inclusion.csv — triaged AN/DN roster for the UMAP), and (gitignored) live-pull caches (bc_orig_cache.feather, hemibrain_nt_meta_*.csv). The snapshot lets the repo be cloned and analysed with no live SeaTable / GCS calls. |
Snapshot written by R/startup/banc-meta-live.R when invoked with BANC_LIVE=1. Live source: SeaTable (manual curations) + GCS (segmentation properties), merged with the priority SeaTable > GCS > franken for manual columns and GCS > SeaTable > franken for the proofread flag. |
R/startup/banc-meta.R (the dispatcher); every analysis script. |
mincut/ |
Minimum-cut / bottleneck enumeration over the BANC graph at increasing path-length limits (2..12 hops). Two views: bottleneck_distribution/ (per-cut node distribution) and mincuts_by_sensory_modality/ (sources broken down by sensory modality). Exploratory analysis of where information bottlenecks sit between sensory and effector populations. |
Computed by an external bottleneck-enumeration script (not currently in bancpipeline); preserved here as an archival product. |
Currently not consumed by any paper-bound script; available for follow-up analyses. |
nblast/ |
The banc_888_top_match_correct.csv table written by R/text/nblast_top_match_correct.R — per-region NBLAST top-match accuracy for BANC vs FAFB v783 and BANC vs MANC v1.2.1. |
Written by R/text/nblast_top_match_correct.R (consumes NBLAST feathers pulled from GCS into data/cache/). |
R/text/numbers.R — six add_row() calls produce the nblast_pct / nblast_score variables cited in the doc. |
private/ |
Gitignored. keys.csv with private Drive identifiers (Google Doc + Sheet IDs for the manuscript) loaded by R/startup/load-keys.R as the banc.keys list. Never committed; the repo is cloneable without it (Drive-write code paths skip cleanly when the keys are absent). |
Hand-maintained by the corresponding author. | R/text/numbers.R, manuscript/print/text/download_and_clean_gdoc.R. |
registrations/ |
Itk-elastix transform stacks (0_manual_affine.txt → 3_elastix_Bspline_fine.txt → BANC_to_template.txt) for registering BANC into the bilateral template space, split into brain_240721/ and vnc_240721/ (date in folder name = registration build date). |
Built by Jasper S. Phelps via the alignment pipeline at bancpipeline/alignment/ (itk-elastix + manual touch-ups). |
Loaded by bancr::banc_to_template() and bancr::banc_to_jrcvnc() (and the underlying nat / nat.templatebrains transforms). Not directly opened by the analysis scripts here — the R client wraps them. |
schlegel_2024_fafb_neuron_ranks/ |
Per-class ranking tables from the graph-traversal model of Schlegel et al., 2024, Nature (DOI 10.1038/s41586-024-07686-5), applied to FAFB v630. Each file is a one-class run: neuron_class_ranking_df_630_all_230409-<source>-<iter>.{csv,feather} where <source> is the seed cell class (allinputs, allinputs_novisual, allsensory, ascending, etc.). |
External — downloaded from Schlegel et al.'s public release. | R/startup/banc-distances.R (joins with influence to build the Fig. 2b "adjusted influence vs. graph layer" panel); R/figures/panels_influence_validation.R. |
shiu_et_al_2024/ |
Supplementary data from Shiu et al., 2024, Nature (DOI 10.1038/s41586-024-07763, file 41586_2024_7763_MOESM2_ESM.xlsx) and downstream re-renderings (sez_neurons.csv / .pickle — the SEZ neuron roster they identified; shiu_et_al_*_opto_and_model_results.csv — their per-neuron optogenetic-activation + model-prediction table, both in their original neuron-ID space and re-mapped into FAFB IDs). |
External — Shiu et al., 2024 supplementary materials; the FAFB-mapped variant uses our own bridge. | Currently not directly read by paper-bound code; archival reference for the feeding/proboscis-extension circuit literature comparisons in the discussion. |
synapse_capture/ |
Per-dataset synapse-attachment / capture rates by neuropil for BANC (v495, v610), FAFB v783, FANC v1237, and MANC. Used as the canonical "what fraction of synapses are attached to proofread neurons" reference for the methods text. See synapse_capture/README.md for per-file provenance (each row in the file lists who generated it and from which export). |
External / per-dataset (BANC table exports, FlyWire Codex download, MANC Table 1 transcription, FANC v1237 CAVE export). | Methods-section prose only; not read by figure code. |
synapse_nt/ |
Synapse-level neurotransmitter prediction confusion matrices, versioned: v1/ (the BANC v1 prediction) and v2/ (the v2 retraining, July 2025 — nt_prediction_confusion_matrix_*_22072025_test_set.csv). Both raw counts and row-normalised forms are provided. |
Trained and predicted by the BANC neurotransmitter classifier (Methods §"Neurotransmitter prediction"); see synister_banc. |
R/figures/panels_transmitter_predictions.R (ED Fig. 3a–e). |
synapses/ |
Per-synapse human-review sample CSVs used to validate the synapse classifier (Methods §"Synapse detection evaluation"): the 2024-09-20 Aelysia sample, the 2025-10-13 evaluation links and per-synapse outcomes, and the 2026 v2 / v3 sample subsets. | Human-reviewed sets, exported from CAVE. | R/figures/panels_synapse_review.R (ED Fig. 1b–d). |
The BANC metadata uses a 5-level hierarchical taxonomy. The values are
defined and frozen as Supplementary Data 1; the canonical column list
lives in data/banc_annotations/v888/ and on
FlyWire Codex. Methods §"Annotation
taxonomy" has the prose definition.
| Level | What it splits on | Examples |
|---|---|---|
flow |
Direction of information relative to the CNS as a whole | afferent, efferent, intrinsic |
super_class |
Coarse functional bucket below flow |
sensory, motor, visceral_circulatory, ascending, descending, optic, central_complex, kenyon_cell, … |
cell_class |
Functional class below super_class |
chordotonal_organ_neuron, leg_motor_neuron, visual_projection_neuron, gustatory_neuron, … (≥100 values) |
cell_sub_class |
Sub class below cell_class |
wing_steering_motor_neuron, front_leg_hair_plate_neuron, bitter_gustatory_neuron, … |
cell_type |
Matched type from FAFB v783 (brain/DN) or MANC v1.2.1 (VNC/AN); occasionally further split | ORN_DM6, DNge110, AN08B018 |
Auxiliary columns on the same metadata row include region, side,
nerve, hemilineage, neuromere, body_part_sensory,
body_part_effector, peripheral_target_type, cell_function,
cell_function_detailed, neurotransmitter_verified,
neuropeptide_verified, fafb_v783_match_id, manc_v1.2.1_match_id,
plus the per-version root_id columns (root_626, root_850,
root_888). The full schema lives in
data/meta/banc_888_meta_<date>.parquet.
Per-neuron meshes, per-synapse tables, raw segmentation IDs, per-region masks, the full edgelist, NBLAST matrices, neuron skeletons (SWC), the Neuroglancer state JSONs, and the registration template volumes are not in this directory. They live at:
- Harvard Dataverse — BANC v888 deposit — the citable static snapshot.
- GCS bucket
gs://lee-lab_brain-and-nerve-cord-fly-connectome/compiled_data/banc_888/— the lab-managed source-of-truth (parquet/feather, no auth required for read-only). - FlyWire Codex — interactive browser for the most current state (continues to evolve past the static deposit).
See manuscript/print/banc_data_locations.md
for the exact Dataverse + GCS path of every data product.
- Version naming. BANC dataset materialisations are tagged
banc_NNN(v626, v850, v888). Subdirs with per-version snapshots usevNNN/as the subdir name; loose files include_banc_NNNin the filename. - Synapse table version. All paper analysis uses synapses v2.
The
v3synapses (~8–18% more synapses, refined model) are kept alongside v2 in some products but are not the paper version. - Synapse count thresholds. The proofread edgelist is checked in
with no global synapse-count filter. The "≥ 5 synapses" filter is
applied per-call by
influence_calculator_py(count_thresh = 5)and shows up in betweenness inputs at the bancpipeline level. Hop-tracing panels (panels_pre_effector_influence.R) usecount >= 10. - Cache hygiene.
data/cache/is gitignored and safe to delete; R startup will repopulate. If something is mis-cached,rm -rf data/cache/is the canonical reset.
For the per-file-level column dictionary, the most up-to-date reference is the column tooltip in FlyWire Codex and the dataset documentation in fly_connectome_data_tutorial.