Skip to content

Performance footguns: per-call tracemalloc, repeated dataset/parquet reloads, path-keyed caches #99

Description

@NarayanSchuetz

From a full-repo review (line refs from ecb7fe0). Excludes items already tracked in #15 (skill-rank bootstrap scaling) and #16 (HDF5 chunking).

  • forecasting_evaluation/evaluation/evaluator.py:108-113: tracemalloc.start()/stop() wraps every predict call — 2-4x allocation-cost slowdown for a perf_* column with no consumers. Gate behind a flag.
  • Forecasting cache keys hash file paths, not content (data/online_dataset.py:264-286) — regenerating the dataset in place silently reuses stale H5 caches. (Correctness-adjacent; content fingerprint or mtime in the key.)
  • imputation_evaluation/evaluation/pair_aggregator.py:572-588: aggregate_pairs_by_subgroup re-reads all 19 pair parquets per (attr, group) combination.
  • Forecasting metrics pipeline reloads the full dataset + splits 3-4x per run.
  • Track 1 per-task re-materialization: cohort re-bound/drained per (task, split); Linear._features re-pools full history 64x; GRU-D rebuilds its dataset per task.

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance and scalability improvements

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions