fix: run django case study via django's own CI steps; formalize cache policy

.github/workflows/casestudy-django.yml

@@ -31,10 +31,16 @@ jobs:
       - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
         with:
           python-version: "3.14"
-      # django's own test command at this pin. Uses pip (not uv) as django ships no uv lockfile.
-      # SQLite default — no DB service, no secrets. --parallel exercises all 4 vCPU.
-      # No action-level cache: this is the honest cold path (warm = a primed colocated proxy).
+      # django's own Linux CI steps (their .github/workflows/python_matrix.yml): install
+      # libmemcached-dev (pylibmc has no wheels for Python 3.12+ and the runner images don't bundle
+      # the headers, so django installs them itself), upgrade packaging tools, install the test
+      # dependencies, run runtests.py on SQLite. Identical on both runners; no action-level cache
+      # is enabled — the honest cold path (warm = a primed colocated proxy, set up by the operator).
+      - name: Install libmemcached-dev for pylibmc
+        run: sudo apt-get update && sudo apt-get install -y libmemcached-dev
       - name: Install django and test dependencies
-        run: python -m pip install -e . -r tests/requirements/py3.txt
+        run: |
+          python -m pip install --upgrade pip setuptools wheel
+          python -m pip install -r tests/requirements/py3.txt -e .
       - name: Run django's test suite
-        run: python tests/runtests.py --parallel
+        run: python -Wall tests/runtests.py -v2

docs/methodology.md

@@ -53,14 +53,23 @@ recorded alongside every published number.
 ## Cold cache vs warm cache
 
 Dependency download and resolution dominates many CI runs, and caching changes the picture
-substantially. Cold and warm runs are therefore reported **separately**, never blended:
-
-- **Cold cache** — no pre-populated dependency cache; everything is fetched fresh. This is the
-  worst case and the most honest baseline. The reusable workflow does not enable any action-level
-  cache, so the default path is cold.
-- **Warm cache** — dependencies are served from a primed cache. This reflects steady-state CI on
-  a project that runs frequently. For tempus.build runners this means the colocated proxy has
-  already been populated; for the GitHub side an equivalent primed state is set up by the operator.
+substantially. Cold and warm runs are reported **separately**, never blended.
+
+The reusable workflow is **cache-neutral** — it enables no action-level cache — so it stays
+identical across runners and any caching is purely each platform's own. Cold and warm are
+controlled outside the workflow:
+
+- **Cold — the canonical, headline comparison.** No caches primed on either side; everything is
+  fetched fresh. The GitHub-hosted side is cold by construction (no action-level cache, so packages
+  come from upstream every run); the self-hosted side has its transparent dependency cache cleared
+  before each cold run (otherwise a prior run leaves it warm). With neither side caching, the number
+  isolates the runner itself — CPU, I/O, network — which is the most defensible apples-to-apples
+  baseline, and the one published as the headline.
+- **Warm — steady-state, reported separately and labelled.** Each platform uses its own native
+  caching, primed: the self-hosted side's transparent dependency cache populated, the GitHub side
+  with its action-level cache primed. This reflects repeat CI on a frequently-run project. The
+  caching mechanisms differ per platform, so a warm row is always labelled as warm and never
+  compared against a cold one.
 
 A "speedup" claim must state which cache state it refers to. Comparing tempus warm against GitHub
 cold (or vice versa) is not a valid comparison and is not published.