Skip to content

ci: add merge-blocking perf gates — Locust p95 smoke + Lighthouse-CI #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ChelseaKR merged 2 commits into from
Jul 12, 2026
Merged

ci: add merge-blocking perf gates — Locust p95 smoke + Lighthouse-CI #28

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
144e4aa
ci: add merge-blocking perf gates — Locust p95 smoke + Lighthouse-CI
ChelseaKR Jul 3, 2026
f48db32
Merge remote-tracking branch 'origin/main' into codex/pr28-remediate
ChelseaKR Jul 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,12 @@ jobs:
- name: Eval
run: make eval

- name: Performance load smoke
run: make perf-load

- name: Lighthouse CI
run: make lighthouse

- name: Upload audit artifacts
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
Expand Down
9 changes: 9 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ htmlcov/
data/*.db
data/*.sqlite
data/*.sqlite3
data/demo/
snapshots/

# Secrets — kosync keys / API tokens live in the environment / .env, never source
Expand All @@ -24,5 +25,13 @@ snapshots/
docs/audits/pa11y-raw.json
node_modules/

# Perf/Lighthouse gate raw dumps (scripts/perf-smoke.sh, `make lighthouse`) are
# not committed; the curated aggregated perf_stats.csv summary is.
docs/audits/perf_stats_history.csv
docs/audits/perf_failures.csv
docs/audits/perf_exceptions.csv
docs/audits/lighthouse/
.lighthouseci/

# runtime-fetched portfolio-standards (never commit)
.standards/
20 changes: 20 additions & 0 deletions .lighthouserc.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"ci": {
"collect": {
"staticDistDir": "docs/audits",
"url": ["/dashboard.html"],
"numberOfRuns": 3,
"settings": { "preset": "desktop" }
},
"assert": {
"assertions": {
"categories:performance": ["error", { "minScore": 0.9 }],
"categories:accessibility": ["error", { "minScore": 1.0 }]
}
},
"upload": {
"target": "filesystem",
"outputDir": "docs/audits/lighthouse"
}
}
}
11 changes: 10 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ PYTHON3 ?= python3.14
A11Y_HTML := docs/audits/dashboard.html

.DEFAULT_GOAL := help
.PHONY: help install dev verify format lint marker-hygiene typecheck test security a11y eval perf audit clean
.PHONY: help install dev verify format lint marker-hygiene typecheck test security a11y eval perf perf-load lighthouse perf-gates audit clean

help: ## Show this help
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | \
Expand Down Expand Up @@ -89,6 +89,15 @@ eval: ## Stage 7 — offline eval; fails unless the recommender beats popularity
perf: ## Stage 6 — render/pipeline performance budget (also run within `make test`)
$(PYTHON) -m pytest tests/test_perf.py -q -o addopts=""

perf-load: ## Stage 6b — merge-blocking load smoke: p95 < 500ms on the dashboard route
@./scripts/perf-smoke.sh

lighthouse: ## Stage 6c — merge-blocking Lighthouse-CI on the built dashboard HTML
$(PYTHON) -m app.build_static
npx --yes @lhci/cli@0.15.1 autorun --config=.lighthouserc.json

perf-gates: perf-load lighthouse ## Run both merge-blocking performance gates

audit: a11y eval ## Regenerate all committed responsible-tech artifacts
$(PYTHON) -m pytest -q >/dev/null
@echo "✓ audit artifacts regenerated under docs/audits/"
Expand Down
2 changes: 2 additions & 0 deletions app/render.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,8 @@ def _rec_table(recs: Sequence[Recommendation]) -> str:
html { color: CanvasText; background-color: Canvas; }
body { font-family: system-ui, sans-serif; max-width: 75ch; margin: 0 auto; padding: 1rem;
color: inherit; background-color: inherit; }
a { color: LinkText; }
a:visited { color: VisitedText; }
.card, li.reading { border: 1px solid; border-radius: 8px; padding: 1rem; margin: 1rem 0;
list-style: none; }
ul.books { padding: 0; }
Expand Down
17 changes: 13 additions & 4 deletions docs/ROADMAP-FUTURE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,19 @@ your real library" before adding features.

## Deferred merge-blocking gates the standard names

- **Performance (Quality §2)** — k6/Locust smoke (p95 < 500 ms dashboard route) +
Lighthouse-CI on the rendered HTML; make merge-blocking. *(N5)*
- **Reliability (Quality §5)** — restart-recovery test (reads persisted state) and
a chaos test for "kosync down → degrade to KOReader-only". *(N5)*
- ~~**Performance (Quality §2)** — k6/Locust smoke (p95 < 500 ms dashboard route) +
Lighthouse-CI on the rendered HTML; make merge-blocking.~~ **Done (2026-07-03).**
`tests/perf/locustfile.py` + `scripts/perf-smoke.sh` (`make perf-load`) boot the
demo app and fail the build if the aggregated p95 on `/` is >= 500ms;
`.lighthouserc.json` + `make lighthouse` run Lighthouse-CI against the built
`docs/audits/dashboard.html` with `categories:performance >= 0.9` and
`categories:accessibility >= 1.0` as error-level (merge-blocking) assertions.
Both are non-conditional steps in `.github/workflows/ci.yml`. *(N5)*
- ~~**Reliability (Quality §5)** — restart-recovery test (reads persisted state) and
a chaos test for "kosync down → degrade to KOReader-only".~~ **Done.**
`tests/test_reliability.py` (`test_restart_recovery`,
`test_kosync_down_degrades_to_stats`), run merge-blocking as part of
`make test` / the CI `Tests` step. *(N5)*
- **Manual a11y sign-off** — perform + commit the dated VoiceOver/NVDA walkthrough.
*(N1–N6, before first release)*

Expand Down
118 changes: 59 additions & 59 deletions docs/audits/coverage.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
<?xml version="1.0" ?>
<coverage version="7.14.3" timestamp="1783843123098" lines-valid="3024" lines-covered="2969" line-rate="0.9818" branches-valid="688" branches-covered="633" branch-rate="0.9201" complexity="0">
<coverage version="7.14.3" timestamp="1783843323660" lines-valid="3024" lines-covered="2969" line-rate="0.9818" branches-valid="688" branches-covered="633" branch-rate="0.9201" complexity="0">
<!-- Generated by coverage.py: https://coverage.readthedocs.io/en/7.14.3 -->
<!-- Based on https://raw.githubusercontent.com/cobertura/web/master/htdocs/xml/coverage-04.dtd -->
<sources>
<source>/private/tmp/qts-pr26/app</source>
<source>/private/tmp/qts-pr26/ingest</source>
<source>/private/tmp/qts-pr26/recommender</source>
<source>/private/tmp/qts-pr28/app</source>
<source>/private/tmp/qts-pr28/ingest</source>
<source>/private/tmp/qts-pr28/recommender</source>
</sources>
<packages>
<package name="." line-rate="0.9818" branch-rate="0.9201" complexity="0">
Expand Down Expand Up @@ -2261,67 +2261,67 @@
<line number="167" hits="1"/>
<line number="174" hits="1"/>
<line number="182" hits="1"/>
<line number="212" hits="1"/>
<line number="213" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="214" hits="1"/>
<line number="215" hits="1"/>
<line number="221" hits="1"/>
<line number="228" hits="1"/>
<line number="229" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="215" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="216" hits="1"/>
<line number="217" hits="1"/>
<line number="223" hits="1"/>
<line number="230" hits="1"/>
<line number="231" hits="1"/>
<line number="231" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="232" hits="1"/>
<line number="237" hits="1"/>
<line number="246" hits="1"/>
<line number="247" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="233" hits="1"/>
<line number="234" hits="1"/>
<line number="239" hits="1"/>
<line number="248" hits="1"/>
<line number="249" hits="1"/>
<line number="255" hits="1"/>
<line number="263" hits="1"/>
<line number="270" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="271" hits="1"/>
<line number="249" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="250" hits="1"/>
<line number="251" hits="1"/>
<line number="257" hits="1"/>
<line number="265" hits="1"/>
<line number="272" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="273" hits="1"/>
<line number="282" hits="1"/>
<line number="291" hits="1"/>
<line number="292" hits="1"/>
<line number="298" hits="1" branch="true" condition-coverage="50% (1/2)" missing-branches="315"/>
<line number="299" hits="1"/>
<line number="305" hits="1"/>
<line number="315" hits="0"/>
<line number="319" hits="1"/>
<line number="323" hits="1"/>
<line number="333" hits="1" branch="true" condition-coverage="50% (1/2)" missing-branches="351"/>
<line number="334" hits="1"/>
<line number="342" hits="1"/>
<line number="351" hits="0"/>
<line number="353" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="354" hits="1"/>
<line number="360" hits="1"/>
<line number="367" hits="1"/>
<line number="378" hits="1"/>
<line number="385" hits="1" branch="true" condition-coverage="50% (1/2)" missing-branches="391"/>
<line number="386" hits="1"/>
<line number="391" hits="0"/>
<line number="398" hits="0"/>
<line number="410" hits="1"/>
<line number="411" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="275" hits="1"/>
<line number="284" hits="1"/>
<line number="293" hits="1"/>
<line number="294" hits="1"/>
<line number="300" hits="1" branch="true" condition-coverage="50% (1/2)" missing-branches="317"/>
<line number="301" hits="1"/>
<line number="307" hits="1"/>
<line number="317" hits="0"/>
<line number="321" hits="1"/>
<line number="325" hits="1"/>
<line number="335" hits="1" branch="true" condition-coverage="50% (1/2)" missing-branches="353"/>
<line number="336" hits="1"/>
<line number="344" hits="1"/>
<line number="353" hits="0"/>
<line number="355" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="356" hits="1"/>
<line number="362" hits="1"/>
<line number="369" hits="1"/>
<line number="380" hits="1"/>
<line number="387" hits="1" branch="true" condition-coverage="50% (1/2)" missing-branches="393"/>
<line number="388" hits="1"/>
<line number="393" hits="0"/>
<line number="400" hits="0"/>
<line number="412" hits="1"/>
<line number="413" hits="1"/>
<line number="420" hits="1"/>
<line number="432" hits="1"/>
<line number="449" hits="1"/>
<line number="456" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="457" hits="1"/>
<line number="461" hits="1"/>
<line number="464" hits="1"/>
<line number="470" hits="1"/>
<line number="471" hits="1"/>
<line number="480" hits="1"/>
<line number="498" hits="1"/>
<line number="501" hits="1"/>
<line number="504" hits="1"/>
<line number="507" hits="1"/>
<line number="510" hits="1"/>
<line number="413" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="414" hits="1"/>
<line number="415" hits="1"/>
<line number="422" hits="1"/>
<line number="434" hits="1"/>
<line number="451" hits="1"/>
<line number="458" hits="1" branch="true" condition-coverage="100% (2/2)"/>
<line number="459" hits="1"/>
<line number="463" hits="1"/>
<line number="466" hits="1"/>
<line number="472" hits="1"/>
<line number="473" hits="1"/>
<line number="482" hits="1"/>
<line number="500" hits="1"/>
<line number="503" hits="1"/>
<line number="506" hits="1"/>
<line number="509" hits="1"/>
<line number="512" hits="1"/>
</lines>
</class>
<class name="rerank.py" filename="rerank.py" complexity="0" line-rate="1" branch-rate="1">
Expand Down
2 changes: 2 additions & 0 deletions docs/audits/dashboard.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
html { color: CanvasText; background-color: Canvas; }
body { font-family: system-ui, sans-serif; max-width: 75ch; margin: 0 auto; padding: 1rem;
color: inherit; background-color: inherit; }
a { color: LinkText; }
a:visited { color: VisitedText; }
.card, li.reading { border: 1px solid; border-radius: 8px; padding: 1rem; margin: 1rem 0;
list-style: none; }
ul.books { padding: 0; }
Expand Down
4 changes: 4 additions & 0 deletions docs/audits/perf_stats.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Type,Name,Request Count,Failure Count,Median Response Time,Average Response Time,Min Response Time,Max Response Time,Average Content Size,Requests/s,Failures/s,50%,66%,75%,80%,90%,95%,98%,99%,99.9%,99.99%,100%
GET,/,746,0,5,6.47179199990176,2.195666980696842,50.50283300806768,21355.0,43.73460951878272,0.0,5,7,8,8,10,11,17,24,51,51,51
GET,/readyz,256,0,4,5.253673376955703,1.6003749915398657,35.56437499355525,39.0,15.008123373737769,0.0,4,5,6,7,9,10,13,22,36,36,36
,Aggregated,1002,0,5,6.160576064298775,1.6003749915398657,50.50283300806768,15908.996007984031,58.742732892520486,0.0,5,6,7,8,10,11,15,23,44,51,51
3 changes: 3 additions & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ dev = [
# msgpack is a pip-audit→CacheControl transitive; floor raised above the
# fixed release. GHSA-6v7p-g79w-8964 → require >=1.2.1.
"msgpack>=1.2.1",
# Load-smoke gate (Quality §2, merge-blocking): tests/perf/locustfile.py,
# driven headlessly by scripts/perf-smoke.sh / `make perf-load`.
"locust>=2.29",
]

[project.scripts]
Expand Down
Loading
Loading