Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,8 @@ venv/

# Superpowers plugin
.superpowers/
docs/superpowers/
docs/superpowers/*
!docs/superpowers/baselines/

# Docker & Helm (bundled separately)
Dockerfile
Expand All @@ -89,3 +90,7 @@ helm/
neo4j-data/
graph.db/
.worktrees/

# Phase A baseline
.seeds/
docs/superpowers/baselines/**/raw/**
285 changes: 285 additions & 0 deletions docs/superpowers/baselines/2026-04-17/BASELINE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,285 @@
# code-iq Baseline — 2026-04-17

This file is generated by `scripts/baseline/consolidate.sh`. Re-run after
updating any capture script. Raw artifacts under `raw/` are gitignored.

## Toolchain

- Java: openjdk version "25.0.2" 2026-01-20 LTS
- Maven: Apache Maven 3.8.7
- Node: v24.15.0
- npm: 11.12.1

## Maven build & tests

```json
{
"tests": 3059,
"failures": 0,
"errors": 0,
"skipped": 31
}
```

## Coverage (JaCoCo)

```json
{
"inst_covered": 82247,
"inst_missed": 10270,
"br_covered": 5931,
"br_missed": 2388,
"line_covered": 16515,
"line_missed": 1990,
"inst_pct": 88.9,
"br_pct": 71.29,
"line_pct": 89.25
}
```

## Flaky tests

```json
{
"runs": 3,
"failures_per_run": [
0,
0,
0
],
"always_failing": [],
"flaky": []
}
```

## SpotBugs

```json
{
"total_bugs": 1492,
"by_priority": {
"2": 1484,
"1": 8
},
"by_category": {
"STYLE": 546,
"MALICIOUS_CODE": 203,
"I18N": 1,
"BAD_PRACTICE": 736,
"MT_CORRECTNESS": 1,
"PERFORMANCE": 4,
"CORRECTNESS": 1
},
"top_patterns": [
[
"NM_METHOD_NAMING_CONVENTION",
730
],
[
"SF_SWITCH_NO_DEFAULT",
448
],
[
"EI_EXPOSE_REP2",
77
],
[
"MS_PKGPROTECT",
60
],
[
"BC_UNCONFIRMED_CAST",
55
],
[
"EI_EXPOSE_REP",
46
],
[
"NP_NULL_ON_SOME_PATH_FROM_RETURN_VALUE",
26
],
[
"MS_FINAL_PKGPROTECT",
20
],
[
"DLS_DEAD_LOCAL_STORE",
5
],
[
"SF_SWITCH_FALLTHROUGH",
4
],
[
"UC_USELESS_OBJECT",
3
],
[
"CT_CONSTRUCTOR_THROW",
2
],
[
"REC_CATCH_EXCEPTION",
2
],
[
"WMI_WRONG_MAP_ITERATOR",
2
],
[
"RCN_REDUNDANT_NULLCHECK_OF_NONNULL_VALUE",
2
],
[
"ES_COMPARING_STRINGS_WITH_EQ",
2
],
[
"DB_DUPLICATE_BRANCHES",
1
],
[
"DM_DEFAULT_ENCODING",
1
],
[
"UL_UNRELEASED_LOCK_EXCEPTION_PATH",
1
],
[
"UPM_UNCALLED_PRIVATE_METHOD",
1
]
]
}
```

## OWASP dependency-check

```json
{
"status": "FAILED",
"reason": "NVD DB update race: UpdateException (H2 lock) + NoDataException during first NVD sync. Re-run after clearing ~/.m2/repository/org/owasp/dependency-check-data/*.lock and optionally wiping the data dir.",
"captured_log": "docs/superpowers/baselines/2026-04-17/raw/depcheck.log",
"maven_exit_code": 1,
"timestamp": "2026-04-17T08:06:05Z",
"by_severity": {},
"top_25": []
}
```

## Frontend

- Playwright:
```json
{
"passed": 0,
"failed": 575,
"skipped": 0
}
```
- Full logs: `raw/frontend/` (local only).

## Pipeline on seed repos

### spring-petclinic
```json
{
"seed": "spring-petclinic",
"timings": [
"index duration=8s rc=0",
"enrich duration=13s rc=0",
"health=fail"
],
"stats": null,
"health_ok": false
}
```

### realworld-express
```json
{
"seed": "realworld-express",
"timings": [
"index duration=5s rc=0",
"enrich duration=10s rc=0",
"health=fail"
],
"stats": null,
"health_ok": false
}
```

## Known gaps / issues

Ordered by severity. Each item cites the raw artifact it was derived from.

### Critical

- **OWASP dependency-check failed.** NVD initial sync hit `UpdateException: Unable to obtain exclusive lock on H2 database` followed by `NoDataException: No documents exist`. Maven exit 1 after 40 min. No CVE inventory captured. Must re-run (see §Re-run instructions below) before any security posture claim.
- Raw: `raw/depcheck.log`, `raw/depcheck-summary.json` (stub, `status=FAILED`).

- **Playwright E2E: 0 passed / 575 failed.** 100% failure rate. Almost certainly environment/config rather than regressions — the audit script runs `npm run test:e2e` without starting the backend (`java -jar ... serve`), so any test that hits `/api/*` will fail. Needs a harness that spins up the server (or mocks it) before running Playwright, or a `webServer` entry in `playwright.config.ts`.
- Raw: `raw/frontend/playwright.log`, `raw/frontend/playwright-summary.json`.

### High

- **Pipeline serve-smoke failed on both seed repos** (`health=fail`, `stats=null`). `index` and `enrich` succeeded (petclinic 8+13s, express 5+10s) but the 8-second sleep between starting `serve` and `curl /actuator/health` is at the low end of the documented 8–16s Spring Boot + embedded Neo4j cold-start window (see CLAUDE.md §Gotchas). Fix in Phase F hardening: poll `/actuator/health` with a retry budget instead of a fixed sleep.
- Raw: `raw/pipeline/spring-petclinic/`, `raw/pipeline/realworld-express/`.

- **SpotBugs: 8 HIGH-priority findings (priority=1) + 1,484 at priority=2.** Total 1,492. HIGH findings must be triaged individually (read `raw/spotbugs.xml`). Noise-dominant rules (`NM_METHOD_NAMING_CONVENTION`=730, `SF_SWITCH_NO_DEFAULT`=448) should be filtered via a SpotBugs exclude file so real signal surfaces; real-concern patterns that deserve review now: `NP_NULL_ON_SOME_PATH_FROM_RETURN_VALUE` (26), `BC_UNCONFIRMED_CAST` (55), `UL_UNRELEASED_LOCK_EXCEPTION_PATH` (1), `WMI_WRONG_MAP_ITERATOR` (2), `ES_COMPARING_STRINGS_WITH_EQ` (2), `MT_CORRECTNESS` category (1).
- Raw: `raw/spotbugs.xml`, `raw/spotbugs-summary.json`.

### Medium

- **Branch coverage 71.3% is notably below instruction coverage 89.0%.** Expected for a detector-heavy codebase, but targeted branch coverage on the enrichment / linker / LayerClassifier paths (which drive deterministic output) is worth a focused improvement pass in Phase E.
- Raw: `raw/coverage-summary.json`, `raw/jacoco.csv`.

- **31 skipped tests.** Not investigated. Read surefire reports to confirm they're intentional (`@Disabled` / profile-gated) and not silently excluded.
- Raw: `raw/surefire-reports.tar`.

### Low / noise

- `consolidate.sh` prints the Maven version with raw ANSI escape codes (`[1mApache Maven 3.8.7[m`). Strip with `sed 's/\x1b\[[0-9;]*m//g'` in a follow-up. Cosmetic only.

### Green

- **3,059 tests, 0 failures, 0 errors.** Clean.
- **Flaky scan: 0 always-failing, 0 flaky across 3 runs.** Suite is stable.
- **Instruction coverage 89.0%**, line coverage 89.25%. Strong baseline.
- **`npm audit` + Vite build: no blocking issues** recorded in the capture.
- **Pipeline `index` and `enrich` succeeded deterministically** on both seed repos.

## Re-run instructions (for blocked captures)

### OWASP dep-check
```bash
# 1. Stop any lingering dep-check processes
pkill -f dependency-check 2>/dev/null
# 2. Clear NVD locks (and optionally wipe the partial DB)
rm -f ~/.m2/repository/org/owasp/dependency-check-data/11.0/*.lock
# rm -rf ~/.m2/repository/org/owasp/dependency-check-data/11.0 # fallback if DB is corrupt
# 3. Re-run
./scripts/baseline/run-depcheck.sh
```

### Pipeline serve-smoke
Patch `scripts/baseline/run-pipeline.sh` to replace the `sleep 8` with a poll loop:
```bash
for _ in $(seq 1 60); do
if curl -sf "http://127.0.0.1:$PORT/actuator/health" > "$OUT/health.json"; then break; fi
sleep 2
done
```
Then re-run `./scripts/baseline/run-pipeline.sh spring-petclinic` and `realworld-express`.

### Playwright E2E
Add a `webServer` entry to `src/main/frontend/playwright.config.ts` that starts the code-iq server against a fixture repo, or supply a mock backend. Then re-run `./scripts/baseline/run-frontend-audit.sh`.

## Handoff to subsequent phases

- **Phase B (unified config)** — `codeiq.yml` smoke test against both seed repos; validation script gates CI.
- **Phase D (MCP robustness)** — pipeline serve-smoke fix above is a prerequisite for any MCP contract test.
- **Phase E (determinism)** — `index → enrich` reproducibility on the two seed repos above is the seed for graph-snapshot diffing; 31 skipped tests to triage.
- **Phase F (ops & perf)** — Playwright harness + cold-start budgets.
Empty file added scripts/.gitkeep
Empty file.
Empty file added scripts/baseline/.gitkeep
Empty file.
92 changes: 92 additions & 0 deletions scripts/baseline/consolidate.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
#!/usr/bin/env bash
# Build BASELINE.md from captured raw/ artifacts. Idempotent; safe to re-run.
set -euo pipefail
RAW="docs/superpowers/baselines/2026-04-17/raw"
OUT="docs/superpowers/baselines/2026-04-17/BASELINE.md"

read_json() { [[ -f "$1" ]] && cat "$1" || echo "null"; }

TESTS=$(read_json "$RAW/test-counts.json")
COV=$(read_json "$RAW/coverage-summary.json")
FLAKY=$(read_json "$RAW/flaky.json")
SB=$(read_json "$RAW/spotbugs-summary.json")
DC=$(read_json "$RAW/depcheck-summary.json")
FRONT_PW=$(read_json "$RAW/frontend/playwright-summary.json")
PL_PC=$(read_json "$RAW/pipeline/spring-petclinic/summary.json")
PL_RW=$(read_json "$RAW/pipeline/realworld-express/summary.json")

JAVA_V=$(java -version 2>&1 | head -n1)
MVN_V=$(mvn -v 2>&1 | head -n1)
NODE_V=$(node --version)
NPM_V=$(npm --version)

cat > "$OUT" <<MD
# code-iq Baseline — 2026-04-17

This file is generated by \`scripts/baseline/consolidate.sh\`. Re-run after
updating any capture script. Raw artifacts under \`raw/\` are gitignored.

## Toolchain

- Java: $JAVA_V
- Maven: $MVN_V
- Node: $NODE_V
- npm: $NPM_V

## Maven build & tests

\`\`\`json
$TESTS
\`\`\`

## Coverage (JaCoCo)

\`\`\`json
$COV
\`\`\`

## Flaky tests

\`\`\`json
$FLAKY
\`\`\`

## SpotBugs

\`\`\`json
$SB
\`\`\`

## OWASP dependency-check

\`\`\`json
$DC
\`\`\`

## Frontend

- Playwright:
\`\`\`json
$FRONT_PW
\`\`\`
- Full logs: \`raw/frontend/\` (local only).

## Pipeline on seed repos

### spring-petclinic
\`\`\`json
$PL_PC
\`\`\`

### realworld-express
\`\`\`json
$PL_RW
\`\`\`

## Known gaps / issues

Populate this section manually after reading the raw outputs. Examples to
look for: failing tests, flaky set non-empty, SpotBugs HIGH priority bugs,
CVEs ≥ CRITICAL, coverage hot spots.
MD
echo "Wrote $OUT"
Loading
Loading