-
Notifications
You must be signed in to change notification settings - Fork 1
test/ci: enterprise-grade hardening — chaos e2e, mTLS, soak, perf trend, API gate, pipeline reliability #52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
7f47503
test/ci: enterprise-grade hardening — chaos e2e, mTLS, soak, perf tre…
lukekim 7318f33
fix: address Codex adversarial review findings
lukekim 5806576
refactor: apply /simplify review findings
lukekim 4bc43b7
fix: address PR 52 CI failures and review threads
lukekim a0086f7
fix: gate OWASP check on a completed NVD update, not database presence
lukekim File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| name: Setup Spice | ||
| description: > | ||
| Install the Spice CLI (and optionally the runtime) with retries and binary | ||
| verification. The installer resolves the latest release via the GitHub API, | ||
| which intermittently fails from runner IPs — every call site shares this one | ||
| hardened implementation. The release/publish pipeline deliberately does NOT | ||
| use this action: it pins an exact version with checksum verification. | ||
|
|
||
| inputs: | ||
| github-token: | ||
| description: Token for the installer's GitHub API release lookup. | ||
| default: ${{ github.token }} | ||
| install-runtime: | ||
| description: Also install the Spice runtime (spice install). | ||
| default: 'true' | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - name: Install Spice CLI (Unix) | ||
| if: runner.os != 'Windows' | ||
| shell: bash | ||
| env: | ||
| GITHUB_TOKEN: ${{ inputs.github-token }} | ||
| run: | | ||
| for attempt in 1 2 3 4 5; do | ||
| if curl -fsSL https://install.spiceai.org | /bin/bash && "$HOME/.spice/bin/spice" version; then | ||
| break | ||
| fi | ||
| echo "Spice CLI install failed (attempt $attempt); retrying in $((attempt * 10))s" | ||
| sleep $((attempt * 10)) | ||
| done | ||
| "$HOME/.spice/bin/spice" version | ||
| echo "$HOME/.spice/bin" >> "$GITHUB_PATH" | ||
|
|
||
| - name: Install Spice runtime (Unix) | ||
| if: runner.os != 'Windows' && inputs.install-runtime == 'true' | ||
| shell: bash | ||
| env: | ||
| GITHUB_TOKEN: ${{ inputs.github-token }} | ||
| run: $HOME/.spice/bin/spice install | ||
|
|
||
| - name: Install Spice CLI (Windows) | ||
| if: runner.os == 'Windows' | ||
| shell: pwsh | ||
| env: | ||
| GITHUB_TOKEN: ${{ inputs.github-token }} | ||
| run: | | ||
| $ok = $false | ||
| foreach ($attempt in 1..5) { | ||
| try { | ||
| curl.exe -fsSL "https://install.spiceai.org/Install.ps1" -o Install.ps1 | ||
| PowerShell -ExecutionPolicy Bypass -File ./Install.ps1 | ||
| & (Join-Path $HOME ".spice\bin\spice.exe") version | ||
| if ($LASTEXITCODE -eq 0) { $ok = $true; break } | ||
| } catch { | ||
| Write-Host "Spice CLI install failed (attempt $attempt): $_" | ||
| } | ||
| Start-Sleep -Seconds ($attempt * 10) | ||
| } | ||
| if (-not $ok) { throw "Spice CLI install failed after retries" } | ||
| Add-Content $env:GITHUB_PATH (Join-Path $HOME ".spice\bin") | ||
|
|
||
| # Native Windows runtime install is not supported by the Spice CLI | ||
| # ("Open WSL and run the Linux Spice CLI there instead") — Windows jobs | ||
| # install the CLI only and validate the SDK against the in-process suite. | ||
| - name: Runtime not installed (Windows) | ||
| if: runner.os == 'Windows' && inputs.install-runtime == 'true' | ||
| shell: pwsh | ||
| run: Write-Host "Spice runtime is not supported on native Windows; live-integration tests will self-skip." |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| name: Start Spice quickstart app | ||
| description: > | ||
| Initialize a spicepod with the taxi_trips quickstart dataset, start the | ||
| runtime in the background, and wait for /v1/ready — the single readiness | ||
| mechanism shared by every job that needs a live runtime (replaces the | ||
| flaky fixed sleeps that guessed at dataset load time). | ||
|
|
||
| inputs: | ||
| app-dir: | ||
| description: Directory name for the spice app. | ||
| default: spice_qs | ||
| ready-timeout-seconds: | ||
| description: Maximum seconds to wait for /v1/ready. | ||
| default: '120' | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - name: Init and start spice app (Unix) | ||
| if: runner.os != 'Windows' | ||
| shell: bash | ||
| run: | | ||
| spice init "${{ inputs.app-dir }}" | ||
| cd "${{ inputs.app-dir }}" | ||
| # Define the dataset inline rather than fetching spiceai/quickstart from | ||
| # the Spicepod registry, which currently returns a body the CLI cannot | ||
| # unpack ("Failed to extract Spicepod archive: Could not find EOCD"). | ||
| cat >> spicepod.yaml <<'YAML' | ||
|
|
||
| datasets: | ||
| - from: s3://spiceai-demo-datasets/taxi_trips/2024/ | ||
| name: taxi_trips | ||
| description: taxi trips in s3 | ||
| params: | ||
| file_format: parquet | ||
| acceleration: | ||
| enabled: true | ||
| YAML | ||
| spice run &> spice.log & | ||
| for i in $(seq 1 "${{ inputs.ready-timeout-seconds }}"); do | ||
| if [ "$(curl -s -m 5 http://localhost:8090/v1/ready)" = "ready" ]; then | ||
| echo "runtime ready after ${i}s" | ||
| break | ||
| fi | ||
| sleep 1 | ||
| done | ||
| [ "$(curl -s -m 5 http://localhost:8090/v1/ready)" = "ready" ] | ||
|
|
||
| # The Spice runtime does not run on native Windows, so there is no app to | ||
| # start there: Windows jobs exercise the in-process test suite only (the | ||
| # availability-gated live tests self-skip). | ||
| - name: No runtime app on Windows | ||
| if: runner.os == 'Windows' | ||
| shell: pwsh | ||
| run: Write-Host "Skipping runtime app start on Windows (native runtime unsupported)." |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.