Skip to content

ci: define the test dataset inline instead of fetching it from the registry - #78

Open
lukekim wants to merge 1 commit into
trunkfrom
fix/ci-inline-quickstart-dataset
Open

ci: define the test dataset inline instead of fetching it from the registry#78
lukekim wants to merge 1 commit into
trunkfrom
fix/ci-inline-quickstart-dataset

Conversation

@lukekim

@lukekim lukekim commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

What

Replaces spice add spiceai/quickstart with the same dataset declared inline in the generated spicepod. Both the Unix and WSL setup steps.

Why

Every job currently dies before a single test runs:

ERROR Invalid argument: Failed to extract Spicepod archive: invalid Zip archive: Could not find EOCD
##[warning]attempt 3/3 failed: spice add spiceai/quickstart

The Spicepod registry is returning a body the CLI can't unpack. The retry wrapper doesn't help — all three attempts fail the same way. Nothing about the runtime or the SDK is broken; only the fetch is.

This reproduces off CI too: running spice add spiceai/quickstart on a local machine gives the identical error, so it isn't a runner or network quirk.

Pulling a fixture from a remote registry means every PR in this repo depends on that service being healthy. Declaring the dataset inline removes the dependency, and it's the same data spiceai/quickstart provides.

Mirrors spicepy#176, which takes the same approach and is green.

Verification

Ran the exact spicepod this produces against a local runtime:

$ spice init qs && cd qs && cat >> spicepod.yaml <<'YAML' ... YAML
$ spiced
QUERYABLE after 6s
[{"n":2964624}]
  • taxi_trips loads and is queryable within seconds
  • Workflow YAML parses, and the heredoc dedents correctly under the run: | block scalar — bash receives valid, unindented YAML
  • spice init emits version: v2, so the generated spicepod is on the current version

Note the cloud-backed tests (tpch.customer, eth.recent_blocks) are unaffected — they authenticate with SPICE_API_KEY and don't come from the quickstart pod.

…gistry

`spice add spiceai/quickstart` makes every run depend on the Spicepod registry.
That service is currently returning a body the CLI cannot unpack, so every job
fails at:

  Invalid argument: Failed to extract Spicepod archive: invalid Zip archive:
  Could not find EOCD

The runtime and the SDK are fine; only the fetch is broken. Declaring the same
dataset spiceai/quickstart provides removes the dependency.

Applies to both the Unix and WSL setup steps. Verified locally: taxi_trips
loads and is queryable within seconds (2,964,624 rows).

Mirrors spicepy#176.
@claudespice

Copy link
Copy Markdown
Contributor

The six red Build and test jobs here are a stale run, not a defect in this change — a re-run should clear the board.

All six come from a single run, 30311535133, dispatched 2026-07-27T22:40Z. The failures are Spice.ai Cloud authentication, not the dataset inlining:

query_test.go:247: Sql with auth failed: rpc error: code = Unauthenticated desc = invalid Basic authentication
adbc_test.go:44:  error opening ADBC connection: Unauthenticated: [FlightSQL] invalid Basic authentication
--- FAIL: TestSqlWithAuth
--- FAIL: TestADBCCloudBasicQuery   (all 3 subtests)

TestADBCCloudBasicQuery doesn't touch the quickstart dataset at all, which was the first hint the cause is environmental.

Both tests now pass in this repo on the same workflow and the same matrix leg. From PR #77's run 30498517724, Build and test ubuntu-latest go1.26, 2026-07-29T23:08:41Z:

--- PASS: TestADBCCloudBasicQuery (1.29s)     (all 3 subtests)
--- PASS: TestSqlWithAuth (1.01s)             (both subtests)

The local half of this PR's change is also visibly working in the failing run's own logs — the runtime came up and registered the dataset before the auth tests ran:

Dataset taxi_trips registered (s3://spiceai-demo-datasets/taxi_trips/2024/), acceleration (arrow)
Loaded 2,964,624 rows (399.38 MiB) for dataset taxi_trips in 3s 252ms.
All components are loaded. Spice runtime is ready!

I can't re-run it from here (Must have admin rights to Repository), so this needs someone with write access: Re-run failed jobs on run 30311535133, or push any commit to fix/ci-inline-quickstart-dataset.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants