feat: add refresh options for on-demand dataset refresh

[lukekim]

What

Adds RefreshOptions so callers can override a dataset's configured refresh settings for a single on-demand refresh.

using Spice;
using Spice.Datasets;

await client.RefreshDatasetAsync("taxi_trips", new RefreshOptions()
    .WithRefreshSql("SELECT * FROM taxi_trips WHERE tip_amount > 10.0")
    .WithRefreshMode(RefreshMode.Append)
    .WithMaxJitter(TimeSpan.FromSeconds(10)));

Why

RefreshDatasetAsync could only trigger a refresh using whatever was configured in the Spicepod. The runtime has always accepted per-refresh overrides on POST /v1/datasets/{name}/acceleration/refresh, and every other Spice SDK already exposes them:

SDK	Options type
Go	DatasetRefreshRequest
Python	RefreshOpts
Rust	DatasetRefreshRequest
Java	RefreshOptions
TypeScript	RefreshAccelerationOptions
.NET	— (this PR)

This was the one capability present in all five sibling SDKs and missing from .NET. Without it there is no way to run a filtered or append-mode refresh from .NET short of editing the Spicepod and redeploying.

Wire format

Maps onto the runtime's RefreshOverrides, matching the other SDKs:

Property	JSON field
RefreshSql	refresh_sql
RefreshMode (Full / Append)	refresh_mode ("full" / "append")
MaxJitter (TimeSpan)	refresh_jitter_max ("10s", "1500ms")

Unset options are omitted from the body so the runtime falls back to the dataset configuration.

MaxJitter is exposed as a TimeSpan rather than a duration string so the type system does the validation; it is formatted to the runtime's duration syntax on the way out.

Compatibility

• The existing RefreshDatasetAsync(string) overload is kept with an unchanged signature — source and binary compatible.
• That overload now posts {} with Content-Type: application/json instead of an empty body with no content type. This is what the other SDKs send; the runtime treats both as "no overrides".
• netstandard2.0 gains an explicit System.Text.Json package reference. It is in-box for .NET 8.0+, so the reference is conditioned on the netstandard2.0 target only.

Testing

22 new tests, all offline — no runtime or API key needed:

• RefreshOptionsTest — JSON field names, omission of unset options, duration formatting, enum wire values, JSON escaping of user-supplied refresh SQL, and argument validation.
• RefreshDatasetRequestTest — drives the real client against a loopback HttpListener and asserts the method, path, content type and body actually put on the wire.

dotnet build --configuration Release   # 0 warnings, 0 errors (TreatWarningsAsErrors=true)
dotnet test  --configuration Release --framework net8.0    # 137 passed, 33 skipped
dotnet test  --configuration Release --framework net9.0    # 137 passed, 33 skipped
dotnet test  --configuration Release --framework net10.0   # 137 passed, 33 skipped

Baseline on trunk was 115 passed / 33 skipped. The 33 skips are the pre-existing tests gated on SCP_SPICEAI_TPCH_API_KEY.

Follow-ups (not in this PR)

• The README documents a client.Query(sql, Dictionary<string, object>) overload with named :param placeholders in both the self-hosted and Cloud sections. No such overload has ever existed — parameterized queries are QueryWithParams with positional $1 placeholders. Copy-pasting the documented example does not compile. Worth either adding the overload or correcting the docs.
• No public API accepts a CancellationToken, and no gRPC deadline is set, so a query cannot be given a timeout or cancelled. Go and Rust both support cancellation; Java and TypeScript do not, so this is a smaller gap than the one fixed here, but it is a real one for ASP.NET Core callers wanting to propagate a request-abort token.

[lukekim]

CI note: the red is pre-existing, not from this PR

The Build and test jobs are red, but the failures are the 33 tests that run against Spice Cloud and are gated on the SCP_SPICEAI_TPCH_API_KEY secret — TestTpchQ1–TestTpchQ10 and the 23 Test_QueryWithParams_* cases. They all fail on the Flight handshake:

System.InvalidOperationException : Can't get the call trailers because the call has not completed successfully.
   at Grpc.Net.Client.Internal.GrpcCall`2.GetTrailers()

A workflow_dispatch run of unmodified trunk at the same base commit (a6a0425) fails with the identical 33 tests: https://github.com/spiceai/spice-dotnet/actions/runs/30173590110

Per-job comparison on this PR — every job that got as far as running tests reports exactly the same 33 unique failures, and zero failures among the new tests:

Job	Unique failed tests	New refresh tests failed
ubuntu-latest 8.0 / 9.0 / 10.0	33	0
windows-latest 8.0 / 9.0 / 10.0	33	0
windows-2022 8.0 / 9.0 / 10.0	33 (same handshake error)	0
macos-latest 9.0 / 10.0	33	0

All 22 tests added here pass on every platform, including Windows:

Passed Test_RefreshWithOptions_PostsOverridesToAccelerationRefreshEndpoint
Passed Test_RefreshWithoutOptions_PostsEmptyJsonObject
Passed Test_RefreshWithNullOptions_PostsEmptyJsonObject
Passed Test_RefreshWithPartialOptions_OmitsUnsetFields
Passed Test_RefreshWithBlankDatasetName_Throws
Passed Test_RefreshSqlWithQuotes_IsJsonEscaped
Passed Test_RefreshMode_MapsToLowercaseWireValues
... (22 total)

Locally, where the Cloud secret is absent and those 33 skip instead of failing, the suite is fully green on all three frameworks:

net8.0   Failed: 0, Passed: 137, Skipped: 33
net9.0   Failed: 0, Passed: 137, Skipped: 33
net10.0  Failed: 0, Passed: 137, Skipped: 33

(Baseline on trunk was 115 passed / 33 skipped, so this is +22 passing and no change in skips.)

The Cloud connectivity failure looks worth a separate look — it is unrelated to this change, which only touches the HTTP refresh path.

[Copilot]

Pull request overview

Adds first-class support in the .NET SDK for per-refresh override settings when triggering an on-demand dataset acceleration refresh, aligning the API with other Spice SDKs and the runtime wire format.

Changes:

• Introduces Spice.Datasets.RefreshOptions (+ RefreshMode) to serialize refresh overrides (refresh_sql, refresh_mode, refresh_jitter_max) and omit unset fields.
• Adds RefreshDatasetAsync(string, RefreshOptions?) overloads through SpiceClient → SpiceHttpClient, and changes the no-options path to POST {} with application/json.
• Adds offline unit/integration-style tests validating serialization and the actual HTTP request body/headers; updates README docs and adds a netstandard2.0-only System.Text.Json reference.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
SpiceTest/RefreshOptionsTest.cs	Unit tests for RefreshOptions JSON shape, omission behavior, duration formatting, and validation.
SpiceTest/RefreshDatasetRequestTest.cs	Loopback HTTP test asserting the refresh request method/path/content-type/body.
Spice/src/SpiceClient.cs	Adds public overload RefreshDatasetAsync(string, RefreshOptions?) and routes to HTTP client.
Spice/src/Http/SpiceHttpClient.cs	Implements refresh POST with {}/overrides JSON body and application/json content type.
Spice/src/Http/ISpiceHttpClient.cs	Extends the public HTTP client interface with the new overload.
Spice/src/Datasets/RefreshOptions.cs	New options + enum type; JSON serialization and duration formatting logic.
Spice/Spice.csproj	Adds conditional System.Text.Json package reference for netstandard2.0.
README.md	Documents refresh overrides and usage examples; adds an options table.

Comments suppressed due to low confidence (2)

Spice/src/Datasets/RefreshOptions.cs:99

• On the netstandard2.0 path, ThrowHelper.ThrowIfNullOrWhiteSpace throws ArgumentException even when refreshSql is null, which makes exception behavior differ across target frameworks (NET8 throws ArgumentNullException for null). Consider adding an explicit null check in the netstandard2.0 branch so callers see consistent exceptions across TFMs.

#if NET8_0_OR_GREATER
        ArgumentException.ThrowIfNullOrWhiteSpace(refreshSql);
#else
        ThrowHelper.ThrowIfNullOrWhiteSpace(refreshSql, nameof(refreshSql));
#endif

Spice/src/Datasets/RefreshOptions.cs:173

• FormatDuration uses TimeSpan.TotalMilliseconds (double) and casts to long, which truncates sub-millisecond values and can introduce rounding issues for large durations. Prefer integer math based on ticks to avoid double conversion/truncation surprises.

        var totalMilliseconds = (long)value.TotalMilliseconds;

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.