docs: v0.7.0 release preparation — release notes, upgrade guide, QA hardening

README.md

@@ -12,7 +12,7 @@ Add the following dependency to your Maven project:
 <dependency>
   <groupId>ai.spice</groupId>
   <artifactId>spiceai</artifactId>
-  <version>0.6.0</version>
+  <version>0.7.0</version>
   <scope>compile</scope>
 </dependency>
 ```
@@ -22,7 +22,7 @@ Add the following dependency to your Maven project:
 Add the following dependency to your Gradle project:
 
 ```groovy
-implementation 'ai.spice:spiceai:0.6.0'
+implementation 'ai.spice:spiceai:0.7.0'
 ```
 
 ### Manual installation

docs/release_notes/v0.7.0.md

@@ -2,15 +2,46 @@
 
 ## Highlights
 
-This release is a **performance overhaul**. Parameterized queries move off ADBC onto native Arrow Flight SQL prepared statements running on the same tuned connection as regular queries, prepared statements are cached and reused, retries use real exponential backoff, and the Netty/gRPC dependency stack is aligned onto supported versions.
+v0.7.0 is a **performance and operability release**. Parameterized queries move off ADBC onto native Arrow Flight SQL prepared statements running on the same tuned connection as regular queries, prepared statements are cached and reused, retries use real exponential backoff, and the Netty/gRPC dependency stack is aligned onto supported versions. The release also delivers **mTLS client certificates** and **health, readiness, and runtime status checks**, both landing in a release for the first time.
+
+Most applications can upgrade without code changes — see the [upgrade guide](../upgrade_guides/v0.6.0-to-v0.7.0.md).
 
 ## What's New
 
+### 🔐 mTLS Client Certificates
+
+Connect to Spice deployments that require mutual TLS, and/or trust a custom certificate authority. PEM-encoded files are configured on the builder and apply to Flight (gRPC) queries, parameterized queries, and HTTP operations alike:
+
+```java
+SpiceClient client = SpiceClient.builder()
+    .withFlightAddress(new URI("grpc+tls://spice.example.com:50051"))
+    .withTlsClientCertFile("/etc/certs/client.pem")   // client identity
+    .withTlsClientKeyFile("/etc/certs/client-key.pem")
+    .withTlsRootCertFile("/etc/certs/ca.pem")          // custom CA (optional)
+    .build();
+```
+
+### 🏥 Health, Readiness, and Runtime Status
+
+Probe the runtime before sending queries, and introspect per-component state:
+
+```java
+if (!client.isHealthy()) { /* runtime is down — liveness probe (/health) */ }
+if (!client.isReady())   { /* datasets still loading — readiness probe (/v1/ready) */ }
+
+// Per-connection detail (/v1/status): http, flight, metrics, opentelemetry
+for (ConnectionDetails details : client.runtimeStatus()) {
+    System.out.println(details.getName() + ": " + details.getStatus());
+}
+```
+
+`isHealthy()`/`isReady()` return booleans (never throw on an unreachable runtime) so they can be polled directly in startup loops. See the [health and status example](/src/main/java/ai/spice/example/ExampleHealthAndStatus.java).
+
 ### ⚡ Native Flight SQL Prepared Statements (ADBC removed)
 
 `queryWithParams` is now implemented directly on Arrow Flight SQL prepared statements instead of the ADBC driver. This is a pure win with no API change — the method still returns an `ArrowReader`:
 
-- **One connection instead of two.** Parameterized queries previously ran on a separate ADBC-managed gRPC channel that had none of the SDK's transport tuning. They now share the primary channels, inheriting DNS re-resolution (`dns:///`), HTTP/2 keep-alive, and — new in this release for parameterized queries — **mTLS/custom CA support**.
+- **One connection instead of two.** Parameterized queries previously ran on a separate ADBC-managed gRPC channel that had none of the SDK's transport tuning. They now share the primary channels, inheriting DNS re-resolution (`dns:///`), HTTP/2 keep-alive, and mTLS.
 - **Prepared statement caching.** Statements are cached per SQL string and reused across executions, removing the `CreatePreparedStatement` and `ClosePreparedStatement` round trips from every repeated query (5 → 3 round trips per call). Configure with `withPreparedStatementCacheSize(n)` (default 64, `0` disables).
 - **Transparent recovery.** If the server no longer recognizes a cached statement handle (e.g. after a server restart), the SDK re-prepares and retries automatically.
 - **Multi-endpoint results.** The `ArrowReader` consumes every endpoint of a partitioned result.
@@ -34,44 +65,52 @@ SpiceClient client = SpiceClient.builder()
 
 Retries previously waited 1–2ms between attempts — useless against real outages. They now use exponential backoff with jitter (~250ms, 500ms, 1s, … capped at 10s), so the default 3 retries cover ~2s outages such as load-balancer failovers.
 
-### 🔐 Automatic Re-Authentication
+### 🔄 Automatic Re-Authentication
 
 If the server reports `UNAUTHENTICATED` (e.g. an expired handshake bearer token on a long-lived client), the SDK re-handshakes once and retries the query automatically. Previously this required a manual `reset()`.
 
+### 🛡️ Safer reset() Under Concurrency
+
+`reset()` (and the automatic re-authentication above) now *retires* old connections gracefully instead of closing them inline: in-flight queries and open result streams complete on the old transport while new queries move to fresh connections. Previously a concurrent query could fail with an allocator error, and a race with `queryWithParams` could throw `NullPointerException`.
+
+### 🐝 HikariCP / JDBC Interop
+
+`SpiceClient` is thread-safe and needs no external pool (use `withChannelCount`), but applications with JDBC infrastructure can pool connections to Spice with HikariCP via the Arrow Flight SQL JDBC driver. This combination — including authentication and prepared statements — is verified by `HikariCpCompatTest`, and the README documents the recommended configuration.
+
 ### 🧹 Smaller Fixes
 
-- Fixed a race where `reset()` concurrent with `queryWithParams` could throw `NullPointerException`.
 - Parameter bind roots are sized for their single row instead of Arrow's ~3970-slot default (~48KB saved per string parameter per query).
 - `refreshDataset` now sets a 60s request timeout (previously it could hang forever) and the HTTP client is created lazily.
 - The user agent string is computed once instead of per request; fixed the OS version reported on Windows.
-- `query()` logs a warning if the server returns a multi-endpoint result (only the first endpoint is consumed by the `FlightStream` API; use `queryWithParams` to consume all).
+- `query()` logs a warning if the server returns a multi-endpoint result (only the first endpoint is consumed by the `FlightStream` API; use `queryWithParams` to consume all), and raises a clear error if the server returns no endpoints.
 
 ## Dependency Changes
 
 | Dependency                          | v0.6.0        | v0.7.0       |
 | ----------------------------------- | ------------- | ------------ |
-| Apache Arrow ADBC Driver Flight SQL | 0.23.0        | **removed**  |
-| Apache Arrow ADBC Core              | 0.23.0        | **removed**  |
+| Apache Arrow Flight SQL             | 19.0.0        | 19.0.0       |
+| Apache Arrow ADBC Driver Flight SQL | 0.22.0        | **removed**  |
+| Apache Arrow ADBC Core              | 0.22.0        | **removed**  |
 | netty-all                           | 4.2.12.Final  | **removed**  |
-| Netty (via netty-bom)               | 4.2.12.Final (forced) | 4.1.130.Final |
-| gRPC (via grpc-bom)                 | 1.79.0        | 1.79.0 (pinned) |
+| Netty (via netty-bom)               | 4.2.12.Final (forced by netty-all) | 4.1.130.Final |
+| gRPC (via grpc-bom)                 | transitive    | 1.79.0 (pinned) |
 | netty-transport-native-epoll        | —             | 4.1.130.Final (Linux runtime) |
-| BouncyCastle (bcprov/bcpkix)        | transitive    | 1.80 (explicit) |
+| BouncyCastle (bcprov/bcpkix)        | transitive via ADBC | 1.80 (explicit) |
 
-The `netty-all 4.2.12` pin forced every Netty module to the 4.2 line, which `grpc-netty 1.79` (built against Netty 4.1.130) does not support, and dragged unused codecs (HTTP/3, MQTT, SMTP, Redis, …) onto consumers' classpaths. The stack is now aligned on Netty 4.1.130.Final via `netty-bom`, with native epoll transports added for Linux. BouncyCastle (used for mTLS PEM parsing) is now an explicit dependency instead of riding on ADBC's transitives.
-
-### 🐝 HikariCP / JDBC Interop
+The `netty-all 4.2.12` dependency forced every Netty module to the 4.2 line, which `grpc-netty 1.79` (built against Netty 4.1.130) does not support, and dragged unused codecs (HTTP/3, MQTT, SMTP, Redis, …) onto consumers' classpaths. The stack is now aligned on Netty 4.1.130.Final via `netty-bom`, with native epoll transports added for Linux. BouncyCastle (used for mTLS PEM parsing) is now an explicit dependency instead of riding on ADBC's transitives.
 
-`SpiceClient` is thread-safe and needs no external pool (use `withChannelCount`), but applications with JDBC infrastructure can pool connections to Spice with HikariCP via the Arrow Flight SQL JDBC driver. This combination — including authentication and prepared statements — is now verified by `HikariCpCompatTest`, and the README documents the recommended configuration.
+Test-scope additions (not shipped to consumers): HikariCP 5.1.0 and `flight-sql-jdbc-core` 19.0.0 for the JDBC interop tests.
 
 ## Testing
 
-- New in-process mock Flight SQL server (`TestFlightSqlServer`) with per-RPC counters and failure injection — the full query, prepared-statement, retry, timeout, re-auth, and multi-endpoint behavior is now covered without an external runtime.
-- New test suites: `LocalFlightServerTest`, `StatementCacheTest`, `ResilienceTest`, `FlightInfoReaderTest`, `ParamRootTest`, `RefreshHttpTest`, `HikariCpCompatTest`.
+- New in-process mock Flight SQL server (`TestFlightSqlServer`) with per-RPC counters and failure injection — query, prepared-statement, retry, timeout, re-auth, and multi-endpoint behavior is now covered without an external runtime.
+- New test suites: `LocalFlightServerTest`, `StatementCacheTest`, `ResilienceTest`, `FlightInfoReaderTest`, `ParamRootTest`, `RefreshHttpTest`, `HikariCpCompatTest`, `RuntimeStatusTest`.
 - New `PerfBenchmarkTest` printing latency percentiles and asserting the round-trip contract (cached path: zero prepares per query; uncached: one prepare + one close per query).
+- 284 tests in total.
 
 ## Compatibility
 
-- Public API unchanged: `query`, `queryWithParams`, `refreshDataset`, `reset`, `close`, and all existing builder methods behave as before.
+- Public API unchanged: `query`, `queryWithParams`, `refreshDataset`, `reset`, `close`, and all existing builder methods behave as before. New APIs are additive.
 - `queryWithParams` error causes are now `FlightRuntimeException` instead of `AdbcException` (both surfaced inside `ExecutionException`).
 - To restore the previous prepare-per-query behavior: `withPreparedStatementCacheSize(0)`.
+- Full details: [upgrade guide](../upgrade_guides/v0.6.0-to-v0.7.0.md).

docs/upgrade_guides/v0.6.0-to-v0.7.0.md

@@ -0,0 +1,110 @@
+# Upgrading from v0.6.0 to v0.7.0
+
+**TL;DR: for most applications this is a drop-in upgrade.** Bump the version, rebuild, done:
+
+```xml
+<dependency>
+  <groupId>ai.spice</groupId>
+  <artifactId>spiceai</artifactId>
+  <version>0.7.0</version>
+</dependency>
+```
+
+The public API is unchanged and all new capabilities are additive. Read on for the behavior changes worth knowing about, the dependency-tree changes that can affect applications with their own Arrow/Netty usage, and the new features you may want to adopt.
+
+## Behavior changes
+
+### Prepared statement caching is on by default
+
+`queryWithParams` now caches server-side prepared statements per SQL string (up to 64 idle statements) and reuses them across calls, saving two network round trips per repeated query. Statements are closed on `reset()` and `close()`, and stale handles (e.g. after a server restart) transparently re-prepare.
+
+- Impact: the Spice runtime will show long-lived prepared statements for clients that repeat queries. This is intended.
+- Opt out (restores the v0.6.0 prepare-per-query behavior):
+
+```java
+SpiceClient.builder().withPreparedStatementCacheSize(0).build();
+```
+
+### Retries now take real time
+
+v0.6.0 retried failed queries after 1–2ms waits, which effectively meant no recovery window at all. v0.7.0 uses exponential backoff with jitter (~250ms, 500ms, 1s, … capped at 10s).
+
+- Impact: with the default `withMaxRetries(3)`, a query against an unavailable runtime now surfaces its error after roughly 2 seconds instead of near-instantly. This is what makes retries actually survive load-balancer failovers and restarts.
+- If you need fail-fast behavior (e.g. latency-sensitive fallback paths), set `withMaxRetries(0)` and handle retries yourself, or set `withQueryTimeout(...)`.
+
+### Exception causes for parameterized queries changed type
+
+`queryWithParams` still throws `java.util.concurrent.ExecutionException`, but the *cause* is now an `org.apache.arrow.flight.FlightRuntimeException` (previously `org.apache.arrow.adbc.core.AdbcException`, which no longer exists on the classpath).
+
+```java
+// Before (v0.6.0)
+catch (ExecutionException e) {
+    if (e.getCause() instanceof AdbcException) { ... }
+}
+
+// After (v0.7.0)
+catch (ExecutionException e) {
+    if (e.getCause() instanceof FlightRuntimeException) {
+        FlightStatusCode code = ((FlightRuntimeException) e.getCause()).status().code();
+        ...
+    }
+}
+```
+
+If you never inspected the cause type, nothing changes.
+
+### Expired auth tokens recover automatically
+
+Long-lived authenticated clients whose handshake bearer token expires no longer fail until a manual `reset()`: the SDK re-handshakes once on `UNAUTHENTICATED` and retries. Remove any workaround code that called `reset()` for this case (it stays useful for other stuck-transport situations).
+
+### reset() is gentler to concurrent queries
+
+`reset()` now retires old connections gracefully — queries already in flight complete on the old transport, new queries use fresh connections. In v0.6.0, concurrent queries could fail with allocator errors or (in a narrow race) `NullPointerException`.
+
+### refreshDataset has a request timeout
+
+`refreshDataset` now fails after 60 seconds instead of potentially hanging forever on an unresponsive runtime.
+
+## Dependency changes that can affect your build
+
+The SDK's dependency tree changed significantly. If your application only uses the SDK's public API, no action is needed. Check the cases below if you use Arrow, Netty, ADBC, or BouncyCastle directly:
+
+| Change | Who is affected | Action |
+| ------ | --------------- | ------ |
+| **ADBC removed** (`adbc-driver-flight-sql`, `adbc-core`) | Applications importing `org.apache.arrow.adbc.*` classes that arrived transitively through this SDK | Declare the ADBC artifacts in your own build |
+| **netty-all removed** | Applications relying on Netty modules (codecs, transports) that arrived transitively via `netty-all` | Declare the specific Netty modules you use |
+| **Netty now 4.1.130.Final** (was a forced 4.2.12) | Applications pinning their own Netty version | 4.1.130.Final is the line `grpc-netty 1.79` supports; avoid forcing 4.2.x onto the SDK's classpath |
+| **gRPC pinned to 1.79.0** via `grpc-bom` | Applications pinning their own gRPC version | Align on 1.79.x, or verify your version against Netty 4.1.130 |
+| **BouncyCastle explicit at 1.80** (`bcprov-jdk18on`, `bcpkix-jdk18on`) | Applications with their own BouncyCastle pin | Maven nearest-wins applies as usual; 1.80 is only required for the SDK's mTLS PEM parsing |
+| **netty-transport-native-epoll added** (Linux classifiers, runtime scope) | Nobody adversely — enables epoll on Linux for lower-latency I/O | None |
+
+## New capabilities worth adopting
+
+```java
+SpiceClient client = SpiceClient.builder()
+    // mTLS / custom CA (PEM files) — applies to queries, parameterized queries, and HTTP
+    .withTlsClientCertFile("/etc/certs/client.pem")
+    .withTlsClientKeyFile("/etc/certs/client-key.pem")
+    .withTlsRootCertFile("/etc/certs/ca.pem")
+    // Connection pool for highly concurrent workloads (default 1)
+    .withChannelCount(4)
+    // Deadline for query planning / prepare / bind RPCs (not result streaming)
+    .withQueryTimeout(Duration.ofSeconds(30))
+    // Prepared statement reuse (default 64; 0 disables)
+    .withPreparedStatementCacheSize(128)
+    .build();
+
+// Health and readiness probes (never throw; poll them in startup loops)
+client.isHealthy();          // GET /health — liveness
+client.isReady();            // GET /v1/ready — datasets loaded
+client.runtimeStatus();      // GET /v1/status — per-connection detail
+
+// Kubernetes-style wait-for-ready
+while (!client.isReady()) { Thread.sleep(500); }
+```
+
+For JDBC ecosystems (ORMs, HikariCP): the SDK itself needs no connection pool, but pooling JDBC connections to Spice via the Arrow Flight SQL JDBC driver is supported and tested — see "Connection Pooling and HikariCP" in the README.
+
+## Supported Java versions
+
+Unchanged: Java 11+ (see the README support matrix for the tested JDK distributions).