Go runtime lifecycle with dlopen: multi-driver loading and dlclose safety

[metegenez]

Multiple Go-based ADBC drivers in the same process

I'm integrating the ADBC C driver manager into StarRocks (a C++ database) via dlopen to load drivers at runtime.

When two cgo -buildmode=c-shared drivers (e.g., Flight SQL + Snowflake) are dlopen-ed into the same process, each embeds its own Go runtime, meaning two goroutine schedulers, two sets of cgo signal handlers, and competing SIGURG-based async preemption. The static-linking docs warn against linking more than one Go driver, but there's no guidance for the dynamic-loading case. Is this expected to work, or would it make sense for the driver manager to expose a metadata flag so embedders can detect and refuse the conflict at catalog-creation time?

dlclose on Go-based driver shared libraries

I noticed that ADBC 22's Rust driver manager stopped calling dlclose on driver libraries because Go drivers hang on some platforms (referencing issue #3844 and the decade-old golang/go#11100). The C driver manager, though, seems to leave dlclose behavior up to the embedder. I couldn't find a documented contract saying "never call dlclose on a driver handle." For my integration I've adopted a never-dlclose policy (driver handles stay resident until process exit, teardown goes through AdbcStatementRelease → AdbcConnectionRelease → AdbcDatabaseRelease only), but I had to piece this together from the Rust manager's changelog rather than from the C API docs.

Does this make sense to you guys?

@ianmcook @lidavidm

[lidavidm]

The static-linking docs warn against linking more than one Go driver, but there's no guidance for the dynamic-loading case. Is this expected to work, or would it make sense for the driver manager to expose a metadata flag so embedders can detect and refuse the conflict at catalog-creation time?

This will not work on Intel macOS due to a bug in Go that the maintainers have deemed as not worth fixing. We expect it works on other platforms when dynamically linked. It will not work for static linking again because of an issue in Go.

I couldn't find a documented contract saying "never call dlclose on a driver handle." For my integration I've adopted a never-dlclose policy

In principle, I would like drivers to be dlclose-able. In practice, Go again makes this impossible; we should document this. I also have never tested the Rust drivers to see if this works or not.