Discussion: ADBC Driver for GaussDB

[metegenez]

Summary

Proposing an ODBC adapter driver for ADBC, analogous to the existing JDBC adapter (java/driver/jdbc/). This driver would wrap ODBC data sources and expose them through the ADBC interface, converting row-oriented ODBC result sets into Arrow columnar format.

Motivation

ADBC currently has a JDBC adapter, but it is Java-only and cannot be used from C/C++, Go, Python (without JNI overhead), or Rust. An ODBC adapter would complement the JDBC adapter by providing a language-agnostic, C-level bridge to any ODBC-accessible database.

Current State

• The JDBC adapter (java/driver/jdbc/) demonstrates the wrapper pattern: JdbcDriver → JdbcConnection → JdbcStatement → Arrow conversion via JdbcToArrow.
• We already have a library that implements odbc to arrow: https://github.com/pacman82/arrow-odbc

If this is already discussed and rejected, I couldn't see a related issue.

[CurtHagenlocher]

One thing to be aware of for multi-row binding is that you generally need to preallocate enough space to hold the maximum-sized values for a particular column. This becomes problematic when the table has IMAGE or TEXT columns with a very large upper bound on the size. Single-row binding is usually okay, because you can bind a smaller buffer and then fetch that entire value explicitly with SQLGetData when there's an overflow. But this doesn't usually work for multi-row binding because most drivers don't let you go back and do SQLGetData against a previous row -- you need (IIRC) a local cursor for that.

[metegenez]

thanks for the input man, i will definitely think about that. this is quite a big effort. I will try to bring a plan first as soon as possible.

[ianmcook]

Hi @metegenez, this sounds like a potentially valuable adapter, but let us know which specific databases and query engines you want ADBC to support, and we might have a more direct solution for you. We're building new ADBC drivers fast and furious in the adbc-drivers contrib organization, and we're always looking for input on which drivers to build next. In general, an ADBC driver that's built from scratch and tuned for performance will be faster and less buggy than an ADBC driver that's created by wrapping an existing ODBC driver.

[metegenez]

It is not open-source, unfortunately. It is Huawei's GaussDB, and it is not the same as openGauss (PostgreSQL-based). I could try to implement it in a closed-source way; however, handling complex security connection problems is not trivial. An ODBC adapter might help those in our condition in an enterprise setting.

[ianmcook]

There are several closed source ADBC drivers. Closed source is a common first step for building new drivers that work with commercial database systems. Let us know if you're interested to try to build one for GaussDB in a private repo that you control. We have several tools such as driver templates for Go and Rust and the validation framework which make this straightforward and tractable for agentic coding workflows. Later we can discuss how to work with you to make this open source and wire it up to a Huawei Cloud account that the ADBC Drivers organization controls for CI.

[metegenez]

How about CPP? I plan to implement this CPP. Making it opensource or giving this type of decision is way above me tbh :d

edit: If I code this in Rust, can I distribute this into cpp and java directly via their driver-manager?

edit2: I should be able to distribute "libadbc_driver_gsc.so" that i create, your rust driver template helped a lot btw, thanks for that. If you have any recommendation distributing this, it'd be really good. Basically, i need to rewrite StarRocks/starrocks#70297 this PR's driver loading.

[lidavidm]

@metegenez glad you got something working! Does this mean you're interested in open-sourcing it? (It should work with C++ and the Java JNI bindings)

(Also note that we don't necessarily get pings for edits - it would be great if you could reply to a comment so we're more likely to see it!)

[metegenez]

to be honest, i am not sure we can opensource it right away. maybe next year we can push it, we are willing to contribute open-source as much as we can.

[ianmcook]

How about CPP

Yes, C++ works too. See https://arrow.apache.org/adbc/23/cpp/driver_example.html

If I code this in Rust, can I distribute this into cpp and java directly via their driver-manager?

Yes. You can compile the driver to a shared library (.so / .dylib / .dll) and load it with the ADBC driver manager for any language.

If you have any recommendation distributing this, it'd be really good.

To distribute an ADBC driver privately for internal use, I recommend creating a package that includes the shared library and a driver manifest.

To distribute a production-quality open source ADBC driver for public use, start a discussion at github.com/adbc-drivers and we can help you get the driver on the public driver registry so anyone can use dbc to install it.

[metegenez]

I implemented a v0.0.1 in Rust, thanks for the support.

Yes. You can compile the driver to a shared library (.so / .dylib / .dll) and load it with the ADBC driver manager for any language.

Is there a minimum version ADBC for this capability?

[ianmcook]

Is there a minimum version ADBC for this capability?

The minimum version for this is quite old, but I recommend setting at least ADBC 19 as the minimum in version constraints, because that's the release that added support for driver manifests: https://arrow.apache.org/blog/2025/07/08/adbc-19-release/