New Arrow .NET release

[raulcd]

The last release for Arrow .NET was ~6 months ago.

A user raised on the ML the following request:
https://lists.apache.org/thread/xyf26tqmznyytzydx2xmlvto193fjwws

I think it's reasonable to kick a new release. I might have some time to do it, I have to check a couple of things on my end but as I plan to kick the apache/arrow 24.0.0 release in a couple of weeks I might be able to do those in parallel.

@adamreeve @CurtHagenlocher @kou what are your thoughts? What should be the release? 22.2.0, 23.0.0? Is there anything you were hoping to get in the release?

[CurtHagenlocher]

Wow, I hadn't realized it's been quite this long. There are a few things in flight, so let's set a cutoff of April 1st for a new release.

[raulcd]

Is it an April's fool? Just joking :)
The feature freeze for Arrow 24.0.0 is scheduled on the 7th of April, after Easter, in case it's relevant.

[CurtHagenlocher]

There's not a specific reason we need to align with Arrow/C++ and it might be easier to space them. That said, I have no specific preference in that regard. April 7th would give us extra time to try to push through some of the open PRs.

I have to admit I haven't been paying much attention to the changes in the main repo. Are there any big features we might want to align with? E.g. if pyarrow has variant support, then that's an argument for trying to finish the remaining tasks for variants in this repo.

[CurtHagenlocher]

Okay, before performing the release I'd like to finish six things:

0. Add overflow checking to conversions in the IPC code path.
1. Factor JSON support out of Apache.Arrow.Variant into Apache.Arrow.Operations.
2. Rename Apache.Arrow.Variant to Apache.Arrow.Scalars.
3. Move some scalar types to Apache.Arrow.Scalars (validating that it doesn't break compatibility).
4. Add some shredding support to variants.
5. Add VariantArray as an extension type in Apache.Arrow.

Many of these are already done locally and simply need to be cleaned up and submitted for review. 0-2 I think are required before the release. 3-5 are more optional, and I suspect the wiser thing to do would be to hold off on at least 4 and 5.

[CurtHagenlocher]

As of April 22nd, I have a PR for 0 that I need to make a few changes to this evening. I've implemented 1 and 2 (but have to fix a few namespaces that now seem wrong -- I'll do that shortly). I have a PR for 5 that's ready for review. I will defer 3 entirely and decide on deferring 4 in the next day. I hope to start the release process by tomorrow evening PDT.

[CurtHagenlocher]

Okay, I have one last PR with a breaking change related to Variant (#342) and then I think we're good to roll a release.

[bparks13]

Hi @CurtHagenlocher, I just opened a PR that bumps a version number for a dependency in the compression library, would it be possible to have that be a part of this release cycle? Thanks