Once #8709 is done, we'll have a benchmark we can use to try to improve the REE casting performance. @alamb and @tustvold have a neat suggestion to use the filter kernel:
I think what @tustvold is getting at is that the filter kernel is (very) fast and thus using it rather than the take kernel is likely to be faster.
So I think the idea here is instead of building up value_indexes it would be to implement some way to turn the result of Partition into a boolean array and pass to the filter kernel
let partitions = partition(&[Arc::clone(cast_array)])?;
// ....
let filter: BooleanArray = partitions.into_inner(); // not sure about this API
// call filter instead of take
let values_array = filter(&cast_array, &filter)?;
Originally posted by @alamb in #8589 (comment)
Once #8709 is done, we'll have a benchmark we can use to try to improve the REE casting performance. @alamb and @tustvold have a neat suggestion to use the
filterkernel:Originally posted by @alamb in #8589 (comment)