Return a RecordBatch or a Vec<ArrayRef>

[surister]

Hi, I have a function that returns a Vec<ArrayRef>, what do you think would be more efficient, to return a RecordBatch instead and return Ok(PyRecordBatch::new(rb).to_pyarrow(py)?) or return the vector and compose the record batch in python?

And, how would one return a Vec<ArrayRef>?

[kylebarron]

It really depends on what the Vec<ArrayRef> represents. If they semantically represent multiple columns of the same length, then return them as a RecordBatch. If they represent vertical chunks of the same column then return them as a ChunkedArray or an ArrayReader.

[surister]

Yes, they are part of the same 'table'. Will return them as a RecordBatch, thank you.