`base64ct::Encoder` with alloc?

[tisonkun]

I'm migrating from base64 and notice that the following code snippet cannot easily migrate:

// before
    let mut buf = vec![];
    {
        let encoder = base64::write::EncoderWriter::new(&mut buf, &BASE64_STANDARD);
        let mut writer =
            arrow::ipc::writer::StreamWriter::try_new(encoder, &schema).or_raise(make_error)?;

        for batch in batches {
            writer.write(&batch).or_raise(make_error)?;
        }

        writer.finish().or_raise(make_error)?;
    }
    String::from_utf8(buf).or_raise(make_error)

// after
    let mut buf = vec![];
    {
        let encoder = base64ct::Encoder::<base64ct::Base64>::new(&mut buf).unwrap();
        let mut writer =
            arrow::ipc::writer::StreamWriter::try_new(encoder, &schema).or_raise(make_error)?;

        for batch in batches {
            writer.write(&batch).or_raise(make_error)?;
        }

        writer.finish().or_raise(make_error)?;
    }
    String::from_utf8(buf).or_raise(make_error)

Note that base64ct::Encoder accepts a &mut [u8] and thus you must know the result length beforehand.

In the contrast, base64::write:EncoderWriter takes W: io::Write so it takes &mut Vec<u8> whose write can grow the capacity on demand.

Although base64ct is defined to best effort being constant time, since it provides allloc feature for convience already, is it reasonable to implement a (new) Encoder that allocs on demand?

cc @tarcieri

[tarcieri]

We don't have a buffered encoder like this which can control allocations in a Vec as-you-go, no.

Unless it's being used to implement a cryptographic format, I'm not sure I want to take on the burden of maintaining the extra complexity. base64ct is less about being a full-featured replacement for base64 as it is a library whose goal is to be used to implement cryptographic encoding formats.

I might suggest starting by implementing something like that out-of-tree in its own crate, and if it turns out to be generally useful, we can consider upstreaming it.

[tisonkun]

Thanks for your suggestion. Let me try to make a crate to deliver the functionality and see if it helps.