Speedup the pipeline

[rafelafrance]

Right now I'm running an entire batch of OCR and then an entire batch of parsing, etc.

This is wasteful on thruput. I need to get a small batch of OCR done and feed that into the parser while the rest of the OCR is continuing. It's a story older than computing.

[rafelafrance]

First create a pipeline in a single module and then imbricate small batches thru the pipeline. Of course I should allow for multiple jobs to run at the same time. For instance: I should allow a local OCR and at the same time an OCR job on my old workstation.

[rafelafrance]

Maybe try speculative decoding and other techniques for speeding LLM inference.

No thinking << MTP

[rafelafrance]

Try merging the OCR and parsing steps into a single LLM request.

I will need to measure this to see if it works OK