Problem : we need to be able to offer similar functionalities to users even when some models are down.
For this, our dear inference provider is providing different models.
Solution : taking inspiration from the fallback of classical LLMs with the health checks, we should establish the OCR model fallback.
Scope Limitations : The use of OCR in conversations is punctual, so the fallback strategy established can and should be much simpler. After a health check or a failed call to our main OCR, we can call the fallback one.
Problem : we need to be able to offer similar functionalities to users even when some models are down.
For this, our dear inference provider is providing different models.
Solution : taking inspiration from the fallback of classical LLMs with the health checks, we should establish the OCR model fallback.
Scope Limitations : The use of OCR in conversations is punctual, so the fallback strategy established can and should be much simpler. After a health check or a failed call to our main OCR, we can call the fallback one.