Wondering if it is possible to add streaming support for models hosted on Sagemaker. This example makes it seem possible, but the implementation looks awkward.
https://github.com/aws/amazon-sagemaker-examples/blob/main/inference/generativeai/llm-workshop/lab6-stream-with-pagination/stream_pagination_lmi.ipynb
Wondering if it is possible to add streaming support for models hosted on Sagemaker. This example makes it seem possible, but the implementation looks awkward.
https://github.com/aws/amazon-sagemaker-examples/blob/main/inference/generativeai/llm-workshop/lab6-stream-with-pagination/stream_pagination_lmi.ipynb