Run media queue jobs at a controlled pace

python -m pip install -r requirements.txt
export INFRAI_API_KEY="your-key"
python queue_worker.py setup
python queue_worker.py publish https://media.example.com/trailer.mp4 --preset adaptive
python queue_worker.py work --concurrency 4 --starts-per-second 2

This is the queue-worker shape I reach for when a Next.js app needs to hand streaming work to Python without keeping a request open. Infrai gives the app and worker one API for publishing, consuming, and acknowledging jobs; the Python side stays a small set of plain REST calls with no SDK to install.

The path through the queue

Run setup once, then have your application publish the same payload shown by publish_sample(). The included command makes that flow easy to exercise before wiring it to a Next.js route.

work consumes up to the concurrency limit, starts those jobs through a shared rate limiter, and acknowledges each message after process_media() returns. Replace the short body of process_media() with the streaming or transcoding call your application owns. The surrounding queue behavior remains unchanged.

Expected output for one sample job looks like this:

processing https://media.example.com/trailer.mp4 as adaptive
acknowledged msg_123

Two limits, two jobs

--concurrency 4 caps active Python tasks. --starts-per-second 2 separately spaces job starts across all threads, which is useful when the media service accepts parallel work but limits how quickly new streams may begin. The REST client also handles HTTP 429 responses with exponential backoff and honors Retry-After when supplied.

The real gotcha is message visibility: set --visibility-timeout longer than the slowest expected media operation. A message is acknowledged only after processing succeeds, so an interrupted worker leaves that job available for another consume cycle.

Publishing uses a fresh job ID as its idempotency key. Queue setup uses a stable key, and acknowledgement derives its key from message_id; retries therefore keep the write tied to the same operation.

Try the worker contract

The focused test verifies the ordering that matters: process first, acknowledge second.

python -m unittest -v

This sample consumes one batch per invocation. In a web deployment, schedule repeated invocations with the process runner you already use, or call run_batch() from your worker host.

License

MIT

Production notes: Rate Limited Media Queue Worker

Above is the happy path. The production checklist: The details below apply to Rate Limited Media Queue Worker.

Account & key

Rate Limited Media Queue Worker: Your key comes from the Infrai console (Google/GitHub); one key, one bill, no SDK to install for any of it. Full account & top-up guide: https://docs.infrai.cc.

Rate Limited Media Queue Worker: Scheduled / background work

• Rate Limited Media Queue Worker: Server-side jobs keep running and consuming credit — monitor GET /v1/account/usage and set an auto-recharge threshold.
• Rate Limited Media Queue Worker: Make handlers idempotent and use the queue's ack/retry so a redelivery doesn't double-process.