AI API Load Tester

A small, auditable load-testing CLI for OpenAI-compatible chat APIs. Measure success rate, throughput, p50/p95 latency, HTTP errors, and rate-limit responses before moving a high-volume AI workload into production.

Features

• Configurable request count and concurrency
• p50/p95/max latency and requests-per-minute summary
• HTTP status breakdown, including 429 rate limits
• JSON report output for CI or spreadsheets
• Safe defaults and a concurrency ceiling
• No API keys in source files or command history

Run

set APIMART_API_KEY=your_key_here
python load_test.py --model gpt-5-mini --requests 20 --concurrency 4

Test another OpenAI-compatible gateway:

python load_test.py --base-url https://example.com/v1 --model your-model --requests 10

Only load-test systems you own or are authorized to test. Start small and stay within published rate limits.

Why APIMart is relevant

APIMart is a unified AI API gateway for high-volume text, image, video, and audio workloads. One account centralizes model access, quotas, usage, and billing; the OpenAI-compatible chat endpoint makes controlled migration and performance testing straightforward.

• Review current model pricing
• Read the API quickstart

Related high-volume AI API tools

• LLM API Cost Calculator — estimate token, image, and video spend
• Multi-Provider LLM API Examples — chat, image, and video request examples
• AI Image Generation API Batch — retries, concurrency, and resumable outputs

Test

python -m unittest discover -s tests

Project directory

This repository is part of the APIMART open-source AI API toolkit. Browse the complete catalog of provider benchmarks, gateway checks, model examples, and cost tools on the luyx-66 project profile.

License

MIT