Skip to content

docs: add governed endpoint benchmark example#868

Closed
cerebrixos wants to merge 1 commit into
vllm-project:mainfrom
cerebrixos:tuning-engines-benchmark-example
Closed

docs: add governed endpoint benchmark example#868
cerebrixos wants to merge 1 commit into
vllm-project:mainfrom
cerebrixos:tuning-engines-benchmark-example

Conversation

@cerebrixos

@cerebrixos cerebrixos commented Jun 27, 2026

Copy link
Copy Markdown

This adds a docs example for benchmarking a governed OpenAI-compatible endpoint with GuideLLM.

Why this felt useful here:

  • GuideLLM already targets OpenAI-compatible endpoints through openai_http
  • many teams benchmark a raw model server, but deploy a more complex production path
  • this example shows how to measure the user-visible latency and throughput of the full governed path without changing GuideLLMs normal backend model

The concrete example uses Tuning Engines, but the workflow applies to any OpenAI-compatible control plane or gateway.


git log

commit c97fdd7
Author: cerebrixos 248534569+cerebrixos@users.noreply.github.com
Date: Sat Jun 27 10:49:19 2026 -0400

docs: add governed endpoint benchmark example

@mergify

mergify Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Hi @cerebrixos, the DCO check has failed. Please click on DCO in the Checks section for instructions on how to resolve this.

@cerebrixos

Copy link
Copy Markdown
Author

I opened #869 as a signed-off replacement for this PR so the DCO check can pass cleanly. Closing this one to keep review on the compliant branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant