Add SGLang Qwen3-8B baseline benchmark harness#263
Closed
chhwang wants to merge 5 commits into
Closed
Conversation
…mpatible SGLang image (CUDA 12.x, record tag+digest), launch Qwen3-8B TP=8 best config, measure batch=1 prefill TTFT (prompt 2048) and decode per-token latency (generate 128), capture GPU clocks. Deliver examples/qwen3/bench/ (launch+measure scripts), BASELINE.md with numbers and reproduce.md. Branch qwen3-q1-baseline off main.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #263 +/- ##
=======================================
Coverage 85.70% 85.70%
=======================================
Files 129 129
Lines 6457 6457
=======================================
Hits 5534 5534
Misses 923 923 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
added 2 commits
June 12, 2026 03:56
…es in , then force-push to .
…es in , then force-push to .
chhwang
changed the title
Replace the internal node identifier with a generic hardware-class description (8×A100-80GB). Reproducibility docs describe hardware by class only; no environment-specific identifiers.
…ll-recommends to apt-get install lcov in .github/workflows/ut.yml to prevent OOM kill during dpkg unpack of fontconfig-config
Contributor
Author
|
Closing: the SGLang baseline/profiling harness is kept local for evaluation only and will not be merged into this repo. Only the ARK implementation lands here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add SGLang Qwen3-8B baseline benchmark harness
Add
examples/qwen3/bench/with scripts and docs for measuring SGLangQwen3-8B latency on 8×A100 at batch=1:
launch_sglang.sh: launch server with pinned A100-compatible image(CUDA 12.x, exact tag + digest recorded), TP=8 config.
measure_baseline.py: measure prefill TTFT (prompt 2048 tokens) anddecode per-token latency (generate 128 tokens); capture GPU clocks.
test_measure_baseline.py: unit tests for the measurement script.BASELINE.md: placeholder template for TP=1 and TP=8 results.reproduce.md: step-by-step reproduction instructions.Numbers are filled out-of-band on
mscclpp-a100-dev; the PR ships theharness and templates. Q2 (profiling) depends on these scripts.