Skip to content

Multi-worker (dp_size>1) reserved_socket guard — hardware validation deferred #62

Description

@KHAEntertainment

Background

PR #52 (closes Issue #17 item 2) added a dp_size guard around the reserved_socket pre-binding in python/sglang/srt/entrypoints/http_server.py (around lines 2416-2479). When dp_size == 1, the server pre-binds reserved_socket and passes reserved_socket.fileno() to uvicorn. When dp_size > 1, the pre-bind is skipped, an info log is emitted, and fd=None is passed.

Validation status at merge time

  • dp_size == 1 path: empirically validated on H100 during cascade smoke testing. Server starts cleanly, reaches HTTP 200 health within 60s, no "Skipping reserved" log line confirming pre-bind path was taken.
  • dp_size > 1 path: static code review only. Multi-GPU hardware to exercise the dp_size > 1 runtime path was not available to the team at PR fix(http_server): guard reserved_socket pre-bind behind dp_size == 1 (KHA-305) #52 merge time. --gpu-id-step 0 to share a single GPU across workers was rejected by check_server_args (must be positive).

Why it's safe to merge as-is

  1. The diff is small and isolated to one file. The dp_size > 1 branch is essentially "do less, pass None" with null-tolerant uvicorn call sites on both sides.
  2. The Engram fork only runs single-worker today (per PR fix(http_server): guard reserved_socket pre-bind behind dp_size == 1 (KHA-305) #52 body). The dp_size > 1 path is defensive code for a future configuration, not exercised in production.
  3. Any latent bug at dp_size > 1 would surface as a startup failure on configuration flip — there is no silent-corruption risk path.

Follow-up

When multi-GPU hardware access opens up (likely concurrent with the cluster work needed for larger model classes), validate the dp_size > 1 runtime path on hardware:

  • Server starts cleanly at dp_size = 2 (and higher).
  • The pre-bind is skipped (info log shows it).
  • uvicorn workers come up healthy without the pre-bound socket.
  • Health endpoint returns 200 in the multi-worker configuration.

This issue tracks the deferred validation publicly so the gap is visible in PR #52's history.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions