feat(rtcconfig): add advertise_private_ips for hybrid topologies#84
Closed
lccool6683 wants to merge 1 commit into
Closed
feat(rtcconfig): add advertise_private_ips for hybrid topologies#84lccool6683 wants to merge 1 commit into
lccool6683 wants to merge 1 commit into
Conversation
|
|
lccool6683
force-pushed
the
feat/advertise-private-ips
branch
from
May 2, 2026 18:08
6d678de to
2955c2b
Compare
boks1971
approved these changes
May 2, 2026
boks1971
left a comment
boks1971
left a comment
Contributor
There was a problem hiding this comment.
lgtm! Thank you for doing this!
lccool6683
force-pushed
the
feat/advertise-private-ips
branch
from
May 3, 2026 08:38
2955c2b to
e30e64a
Compare
Introduce a new opt-in RTCConfig field AdvertisePrivateIPs (default false). When set together with use_external_ip, NewWebRTCConfig publishes both the private host candidate (the local IP) and the STUN-derived public IP as a server-reflexive candidate, instead of replacing the host candidate with the public IP. This unblocks hybrid Kubernetes topologies where some peers reach LiveKit via cluster-internal routing on the private IP while others reach it via the public IP. The config has no effect when use_external_ip is false or external_ip_only is true. Implementation - chooseNAT1To1Mode picks (Replace+Host) by default and switches to (Append+Srflx) only when AdvertisePrivateIPs is true and STUN actually returned external mappings. - SetNAT1To1AddressRewriteRulesAppendSrflx wires the new mode through to pion's webrtc.SetICEAddressRewriteRules in append mode. - buildNAT1To1Rules is the shared rule builder. The catch-all rule (one entry per external IP, no Local) is emitted regardless of mode: pion's gatherCandidatesSrflxMapped opens its UDP socket on 0.0.0.0, and the rule lookup keyed on that wildcard address must hit the catch-all to publish the correct external IP. Without it, srflx candidates would fall back to advertising 0.0.0.0 as the base address. - Self-mappings (External == Local) are skipped in append mode because publishing "<private> -> <private>" as srflx is meaningless and only inflates ICE check overhead. Validation guards RTCConfig.Validate() rejects two configuration combinations that would silently break the patch path. validateAdvertisePrivateIPs runs before the port-defaulting block so operator typos (e.g. setting only port_range_end) are caught with their original values rather than silently masked by the production defaults. - ForceTCP: rejected because pion's gatherCandidatesSrflxMapped skips TCP network types, so the public IP would never be advertised in TCP-only mode. - Anything other than full port-range mode (single-port udp_port, partial port range with only one bound set, or unbounded UDP gather): rejected because pion's srflx mapped gather opens its own UDP listener via listenUDPInPortRange and falls back to an OS-ephemeral port when ICEPortRangeStart/End are not both non-zero. That port isn't covered by single-port firewall rules, so the published srflx candidate would be unreachable from external clients. Operator note (also captured on the field comment): external clients on networks that block UDP but allow outbound TCP/7881 lose the public TCP host candidate they had under the legacy (Replace+Host) mode. Such clients require TCP TURN at 443 to reach the SFU; ensure TURN/TLS is provisioned before opting in. Tests cover both legacy replace+host and the new append+srflx shape (catch-all and self-mapping behavior) plus all combinations of the new validation guards (force_tcp, single-port udp_port, partial port range in either direction, no port config at all, and the valid full port-range case). Closes #4487
lccool6683
force-pushed
the
feat/advertise-private-ips
branch
from
May 5, 2026 16:36
e30e64a to
6ed90b3
Compare
Contributor
|
@lccool6683 can this be closed as well given #88 ? |
Author
|
yes! thnak you |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds opt-in
RTCConfig.AdvertisePrivateIPs(yaml:advertise_private_ips, defaultfalse). When enabled together withuse_external_ip,NewWebRTCConfigpublishes both the private host candidate and the STUN-derived public IP as a server-reflexive candidate, instead of replacing the host candidate with the public IP.Implements the design discussed in livekit/livekit#4487.
Why
Unblocks hybrid Kubernetes topologies where some peers reach LiveKit via cluster-internal routing on the private IP while others reach it via the public IP. Without this, the existing
use_external_ippath replaces the private host candidate with the public IP, leaving in-cluster peers unable to take a direct host path.How
chooseNAT1To1Mode(advertisePrivateIPs, externalIPOnly, externalMappingsCount)picks(Replace, Host)by default and(Append, Srflx)only whenadvertise_private_ips=true,external_ip_only=false, and STUN actually returned mappings.SetNAT1To1AddressRewriteRulesAppendSrflxwires the new mode into pion'swebrtc.SetICEAddressRewriteRulesin append mode.buildNAT1To1Rulesis the shared rule builder. The catch-all rule (one entry per external IP, noLocal) is emitted regardless of mode — pion'sgatherCandidatesSrflxMappedopens its UDP socket on0.0.0.0, and rule lookup keyed on the wildcard local IP must hit the catch-all to publish the right external IP. Without it, srflx candidates would fall back to advertising0.0.0.0as base address.External == Local) are skipped in append mode because publishing<private> → <private>as srflx is meaningless and only inflates ICE check overhead.Default behavior
Default
false. Whenadvertise_private_ips=false, the rewrite path is unchanged from master (legacyReplace+Hostsemantics preserved throughSetNAT1To1AddressRewriteRules→defaultNAT1To1Mode).Tests
go test -race ./pkg/rtcconfig/green. Added coverage:TestBuildNAT1To1Rules— replace+host shape, append+srflx shape, append+srflx with self-mapping skip, append+srflx with unmapped catch-all.TestChooseNAT1To1Mode— all four combinations ofadvertise_private_ips × external_ip_only × len(ips) > 0.Related
feat/skip-external-ip-validation— independent, but touches overlapping file regions inpkg/rtcconfig/config.goandpkg/rtcconfig/webrtc_config_test.go. Resolves trivially; happy to rebase whichever lands second.livekitbuildable from a clean checkout at every commit.