[dhcp_server] Wait for caclmgrd-owned docker0 syslog rule on start/stop#28580
[dhcp_server] Wait for caclmgrd-owned docker0 syslog rule on start/stop#28580Xichen96 wants to merge 1 commit into
Conversation
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Pull request overview
This PR updates the rendered dhcp_server container control script template to synchronize start/stop with the iptables rule that caclmgrd now owns (docker0 → host rsyslog RELP tcp/2514), using bounded waits instead of container-side rule add/delete. This helps prevent syslog connectivity drops during control-plane ACL rebuilds by ensuring the exception rule is present before start, and (on explicit feature disable) removed before stop returns.
Changes:
- Add a
preStartActionwait (up to 10s) for the docker0 tcp/2514 ACCEPT rule whenFEATURE|dhcp_serveris enabled. - Add a
stop()wait (up to 10s) for the rule to be removed whenFEATURE|dhcp_serveris disabled.
68faa04 to
a8046d7
Compare
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
a8046d7 to
b3ae12d
Compare
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
The dhcp_server docker0 syslog INPUT exception changed from UDP 514 to RELP over tcp/2514, and its ownership moved from the container start/stop script to caclmgrd, which re-installs it on every control-plane ACL flush-and-rebuild while FEATURE|dhcp_server is enabled (sonic-net/sonic-buildimage#28328, sonic-net/sonic-host-services#412, sonic-net/sonic-buildimage#28580; fixes sonic-net/sonic-buildimage#27584). Update generate_expected_rules() to assert the tcp/2514 exception instead of the old udp/514 rule, and gate the expectation on the FEATURE|dhcp_server state that caclmgrd keys on (read via sonic-db-cli), rather than on mere container presence. Signed-off-by: Xichen Lin <lukelin0907@gmail.com>
The dhcp_server docker0 syslog INPUT exception changed from UDP 514 to RELP over tcp/2514, and its ownership moved from the container start/stop script to caclmgrd, which re-installs it on every control-plane ACL flush-and-rebuild while FEATURE|dhcp_server is enabled (sonic-net/sonic-buildimage#28328, sonic-net/sonic-host-services#412, sonic-net/sonic-buildimage#28580; fixes sonic-net/sonic-buildimage#27584). Update generate_expected_rules() to assert the tcp/2514 exception instead of the old udp/514 rule, and gate the expectation on the FEATURE|dhcp_server state that caclmgrd keys on (read via sonic-db-cli), rather than on mere container presence. Signed-off-by: Xichen Lin <lukelin0907@gmail.com>
Add test_caclmgrd_dhcp_server_syslog verifying the docker0 RELP tcp/2514 INPUT exception that caclmgrd now owns (sonic-net/sonic-buildimage#28328, sonic-net/sonic-host-services#412, sonic-net/sonic-buildimage#28580; fixes sonic-net/sonic-buildimage#27584): * present above the control-plane catch-all DROP while FEATURE|dhcp_server is enabled, * survives a control-plane ACL flush-and-rebuild (forced via the dummy_acl_rules fixture / acl-loader), and * removed on feature-disable and restored on feature-enable. The test skips when dhcp_server is not enabled on the DUT. Signed-off-by: Xichen Lin <lukelin0907@gmail.com>
b3ae12d to
a10a5ac
Compare
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Add test_caclmgrd_dhcp_server_syslog verifying the docker0 RELP tcp/2514 INPUT exception that caclmgrd now owns (sonic-net/sonic-buildimage#28328, sonic-net/sonic-host-services#412, sonic-net/sonic-buildimage#28580; fixes sonic-net/sonic-buildimage#27584): * present above the control-plane catch-all DROP while FEATURE|dhcp_server is enabled, * survives a control-plane ACL flush-and-rebuild (forced via the dummy_acl_rules fixture / acl-loader), and * removed on feature-disable and restored on feature-enable. The test skips when dhcp_server is not enabled on the DUT. Signed-off-by: Xichen Lin <lukelin0907@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
files/build_templates/docker_image_ctl.j2:930
- Same bounding issue in
stop(): the loop intends to wait ~10s, butiptables -w 5 -Ccan block up to 5s per probe under xtables lock contention, so this path may take far longer than 10s (and the warning text becomes misleading). Tightening-wand retry count keeps the stop path’s worst-case delay aligned with the intended bound.
if [ "$($SONIC_DB_CLI CONFIG_DB HGET 'FEATURE|dhcp_server' state 2>/dev/null)" = "disabled" ]; then
for i in $(seq 1 10); do
iptables -w 5 -C INPUT -i docker0 -p tcp -m tcp --dport 2514 -m comment --comment "dhcp_server_syslog" -j ACCEPT 2>/dev/null || break
sleep 1
done
iptables -w 5 -C INPUT -i docker0 -p tcp -m tcp --dport 2514 -m comment --comment "dhcp_server_syslog" -j ACCEPT 2>/dev/null && \
echo "Warning: caclmgrd-owned dhcp_server docker0 syslog ACCEPT rule still present after 10s despite dhcp_server feature disabled."
fi
Step 3 of 3 of moving the dhcp_server docker0 syslog INPUT exception ownership to caclmgrd (fixes #27584). PR1 (#28328) removed the container-side add/del; PR2 (sonic-net/sonic-host-services#412) made caclmgrd own the rule. This replaces the removed add/del with feature-gated bounded waits that synchronize with caclmgrd: - preStartAction: when FEATURE|dhcp_server is enabled, wait up to 10s for the docker0 tcp/2514 ACCEPT rule to be present before the container starts, so the RELP client's first connection is not dropped by the control-plane catch-all DROP. - stop(): when FEATURE|dhcp_server is disabled, wait up to 10s for the rule to be removed. Plain restarts / shutdown / CONFIG_DB-down skip the wait (positive-state gate). The iptables -C existence check matches the canonical stored form of the rule caclmgrd installs (identical to `iptables -S INPUT` output), so the wait observes the rule reliably across iptables variants. Signed-off-by: Xichen96 <lukelin0907@gmail.com>
a10a5ac to
78384be
Compare
Add test_caclmgrd_dhcp_server_syslog verifying the docker0 RELP tcp/2514 INPUT exception that caclmgrd now owns (sonic-net/sonic-buildimage#28328, sonic-net/sonic-host-services#412, sonic-net/sonic-buildimage#28580; fixes sonic-net/sonic-buildimage#27584): * present above the control-plane catch-all DROP while FEATURE|dhcp_server is enabled, * survives a control-plane ACL flush-and-rebuild (forced via the dummy_acl_rules fixture / acl-loader), and * removed on feature-disable and restored on feature-enable. The test skips when dhcp_server is not enabled on the DUT. Signed-off-by: Xichen Lin <lukelin0907@gmail.com>
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Why I did it
dhcp_serveris the only container that forwards its rsyslog to the host overdocker0(other containers, including bridge-moderedfish, log to127.0.0.1); that forwarded traffic uses RELP (tcp/2514).caclmgrdflushes and rebuilds theINPUTchain on every control-plane ACL change and appends a catch-allDROP. Ownership of thedocker0/2514ACCEPTexception has moved from the container start/stop script tocaclmgrd, which re-installs it on every rebuild keyed onFEATURE|dhcp_server(fixes #27584).This is step 3 of 3:
caclmgrdown the rule.caclmgrd.Work item tracking
How I did it
Re-add the
dhcp_serverbranches infiles/build_templates/docker_image_ctl.j2at the same spots #28328 reverted, but as bounded waits instead of add/del:preStartAction: whenFEATURE|dhcp_serverisenabled, wait up to 10s for thedocker0tcp/2514ACCEPTrule to be present before the container starts, so the RELP client's first (stateNEW) connection is not dropped by the control-plane catch-allDROP.stop(): whenFEATURE|dhcp_serverisdisabled, wait up to 10s for the rule to be removed. Plain container restarts, shutdown, andCONFIG_DB-unreachable all skip the wait (positive-state gate, so a restart with the feature still enabled does not block).The
iptables -Cexistence check matches the canonical stored form of the rulecaclmgrdinstalls (identical toiptables -S INPUToutput:-i docker0 -p tcp -m tcp --dport 2514 -m comment --comment dhcp_server_syslog -j ACCEPT), so the wait observes the rule reliably across iptables variants.How to verify it
Hardware-validated as part of the 3-PR set by hand-patching the rendered
dhcp_serverctl script (these waits) + thecaclmgrdchange onto a running DUT (no image rebuild).bjw-can-720dt-2(720dt,mx),dhcp_serverenabled; RELP tcp/2514 to hostrsyslogdon240.127.1.1:2514(thedocker0IP); a control-plane ACL rule present socaclmgrdprograms its catch-all-A INPUT -j DROP.caclmgrdhas the docker0/2514ACCEPTrule installed (INPUT pos 3, above the catch-all DROP); end-to-enddocker exec dhcp_server logger ...reached host/var/log/syslog.config feature state dhcp_server disabled->caclmgrdremoves the rule and the container reachesExited(0); re-enabledre-adds the rule and the container starts.FEATURE|dhcp_serverbeingdisabled, so a plain container restart with the feature stillenabledskips it.Which release branch to backport (provide reason below if selected)
Coordinated with the caclmgrd change (step 2); the buildimage backport is handled together with #28328 as a set so the branch is never left with neither party owning the rule.
Tracking issue/work item for backport/cherry-pick request (GitHub issue or Microsoft ADO): Microsoft ADO 38745012
Failure type: day-one issue (the container-owned docker0 syslog rule never survived a caclmgrd control-plane ACL rebuild)
Tested branch
Test result
bjw-can-720dt-2(720dt,mx); see "How to verify it" above. The docker0 tcp/2514ACCEPTrule is programmed above the catch-allDROP, end-to-end RELP syslog is delivered, the feature toggle adds/removes the rule cleanly, and the rule survives an unrelated control-plane ACL re-walk (bug Bug: CACL updates remove dhcp_server syslog iptables rule #27584 fix).dhcp_serverctl script andcaclmgrdcode path are identical to 202605; a master-image run is the only remaining validation step.Description for the changelog
[dhcp_server] Replace the container-side docker0 syslog iptables add/del with feature-gated bounded waits on the caclmgrd-owned tcp/2514 rule.