Skip to content

Fix SCTPListener.Close race with concurrent Accept - #89

Open
igor-lazic wants to merge 1 commit into
ishidawataru:masterfrom
igor-lazic:fix-listener-close-race
Open

Fix SCTPListener.Close race with concurrent Accept#89
igor-lazic wants to merge 1 commit into
ishidawataru:masterfrom
igor-lazic:fix-listener-close-race

Conversation

@igor-lazic

Copy link
Copy Markdown

Summary

SCTPListener.Close currently calls syscall.Shutdown and syscall.Close on ln.fd without any synchronisation. Concurrent AcceptSCTP and Addr calls read ln.fd directly. This produces three observable bugs that are easy to hit in any program that calls Close while a Serve loop is blocked in Accept:

  1. A goroutine blocked in accept4() returns EINVAL or EBADF when Close runs concurrently, instead of a sentinel callers can detect (cf. net.TCPListener, which has returned net.ErrClosed since Go 1.16). Real-world failure: downstream callers like go-diameter log every graceful shutdown as "accept error: invalid argument" or "accept error: bad file descriptor".
  2. AcceptSCTP after Close calls accept4 on the now-dead fd integer. If the OS has reassigned that fd between Close and Accept, accept4 either fails or, worse, returns an accepted fd from an unrelated socket.
  3. AcceptSCTP always returns a non-nil *SCTPConn (wrapping fd=-1 on the error path), violating the (net.Conn, error) API contract. Callers that defensively invoke methods on the returned conn hit another EBADF from setsockopt.

The fix

The change mirrors the pattern this package already uses for SCTPConn — same field shape, same atomic primitive, same close protocol:

Element SCTPConn (existing) SCTPListener (this PR)
Field _fd int32 (sctp.go) _fd int32 (sctp.go)
Accessor c.fd() via atomic.LoadInt32(&c._fd) ln.fd() via atomic.LoadInt32(&ln._fd)
Close atomic.SwapInt32(&c._fd, -1) (sctp_linux.go:157) atomic.SwapInt32(&ln._fd, -1)

Specifically:

  • Rename SCTPListener.fd_fd int32, accessed via atomic.LoadInt32 / atomic.SwapInt32.
  • Close uses atomic.SwapInt32(-1) so only one caller observes the real fd; subsequent calls return syscall.EBADF without re-issuing syscall.Close on a possibly-reused fd number.
  • AcceptSCTP and Addr take an atomic snapshot of _fd and bail to ErrListenerClosed / nil if the listener has been closed. When accept4 was already blocked inside the kernel when Close ran, the post-call atomic re-check normalises the resulting EBADF / EINVAL to ErrListenerClosed.
  • AcceptSCTP now returns (nil, err) on every error path.

ErrListenerClosed is a new exported sentinel, defined with errors.New to stay compatible with the existing go 1.12 directive (no net.ErrClosed / errors.Is usage). Callers compare with ==.

Test plan

  • New sctp_listener_close_test.go: Close-during-Accept, Accept-after-Close, double-Close, 16-way concurrent Close, Addr-after-Close
  • Verified RED on master: the first two tests fail with exactly invalid argument (EINVAL) and bad file descriptor (EBADF), with a non-nil *SCTPConn{_fd:-1} returned alongside the error — i.e., the test reproduces the downstream go-diameter failure mode byte-for-byte
  • All five tests pass with this patch
  • Existing test suite remains green under go test -race -count=1

What this PR does not touch

  • sctp_unsupported.go (the non-Linux stub still returns ErrUnsupported)
  • go.mod (still go 1.12)
  • SCTPConn.Close's pre-existing if fd > 0 check (mildly off — should be >= 0 since stdin is a valid fd — but unrelated to this race fix, happy to send as a separate PR)

SCTPListener.Close currently calls syscall.Shutdown and syscall.Close on
ln.fd without any lock or atomic guard. Concurrent AcceptSCTP and Addr
calls read ln.fd directly. This produces three observable bugs:

1. A goroutine blocked inside accept4() returns a bare syscall.EINVAL or
   syscall.EBADF when Close runs concurrently, instead of a sentinel
   that callers can detect (cf. net.TCPListener which returns
   net.ErrClosed since Go 1.16).

2. AcceptSCTP after Close calls accept4 on the now-dead fd integer. If
   the OS has reassigned that fd to an unrelated resource between Close
   and Accept, accept4 either fails or, worse, returns a fresh
   accepted-fd from someone else's socket.

3. AcceptSCTP always returns a non-nil *SCTPConn (wrapping fd=-1 on the
   error path), violating the (net.Conn, error) API contract. Callers
   that defensively call methods on the returned conn end up with
   another EBADF from setsockopt etc.

This change mirrors the pattern already used by SCTPConn.Close:

  - Rename SCTPListener.fd to _fd int32, accessed via atomic load/swap.
  - Close uses atomic.SwapInt32 so only one caller observes a real fd;
    subsequent Close returns syscall.EBADF without re-issuing
    syscall.Close on a possibly-reused fd number.
  - AcceptSCTP and Addr take an atomic snapshot of _fd and bail to
    ErrListenerClosed / nil when the listener has been closed. If
    accept4 was already blocked inside the kernel when Close ran, the
    post-call atomic re-check normalises the resulting EBADF/EINVAL to
    ErrListenerClosed so callers can detect graceful shutdown with a
    plain == comparison instead of inspecting raw errnos.
  - AcceptSCTP now returns (nil, err) on error, matching the contract
    of every other net.Listener.

ErrListenerClosed is a new exported sentinel (defined with errors.New
to stay compatible with the go 1.12 directive — errors.Is and
net.ErrClosed are Go 1.13+ / 1.16+).

Tests: sctp_listener_close_test.go covers Close-during-Accept,
Accept-after-Close, double-Close, concurrent-Close, and Addr-after-Close.
The first two tests fail on master with the exact errnos this patch
fixes (invalid argument / bad file descriptor); all five pass with
this change. Existing tests remain green under -race.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant