Compare commits

...

1 commit

Author SHA1 Message Date
HuJK
082e869d0e sockets: survive a descriptor numbered at or above FD_SETSIZE
fd_set cannot represent a descriptor at or above FD_SETSIZE, which is 1024 and
fixed: fds_bits is a fixed-size array. On bionic FD_SET/FD_CLR/FD_ISSET are
FORTIFY-checked, so exceeding it is not a quietly corrupted bitmap but an
immediate abort:

  FORTIFY: FD_SET: file descriptor 1073 >= FD_SETSIZE 1024
  Fatal signal 6 (SIGABRT) in tid 18004 (v_gpu), pid 15615

Observed on a running system. This library gets linked into a VMM whose GPU
device holds several dma-buf descriptors per guest buffer object, so a few
hundred live objects push the fd table past 1023 and an accepted VNC socket
lands above it. The abort takes the VMM down, and on that platform a dying VMM
resets the device.

What the limit is on is worth stating: the descriptor NUMBER, not how many are
open. Raising RLIMIT_NOFILE does not help and neither does closing other
descriptors, because accept() has already returned 1073.

Two kinds of site, treated differently. Real I/O waits move to poll(), which has
no such ceiling -- clientInput and listenerRun in main.c, rfbReadExactTimeout and
rfbWriteExact in sockets.c. rfbWriteExact matters most in practice: it is reached
on every back-pressured write. The remaining select() sites, which multiplex over
the whole client set, keep select() but skip descriptors the set cannot hold
rather than aborting on them, so one unrepresentable client costs that client and
not the process.
2026-08-30 17:21:09 +08:00
3 changed files with 167 additions and 60 deletions