Eclipse Community Forums: Eclipse Titan » use of select() and file-descriptor limit

Home » Eclipse Projects » Eclipse Titan » use of select() and file-descriptor limit

use of select() and file-descriptor limit [message #1837587]

Tue, 02 February 2021 15:08

Eclipse User

At least in some situations, TITAN is using the venerable select() system call for handling non-blocking I/O to the various file descriptors. select() is well-understood, but also has several quite strong disadvantages:

* it only supports 1024 file descriptors; specifically, it only supports file descriptor numbers less than 1024. So even if you have less than 1024 file descriptors open, as soon as one of them is > 1024 in numeric value, it won't work with select anymore
* it is extremely slow on the system call, as thre is a lot of clearing and scanning of that 1024-bit-array happening all the time

I've been seeing the warnings like this ever since I started to use TITAN for the first time some years ago:

Warning: The maximum number of open file descriptors (500000) is greater than FD_SETSIZE (1024). Ensure that Test Ports using Install_Handler do not try to wait for events of file descriptors with values greater than FD_SETSIZE (1024). (Current caller of Install_Handler is "SCTP_PORT")

On occasion, particularly in large setups with lots of components and ports, I believe I've been running into that limit.

Now I did a bit of grep'ing in the source, and it seems EPOLL is supported. Maybe not in all parts of the code For example core/Port.cc doesn't seem to cotnain EPOLL support, but mctr2 does? Maybe it's simply not enabled in the Debian builds?

Does anyone have more information about this? Thanks!

Also, is there some documentation on which parts of TITAN suffer from this limitation, and how/where does that manifest itself?

Re: use of select() and file-descriptor limit [message #1837610 is a reply to message #1837587]

Wed, 03 February 2021 02:40

Eclipse User

The test ports don't use select/poll/epoll directly. The socket notifications are handled by the TITAN core library and the test ports registers the file descriptors
and its message handler functions are called by the TITAN core.

There are two sets of the event handler functions in the test port API:
- the old one which uses FD_SET.
- the new one which uses the fd directly.

See chapter 2.4.6 of the API guide.

The SCTP test port is an old test port using the old API. I'll update it to the new API.

Re: use of select() and file-descriptor limit [message #1837611 is a reply to message #1837610]

Wed, 03 February 2021 02:56

Eclipse User

The updated code is available in the repository. It includes an IPv6 compatibility update I just finished yesterday.

Re: use of select() and file-descriptor limit [message #1837624 is a reply to message #1837611]

Wed, 03 February 2021 06:19

Eclipse User

Thanks, I've also pushed a suggested patch to the API guide explaining why the old API shall not be used anymore: https://review.gerrithub.io/c/laf0rge/titan.core/+/510139

Re: use of select() and file-descriptor limit [message #1837625 is a reply to message #1837624]

Wed, 03 February 2021 06:38

Eclipse User

I can also see the same "problem" (still using Event_Handler) in (at least) the following test ports:
* UDPasp (RTPasp and DNS still depends on it, it is not using IPL4asp)
* PCAPasp
* SIPmsg

Let me know if you intend to update them, or if we should try that ourselves and submit patches.

Re: use of select() and file-descriptor limit [message #1837715 is a reply to message #1837625]

Thu, 04 February 2021 09:28

Eclipse User

I'm checking them.

But the networking part of the SIP msg is deprecated. It should be a protocol module.

Previous Topic:	Log FileMask / ConsoleMask filter by component type?
Next Topic:	Template evaluation time

Goto Forum:

-=] Back to Top [=-

Current Time: Wed Jul 23 06:20:50 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter