Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Eclipse Titan » use of select() and file-descriptor limit
use of select() and file-descriptor limit [message #1837587] Tue, 02 February 2021 20:08 Go to next message
Harald Welte is currently offline Harald WelteFriend
Messages: 128
Registered: July 2017
Location: Berlin, Germany
Senior Member

At least in some situations, TITAN is using the venerable select() system call for handling non-blocking I/O to the various file descriptors. select() is well-understood, but also has several quite strong disadvantages:

* it only supports 1024 file descriptors; specifically, it only supports file descriptor numbers less than 1024. So even if you have less than 1024 file descriptors open, as soon as one of them is > 1024 in numeric value, it won't work with select anymore
* it is extremely slow on the system call, as thre is a lot of clearing and scanning of that 1024-bit-array happening all the time

I've been seeing the warnings like this ever since I started to use TITAN for the first time some years ago:
Warning: The maximum number of open file descriptors (500000) is greater than FD_SETSIZE (1024). Ensure that Test Ports using Install_Handler do not try to wait for events of file descriptors with values greater than FD_SETSIZE (1024). (Current caller of Install_Handler is "SCTP_PORT")


On occasion, particularly in large setups with lots of components and ports, I believe I've been running into that limit.

Now I did a bit of grep'ing in the source, and it seems EPOLL is supported. Maybe not in all parts of the code For example core/Port.cc doesn't seem to cotnain EPOLL support, but mctr2 does? Maybe it's simply not enabled in the Debian builds?

Does anyone have more information about this? Thanks!

Also, is there some documentation on which parts of TITAN suffer from this limitation, and how/where does that manifest itself?
Re: use of select() and file-descriptor limit [message #1837610 is a reply to message #1837587] Wed, 03 February 2021 07:40 Go to previous messageGo to next message
Gábor Szalai is currently offline Gábor SzalaiFriend
Messages: 86
Registered: December 2015
Member
The test ports don't use select/poll/epoll directly. The socket notifications are handled by the TITAN core library and the test ports registers the file descriptors
and its message handler functions are called by the TITAN core.

There are two sets of the event handler functions in the test port API:
- the old one which uses FD_SET.
- the new one which uses the fd directly.

See chapter 2.4.6 of the API guide.

The SCTP test port is an old test port using the old API. I'll update it to the new API.
Re: use of select() and file-descriptor limit [message #1837611 is a reply to message #1837610] Wed, 03 February 2021 07:56 Go to previous messageGo to next message
Gábor Szalai is currently offline Gábor SzalaiFriend
Messages: 86
Registered: December 2015
Member
The updated code is available in the repository. It includes an IPv6 compatibility update I just finished yesterday.
Re: use of select() and file-descriptor limit [message #1837624 is a reply to message #1837611] Wed, 03 February 2021 11:19 Go to previous messageGo to next message
Harald Welte is currently offline Harald WelteFriend
Messages: 128
Registered: July 2017
Location: Berlin, Germany
Senior Member

Thanks, I've also pushed a suggested patch to the API guide explaining why the old API shall not be used anymore: https://review.gerrithub.io/c/laf0rge/titan.core/+/510139
Re: use of select() and file-descriptor limit [message #1837625 is a reply to message #1837624] Wed, 03 February 2021 11:38 Go to previous messageGo to next message
Harald Welte is currently offline Harald WelteFriend
Messages: 128
Registered: July 2017
Location: Berlin, Germany
Senior Member

I can also see the same "problem" (still using Event_Handler) in (at least) the following test ports:
* UDPasp (RTPasp and DNS still depends on it, it is not using IPL4asp)
* PCAPasp
* SIPmsg

Let me know if you intend to update them, or if we should try that ourselves and submit patches.
Re: use of select() and file-descriptor limit [message #1837715 is a reply to message #1837625] Thu, 04 February 2021 14:28 Go to previous message
Gábor Szalai is currently offline Gábor SzalaiFriend
Messages: 86
Registered: December 2015
Member
I'm checking them.

But the networking part of the SIP msg is deprecated. It should be a protocol module.
Previous Topic:Log FileMask / ConsoleMask filter by component type?
Next Topic:Template evaluation time
Goto Forum:
  


Current Time: Sun Jun 13 15:33:39 GMT 2021

Powered by FUDForum. Page generated in 0.02057 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top