Eclipse iceoryx
Enabling Virtually Limitless Data Transmissions at Constant Time
In domains such as automotive, robotics, and gaming, huge volumes of data must be transferred between different parts of the system. When operating systems, such as Linux, are used, the data must be transferred using Inter-Process Communication (IPC) mechanisms. Eclipse iceoryx is middleware that uses a zero-copy, shared memory approach to optimize inter-process communications.
The technology behind Eclipse iceoryx originated in the automotive domain. Over the last few decades, there’s been an evolution from engine control systems to driver assistance and finally, automated driving. At the same time, the data exchanged between different threads of execution within an Electronic Control Unit (ECU) increased from KB/s to GB/s (Figure 1).
Figure 1: Evolution of internal data exchange in ECUs
Similar to domains such as robotics and the Internet of Things (IoT), the common communications paradigm used in automotive is publish/subscribe. A typical middleware solution for Inter-Process Communication (IPC) copies the messages when passing them to or from the middleware.
Inside the middleware stack, even more copies may be made or the message payload may be serialized. As a result, you can easily end up with at least n+1 copies of messages if n consumers are subscribed to a publisher (Figure 2).
Figure 2: A copy perspective of a typical IPC middleware solution
When speeds reach GB/s, every message copy that’s created in the communications middleware has a significant cost in terms of runtime and latency. The goal should always be to use precious runtime for functional computations, not for shifting around bytes in memory.
True Zero-Copy, Shared Memory Data Transport
Eclipse iceoryx is an IPC technology based on shared memory. On its own, this is not a new innovation as the approach has been used since the 1970s. However, we take the approach further, combining it with a publish/subscribe architecture, service discovery, modern C++, and lock-free algorithms. By adding an application programming interface (API) that avoids copying, we can achieve what we refer to as true zero-copy — an end-to-end approach from publishers to subscribers without creating a single copy.
With the iceoryx API, a publisher writes the message directly to a chunk of memory that was previously requested from the middleware. When the message is delivered, subscribers receive references to these memory chunks while maintaining their own queue with a configurable capacity.
Every subscriber can have a unique view of which messages are still in process and which can be discarded. The iceoryx middleware counts the references behind the scenes and releases a memory chunk when it has no readers left (Figure 3).
Figure 3: True zero-copy communications
The iceoryx API supports polling access and event-driven interactions with callbacks. This enables a wide range of applications, including those involving real-time systems. The shared memory can be divided into segments with different access rights and configurable memory pools.
A Few Technical Details
One important aspect of iceoryx is that publishers can write again while subscribers are still reading because there is no interference from subscribers. The publisher is simply allocated a new memory chunk if the previous one is still in use.
If a subscriber is operating in polling mode and chunks are queued up until the subscriber checks the queue again, we can recycle older memory chunks using the lock-free queue in a process we call “safely overflowing.”
The lock-free queue allows us to guarantee a memory-efficient contract is made with the subscriber with respect to the maximum number of latest messages that are stored in the queue, no matter how long the time between successive subscriber polls. This is a very helpful approach in common use cases such as those with a high-frequency publisher and a subscriber that is only interested in the latest, greatest message.
Because it is simply passing around smart pointers, iceoryx enables data transfers without actually transferring the data. This approach enables a constant time for message transfer, no matter what the message size. The user does have to write the data once to shared memory, but this write is needed whenever data is produced for sending anyway.
Because the message payload is not serialized, a message must have the same memory layout for publishers and subscribers. For IPC on a specific processor, this can be ensured by using the same compiler with the same settings.
The message cannot contain any pointers to memory within the process’ internal virtual address space. This restriction also applies to heap-based data structures. If these constraints cannot be fulfilled, iceoryx can still be used with a top-level layer that handles the serialization in and the deserialization from the shared memory. In this case, iceoryx handles the low-layer transport that creates no copy itself.
Eclipse iceoryx depends on the POSIX API. We currently support Linux and QNX as underlying operating systems. Because there are sometimes slight API differences, small adaptions might be necessary when porting iceoryx to another POSIX-based operating system.
Integration With Existing Middleware Frameworks
Eclipse iceoryx is a data-agnostic shared memory transport mechanism that provides a fairly low-layer API. We assume the API is not accessed directly by users, but is integrated into a larger framework that provides a high-layer API and maybe some tooling. Examples include the AUTOSAR Adaptive platform and the Robotic Operating System (ROS).
In both cases, we ensured the specification supports the zero-copy API. Integration of iceoryx is quite straightforward if the target framework is also based on a publish/subscribe architecture. There are already publicly available integrations of iceoryx for ROS2 and eCAL.
In addition, we have already identified the potential for synergies within the Eclipse family. For example, combining Eclipse Cyclone DDS and iceoryx creates open and powerful communications middleware for IPC and network communications.
Get Started With iceoryx
You can find the code for Eclipse iceoryx here on GitHub.
About the Author
Senior Expert Middleware
Robert Bosch GmbH