Re: [cyclonedds-dev] Intra-process transport mechanism

[Alan Beadle <ab.beadle@xxxxxxxxx>]

Thank you Erik, this answers all of my questions!

-Alan

On Tue, Jun 6, 2023 at 10:19 AM Erik Boasson <eb@xxxxxxxxxxx> wrote:
>
> Hi Alan,
>
> Cyclone has a “local delivery” mechanism that is used to forward data to all local readers (except when Iceoryx is involved), as long as they are in the same process/domain and regardless of the DDS DomainParticipant. The implementation is a simple loop over an array of pointers to reader history caches, and updating each history cache in turn.
>
> When Iceoryx is involved, then the above is bypassed because it would interfere with Iceoryx (otherwise everything would get delivered twice).
>
> The network stack (so UDP, normally) gets involved when it is not in the same process (and not via Iceoryx).
>
> In Cyclone, this *always* happens, you can’t configure it not to do it this way. So I presume the “INTRA” transport mechanism refers to Connext and/or FastDDS, because I believe they used to perform communication between DomainParticipants in the same process via the network stack, and only later added an alternative transport for within the process. I don’t know all that much about them, but if I am correct, then in their approach data for which there are only readers in the same process will still traverse (most) of the RTPS stack, which is not the case for Cyclone.
>
> > Also, if a thread is sleeping in dds_waitset_wait, how is it woken by
> > the thread sending a message? It looks like it uses
> > pthread_cond_timedwait()? What about when UDP is being used?
>
> The reader history cache update causes the “data available” status to be raised and the read/query conditions to be re-evaluated. If waitsets are waiting for any of these, they get triggered by “pthread_cond_broadcast” (on POSIX platforms). Because the trigger is the reader history cache update, it doesn’t matter whether the data came from a local writer or the network.
>
> Since you mention the performance_test tool, it might be interesting to know that while I personally prefer waitsets and read conditions, the fastest way to get data out of Cyclone is to use a listener for the “data available” event. The listeners in Cyclone get called by the thread updating the reader history cache, so it saves a context switch and avoids possible contention. The downside is that you have to be careful what you do in a listener, just like in POSIX you have to be careful what you do inside a signal handler.
>
> As an aside, most of the community interactions happen on GitHub and some on Discord. You’re welcome to use the mailing list, but for questions where you think a wider audience might find it interesting, especially GitHub is probably the better choice.
>
> Best regards,
> Erik
>
> > On 6 Jun 2023, at 15:49, Alan Beadle <ab.beadle@xxxxxxxxx> wrote:
> >
> > Hi,
> >
> > I understand that CycloneDDS would normally use UDP as the transport
> > mechanism between processes. However, I was wondering what mechanism
> > is used for intra-process communication.
> >
> > I am using the performance_test tool [1] from Apex AI and would like
> > to understand the results in better detail. Their docs only state that
> > CycloneDDS uses an "INTRA" transport mechanism, which does not refer
> > to anything in the CycloneDDS docs. Apparently this is meant to
> > indicate that Cyclone will bypass the networking stack for
> > intra-process. Can anyone provide more detail?
> >
> > Also, if a thread is sleeping in dds_waitset_wait, how is it woken by
> > the thread sending a message? It looks like it uses
> > pthread_cond_timedwait()? What about when UDP is being used?
> >
> > Thanks in advance,
> > -Alan
> >
> > [1] https://gitlab.com/ApexAI/performance_test/-/tree/master/performance_test#eclipse-cyclone-dds
> > _______________________________________________
> > cyclonedds-dev mailing list
> > cyclonedds-dev@xxxxxxxxxxx
> > To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/cyclonedds-dev
>
> _______________________________________________
> cyclonedds-dev mailing list
> cyclonedds-dev@xxxxxxxxxxx
> To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/cyclonedds-dev