Re: [tracecompass-dev] Event requests precedence

[Geneviève Bastien <gbastien+lttng@xxxxxxxxxxxx>]

Hi Bernd,

Yes I do remember taking a look at those patches back then and not being
100% enthousiastic about it. Also, back then, the use case was more
hypothetical. Now that I have a concrete use case to use it on, I'm
ready to re-review it. And now that we're with Trace Compass and have
less API constraints, maybe see if the proposal can be improved. I'd
publish the original patches for official review later today, unless you
want to do it Bernd?

To see the coalescing in action, here is my development branch [1],
branch vm_analysis_temp. You can execute the unit tests in
o.e.t.lttng2.analysis.vm.core.tests, the VirtualMachineAnalysisTest
class. By tracing the requests in Eclipse, you can see that in the end,
there is only 1 request that services events from the 2 traces of the
experiment.

[1]
http://git.dorsal.polymtl.ca/~gbastien?p=org.eclipse.tracecompass.git;a=summary

Cheers,
Geneviève

On 14-11-05 01:57 PM, Bernd Hufmann wrote:
> Hi Genevieve
>
> As far as I remember there were no other issues than "no" performance
> improvements. At that time, none of the commiters  (now Trace Compass
> committers including me) had a strong opinion for merging them. After
> asking all of the committers in an email (including you) and no
> responses I abandoned these patches.
>
> From my point of view, I don't plan any re-design around how requests
> are handled and how they are coalesced. So, I would be ok that these
> patches are brought back and eventually merged. Now, it's a good time
> for these changes since we have the freedom of adjusting the API for
> that. Having said that, some others in the team might think
> differently because they like to change things up.
>
> /Bernd
>
> On 11/05/2014 11:45 AM, Geneviève Bastien wrote:
>> Hi Bernd,
>>
>> I tried your patch series and it actually solves my use case, without
>> modifications (at least for the unit tests that always failed
>> otherwise). There probably still needs to add some
>> priority/precedence of request or something to have 100% certainty,
>> but it's good enough for now!
>>
>> Why were those patches not merged? Was there any other issue of
>> discussion beyond no performance improvement? I wouldn't want to work
>> over those patches if they are going to be redesigned anyway. I
>> pushed them as draft on gerrit, for Trace Compass. The first is here
>> [1]. You can follow the chain from there. I think as the author you
>> should have access to them directly?
>>
>> [1] https://git.eclipse.org/r/#/c/35965
>>
>> Cheers,
>> Geneviève
>>
>> On 11/03/2014 07:27 AM, Bernd Hufmann wrote:
>>> Hi Genevieve
>>>
>>> Right now the coalescing is done per event provider
>>> (ITmfEventProvider) which is either the trace (TmfTrace) or the
>>> experiment (TmfExperiment). Coalescing of requests across event
>>> providers is not supported right now. Actually, it was supported a
>>> long time ago where each request was sent by the experiment as event
>>> provider regardless of the request came from the trace or the
>>> experiment. This was changed to allow analyses to be scheduled
>>> independently. This type of coalescing lead to that a trace file was
>>> only read once (during indexing and building of analyses results)
>>> and each event was simply dispatched to the interested traces and
>>> experiment.
>>>
>>> A few month ago, we discussed to implement coalescing of request
>>> across event providers to bring back the original behaviour but
>>> still supporting the new use cases of analyses. At that time, we
>>> were hoping to get some performance improvement when analysing
>>> experiments by avoiding concurrent reading of the traces files. I
>>> implemented a series of patch sets for that support. Even if
>>> functionally, the changes were working and the coalescing goal was
>>> achieved, it didn't show much performance improvement (nor
>>> degradation) and we decided to not merge it at the time.
>>>
>>> So what this series of patches does, is to introduce a parent-child
>>> relation between event providers. An experiment event provider, is
>>> the parent of a trace event provider. Experiments could have other
>>> experiments as children. When sending a request using a trace event
>>> provider, the trace event provider was aware of its parent and would
>>> dispatch the request to the parent. If the parent was able to
>>> coalesce the request it did so, otherwise it would be sent by the
>>> trace.
>>>
>>> For your use case, this series of patches would help. It would make
>>> sure that each event is processed in a coalesced request from the
>>> experiment and its traces. However, the order of how events are
>>> processed is not guaranteed and there is no means right now to order
>>> them. This would have to be implemented on top of the patches.
>>> Having priorities, like Alex suggested, could be one way to go.
>>>
>>> Here is the reference of the series of patches:
>>> 1) https://git.eclipse.org/r/#/c/23631/
>>> 2) https://git.eclipse.org/r/#/c/23632/
>>> 3) https://git.eclipse.org/r/#/c/23633/
>>> 4) https://git.eclipse.org/r/#/c/23634/
>>>
>>> Please have look and see if this will be useful with your use case.
>>>
>>> Best Regards
>>> Bernd
>>>
>>>
>>>
>>> On 10/31/2014 03:06 PM, Geneviève Bastien wrote:
>>>> Hello,
>>>>
>>>> I would like some information on how the event request coalescing
>>>> works as of now and how I could make the following use case work:
>>>>
>>>> * I have an analysis for an experiment (let's call it the virtual
>>>> machine analysis) that runs an event request on the experiment.
>>>> * That analysis also depends on individual analysis from the traces
>>>> of the experiment (concretely the LttngKernelAnalysisModule from
>>>> each kernel trace).
>>>> * When the experiment analysis module is executed, it also makes
>>>> sure the LttngKernelAnalysisModule from each child trace is executed.
>>>> * But when I handle an event in the experiment, I want to be sure
>>>> that tihe LttngKernelAnalysisModules have run with this event
>>>> before handling it in the experiment.
>>>>
>>>> If I just let things run by themselves right now, the order of the
>>>> requests is random so the event may or may not have been handled by
>>>> the traces before it is by the experiment.
>>>>
>>>> How would be the best way to solve this?
>>>>
>>>> Thanks,
>>>> Geneviève
>>>> _______________________________________________
>>>> tracecompass-dev mailing list
>>>> tracecompass-dev@xxxxxxxxxxx
>>>> To change your delivery options, retrieve your password, or
>>>> unsubscribe from this list, visit
>>>> https://dev.eclipse.org/mailman/listinfo/tracecompass-dev
>>>
>>>
>>> _______________________________________________
>>> tracecompass-dev mailing list
>>> tracecompass-dev@xxxxxxxxxxx
>>> To change your delivery options, retrieve your password, or
>>> unsubscribe from this list, visit
>>> https://dev.eclipse.org/mailman/listinfo/tracecompass-dev
>>
>> _______________________________________________
>> tracecompass-dev mailing list
>> tracecompass-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or
>> unsubscribe from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/tracecompass-dev
>
>
> _______________________________________________
> tracecompass-dev mailing list
> tracecompass-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or
> unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/tracecompass-dev