Re: [tracecompass-dev] Performance of CPU usage analyses with many threa

[Genevieve Bastien <gbastien@xxxxxxxxxxxx>]

Hi Rudolf,

Thanks for looking into this.

Some of us started to look at this, at patch
https://git.eclipse.org/r/#/c/137404/  But some things were not working
in our cases.

You're welcome to contribute your solution! We'll gladly review it.
Here's the contributor guidelines
https://wiki.eclipse.org/Trace_Compass/Contributor_Guidelines

Regards,

Geneviève


On 2019-03-07 8:38 a.m., rreinsberger@xxxxxxxxx wrote:
> Hello Genevieve,
>
> We now did some debugging ourselves regarding the CPU usage check box
> tree and the rendering performance.
>
> As it seems the pure graph rendering for the >1500 threads is done in
> about 20s (without any check box handling) - which is acceptable.
>
> With our improvements on the check box handling (in
> TriStateFilteredCheckboxTree.java) we are down to 40s from selecting
> all 1500 threads (i.e. check boxes) to finishing the graph rendering.
> We would be ready to submit these improvements but as we are no TC
> committers (yet) how can we best do that?
>
> However, although this is a huge improvement from the application
> previously becoming unresponsive after 30 minutes, even the remaining
> check box handling overhead is somewhat unsatisfyingly high and we
> should consider further optimizations.
>
> Best Regards,
> Rudolf
>
>
> Am 2019-02-21 18:43, schrieb Genevieve Bastien:
>> Hi Rudolf,
>>
>>
>> There was no benchmark for the CPU Usage analysis, so I made one that
>> queries the backend. For manyThreads, it takes 2.15 minutes to query
>> 10 times the tree on the left and the CPU usage for all threads. There
>> was an easy performance fix, got that down to 1.5 minutes for 10x the
>> full queries (so ~9s for one query of both tree and graph data). So
>> the problem is definitely not the backend (the algorithm is not
>> linear).
>>
>> After doing some more profiling according to your comments, one would
>> think that if the checkboxes are not visible, they would not be
>> rendered, but that is apparently not true, it is still checking the
>> checked state of each box, no matter if it's visible or not. And that
>> still takes forever. Somebody in the team is looking into this.
>> Hopefully we'll "fix" that pretty soon.
>>
>>
>> Thanks and don't hesitate if you notice any other such performance hit!
>>
>> Regards,
>>
>> Geneviève
>>
>>
>> On 2019-02-21 4:36 a.m., rreinsberger@xxxxxxxxx wrote:
>>> Hello Geneviève,
>>>
>>> I tried the same with a collapsed tree so that only the topmost
>>> check box remains. So, selecting this one implicitly selects all
>>> 1535 collapsed threads. Although there is nothing apparently to
>>> render in this case (in terms of individual thread checkboxes) the
>>> result is the same as with the expanded tree view.
>>>
>>> However, trying to render just individual threads from this example
>>> takes between 5s to 7s on my machine (which is well aged but with an
>>> i7 core after all). That would roughly mean about 2 hours to render
>>> all 1535 threads. So, I am not that sure that it's definitely the
>>> GUI check boxes causing this run time?
>>>
>>> As the many-threads example I use is from the test repository do you
>>> know if there is any test protocol for this test case telling about
>>> the expected results and run time?
>>>
>>> Regards,
>>> Rudolf
>>>
>>>
>>>
>>> Am 2019-02-18 16:44, schrieb Genevieve Bastien:
>>>> Hi Rudolf,
>>>>
>>>> Wow, reading this bug report brings back memories... Claudette
>>>> nodes...
>>>>
>>>> Anyway, indeed, on the backend side, there should be no performance
>>>> problems anymore, and from doing a bit of sampling after reading your
>>>> mail, the performance problem you're experience comes from the
>>>> frontend: The UI spends most of its time rendering checkboxes of the
>>>> tree items... Because there's a lot of items in this tree, yet only a
>>>> few of them are visible. I think the 'tree' widget of those views
>>>> should be more lazy and display only the visible items instead of all
>>>> of them. There should be a bug for that. Especially that we have more
>>>> and more of that type of views with a lot of items...
>>>>
>>>> Thanks for bringing this up,
>>>>
>>>> Regards,
>>>>
>>>> Geneviève
>>>>
>>>>
>>>> On 2019-02-18 8:05 a.m., rreinsberger@xxxxxxxxx wrote:
>>>>> Hello,
>>>>>
>>>>> Evaluating the CPU usage analyses and view I used sample/test data
>>>>> from here:
>>>>>
>>>>> https://github.com/tracecompass/tracecompass-test-traces/tree/master/ctf/src/main/resources/many-threads
>>>>>
>>>>>
>>>>> … and from here (see file attachment in the report):
>>>>>
>>>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=460261
>>>>>
>>>>> Trying to update the CPU usage graph with all threads selected the
>>>>> analyses took a very long time (>30 min) until Trace Compass
>>>>> (v4.2) was eventually rendered unresponsive.
>>>>>
>>>>> It seems that in the bug report referred to above this problem (or
>>>>> at least parts of it) were addressed. However, I am not sure what
>>>>> performance and results are to be expected for this test traces.
>>>>> Is the CPU usage supposed to work and produce results with this
>>>>> trace data in TC version 4.2?
>>>>>
>>>>> Regards,
>>>>> Rudolf
>>>>> _______________________________________________
>>>>> tracecompass-dev mailing list
>>>>> tracecompass-dev@xxxxxxxxxxx
>>>>> To change your delivery options, retrieve your password, or
>>>>> unsubscribe from this list, visit
>>>>> https://www.eclipse.org/mailman/listinfo/tracecompass-dev
>>>> _______________________________________________
>>>> tracecompass-dev mailing list
>>>> tracecompass-dev@xxxxxxxxxxx
>>>> To change your delivery options, retrieve your password, or
>>>> unsubscribe from this list, visit
>>>> https://www.eclipse.org/mailman/listinfo/tracecompass-dev
> _______________________________________________
> tracecompass-dev mailing list
> tracecompass-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or
> unsubscribe from this list, visit
> https://www.eclipse.org/mailman/listinfo/tracecompass-dev