[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [platform-ui-dev] Performance regressions
|
>If anybody else has additional ideas or would like to help, let us know!
- make sure you use the same VM as used in the official performance tests
- make sure you give your VM the same amount of memory as in the official
performance tests
- run the test manually on 3.4 and 3.5 and look carefully for differences
(e.g. same views open, Welcome page obscuring something, same amount of
markers,. etc.) that could influence your test
Dani
|------------>
| From: |
|------------>
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|Boris Bokowski <Boris_Bokowski@xxxxxxxxxx> |
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|------------>
| To: |
|------------>
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|platform-ui-dev@xxxxxxxxxxx, Oleg Besedin <obesedin@xxxxxxxxxx>, Hitesh K Soliwal <hsoliwal@xxxxxxxxxx> |
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|------------>
| Cc: |
|------------>
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|Frederic Fusier <frederic_fusier@xxxxxxxxxx> |
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|------------>
| Date: |
|------------>
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|10.04.2009 05:49 |
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|------------>
| Subject: |
|------------>
>--------------------------------------------------------------------------------------------------------------------------------------------------|
|[platform-ui-dev] Performance regressions |
>--------------------------------------------------------------------------------------------------------------------------------------------------|
Hi all, (and in particular, hi Oleg and Hitesh, I need your help :-)
We have a couple of performance regressions in the 3.5 stream. To see them,
go to http://download.eclipse.org/eclipse and click on a recent build's
clock symbol. This should bring you to the performance results main page.
Scroll down and click on org.eclipse.ui, then look at the table for all 141
scenarios.
The regressions are in EditorSwitchTest (two scenarios),
OpenCloseEditorTest (three scenarios), and OpenMultipleEditorTest (six
scenarios). Note that the last one already had a regression in the 3.4
cycle but we were not able to fix it for 3.4, and we are now seeing further
regression in 3.5. Also note that we have been told that the results for
the Windows machine currently cannot be trusted.
I have spent the last days trying to reproduce the measurements, on a
freshly installed Linux machine, with no success. I was not able to
reproduce slowdowns in the same precentage range as shown on the
performance result pages when comparing 3.4.0 with newer 3.5 builds.
Our goal in the short term should be to understand what is going on - I
will have to explain this to the other component leads at the next weekly
planning meeting on Wednesday.
Of course, the real goal is to either (1) fix the performance problem, (2)
determine that it is not a serious problem worth fixing, (3) be able to
explain the performance regression by a bug we fixed that could not have
otherwise be fixed, or (4) find someone else to blame (e.g. a lower level
component, or a change in the test setup that is not in our control).
Obviously, (1) would be the best outcome. :-)
Oleg has already helped a lot by writing a wiki page on how to approach
performance testing (?
http://wiki.eclipse.org/Platform_UI/Performance_Tests) and by implementing
a mechanism to run a single performance test case (
https://bugs.eclipse.org/bugs/show_bug.cgi?id=271456). I hope that on
Monday, I can work with Oleg on the reproducability problem (the short term
goal).
Hitesh, could you please run YourKit on the scenarios mentioned above,
against both a recent 3.5 build and a 3.4.0 build, and compare the
resulting traces? See the wiki page Oleg wrote for steps to follow. Any
insight into what has changed between the two would be very useful. You may
want to run the profiler in the mode where it records invocation counts, in
case this makes the comparison easier. If you find anything that can be
optimized easily, feel free to experiment if you can improve the
performance, but I am not expecting any easy wins.
If anybody else has additional ideas or would like to help, let us know!
Thanks,
Boris.
P.S. Note that some of these tests have rather strange performance numbers.
For example, some have *way* more elapsed time than CPU time. Others show a
jump in both the 3.5 and the baseline (3.4.0) numbers, some in December and
some in January, pointing to a cause outside of the Platform UI code. Also,
make sure that you look at the raw data and not only the pretty graphs
(click on the "Raw data and Stats" link on the individual scenario pages).
We even have tests that sometimes go completely off rails, taking more than
a minute for what used to take less than a second.
_______________________________________________
platform-ui-dev mailing list
platform-ui-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/platform-ui-dev
