Re: [cross-project-issues-dev] Performance, 3.8 versus 4.2

[John Arthorne <John_Arthorne@xxxxxxxxxx>]

Hi Mike,

There is quite a lot missing from our
old IBM build that we have not yet been able to get running on eclipse.org
hardware. There is a reasonably good summary of that state in [1], mentioned
on this list in the past [2]. All of this is the same for both the 3.8
and 4.2 builds, so maybe a little off topic for this thread. In any case,
here are some pointers for anyone interested in helping:

Correctness/regression tests: There
are many tests still failing in Hudson that pass when running on developer
machines. There is a master bug with pointers to all the remaining open
issues [3]. This umbrella bug is also linked to the 4.2 release download
page, along with a pointer to manual verification of the failures that
occurred in the release build. That list is steadily shrinking and webmasters
have been very helpful so far in tracking problems down.

Performance tests: There is an existing
summary with detailed pointers to the work that is needed here: https://bugs.eclipse.org/374441

Code coverage: Details on why it was
removed are found in bug 373594. I have opened this bug to track work that
is needed to re-enable them: https://bugs.eclipse.org/388956

Mac tests: The mac test machine in particular
has been difficult to get any kind of tests to run on consistently. Tests
take a very long time, and frequently DNF. We heard rumours of a new mac
test machine at eclipse.org, which I think would be a big help here.

Any help is certainly welcome. While
significant performance work happened in the Juno release, the first priorities
were compatibility and correctness.

John


[1] http://wiki.eclipse.org/Platform-releng/Transition_Plans_for_Platform_builds_after_Juno_M6
[2] http://dev.eclipse.org/mhonarc/lists/cross-project-issues-dev/msg07471.html
[3] https://bugs.eclipse.org/bugs/show_bug.cgi?id=381873

"Mike Milinkovich" <mike.milinkovich@xxxxxxxxxxx>
Sent by: cross-project-issues-dev-bounces@xxxxxxxxxxx

09/05/2012 05:32 PM

Please respond to
mike.milinkovich@xxxxxxxxxxx; Please respond to
Cross project issues <cross-project-issues-dev@xxxxxxxxxxx>

To	"'Cross project issues'" <cross-project-issues-dev@xxxxxxxxxxx>
cc
Subject	Re: [cross-project-issues-dev] Performance, 3.8 versus 4.2

McQ,
 
I have to admit that today
was the first time that I realized that we had stopped doing performance
testing and test coverage analysis. It was not what I would consider a
pleasant surprise. 
 
The EMO can help. Both in
finding computing resources and and helping to raise awareness that the
community needs to step up to help. 
 
The first step in resolving
most issues is stating the requirements.  Is there a document anywhere
that describes what the platform team would need in order to reinstate
performance testing in a meaningful way?
 
 
 
From: cross-project-issues-dev-bounces@xxxxxxxxxxx
[mailto:cross-project-issues-dev-bounces@xxxxxxxxxxx]
On Behalf Of Mike Wilson
Sent: September-05-12 5:12 PM
To: Cross project issues
Subject: Re: [cross-project-issues-dev] Performance, 3.8 versus 4.2

We definitely *want* to re-enable the performance tests. The two things that have been holding it up currently are:
1) Resources to get them working on the foundation hardware
2) Working through the issues caused by running the performance tests on shared devices.

When we had a lab to hold dedicated performance machines we put a lot of effort into configuring and managing them so they behaved as close to exactly the same as possible. For example, there was one time that the RAM in a machine failed and we were able to see the impact of replacing that RAM (even though it was rated the same) as a difference in the test results.

In a world with potentially other tasks running on the same machines, wildly variable network traffic, etc. I don't think our current performance testing story will work. If that's true, it means it will be a *lot* of work to get them running again. Btw, if anyone has good insights on this and/or wants to help us get the tests running again, we'd love to get your help.

McQ.

"Andrey Loskutov" ---2012/09/05 16:16:17---Hi, Listening to all this 4.2 performance discussions here and for example at

From: "Andrey Loskutov" <loskutov@xxxxxx>
To: cross-project-issues-dev@xxxxxxxxxxx, cross-project-issues-dev-request@xxxxxxxxxxx
Date: 2012/09/05 16:16
Subject: Re: [cross-project-issues-dev] Performance, 3.8 versus 4.2
Sent by: cross-project-issues-dev-bounces@xxxxxxxxxxx

---

Hi,

Listening to all this 4.2 performance discussions here and for example
at 
[1] I would like to ask if the is a plan to re-enable performance 
regression tests for Eclipse (3.8.x / 4.2.x) platform as we had in the
past before they were disabled in Juno (see [2]).

If there is no such plan yet, shouldn't we have one?

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=385272
[2] 
http://wiki.eclipse.org/Platform-releng/Transition_Plans_for_Platform_builds_after_Juno_M6

Regards,
Andrey

On Wed, 05 Sep 2012 15:29:31 +0200, 
<cross-project-issues-dev-request@xxxxxxxxxxx>
wrote:

> Date: Wed, 5 Sep 2012 09:21:10 -0400
> From: John Arthorne <John_Arthorne@xxxxxxxxxx>
> To: Cross project issues <cross-project-issues-dev@xxxxxxxxxxx>
> Subject: Re: [cross-project-issues-dev] Performance, 3.8 versus 4.2
> Message-ID:
> <OF6B7596FB.B62EF228-ON85257A70.0048E946-85257A70.0049582F@xxxxxxxxxx>
> Content-Type: text/plain; charset="us-ascii"
>
> I suggest anyone having problems to add constructive details on that
bug.
> For example profiler output when repeatedly performing a slow operation,
> what plugins are installed, whether it is reproducible with vanilla
> Eclipse SDK, etc. There are some users reporting pervasive slowdowns,
and
> for many others it is performing well. Something like a listener leak
> could have effects like this in conjunction with particular installed
> plugins. It takes time after any major release to isolate and resolve
> problems like this.
>
> John
>
> On Wed, Sep 5, 2012 at 3:08 PM, Thomas Hallgren <thomas@xxxxxxx>
wrote:
> Hi,
>
> For various reasons I had to switch my development environment from
4.2 
> to
> 3.8 today. I was stunned by the performance improvement after the
switch.
> The 3.8 platform is much MUCH faster. It boots faster, it closes windows
> faster, it shows menus faster, etc. It also seems to consume less
memory
> and be less buggy. The way things stand right now, there's just no
way
> I'll switch back to 4.2!
>
> I must say I was very surprised by this. Why is the 4.2 platform what's
> being fronted on the Eclipse download page when it's user experience
and
> quality is lagging behind this much? Is it just me who have had this
> experience?
>
> Regards,
> Thomas Hallgren
>
> ------------------------------
>
> Message: 5
> Date: Wed, 5 Sep 2012 06:29:22 -0700
> From: "Konstantin Komissarchik" <konstantin.komissarchik@xxxxxxxxxx>
> To: "'Cross project issues'" <cross-project-issues-dev@xxxxxxxxxxx>
> Subject: Re: [cross-project-issues-dev] Performance, 3.8 versus 4.2
> Message-ID: <001201cd8b6a$76b74950$6425dbf0$@komissarchik@xxxxxxxxxx>
> Content-Type: text/plain; charset="utf-8"
>
> Thomas,
>
>
> You are certainly not the only one seeing performance issues with
4.2. I 
> go back and forth between 4.2 and 3.8 every day depending on the project
> I need to work on and the difference is quiet noticeable even on very
> fast hardware. The part I notice the most is the lengthy close all
> editors process. After drilling down into some task and opening a
few 
> dozen editors, clearing workbench of open editors takes several seconds.
> I can literally watch tabs disappear one by one. The same operation
is 
> practically instantaneous on 3.8.
>
>
> For stability, user experience and performance reasons, you will find
> that many third party distros have stayed on 3.8 for Juno.
>
>
> I don?t begrudge 4.x its growing pains. It is a complex technological
> shift with a lot of promise. What I find most troubling is the decision
> process that led to the use of 4.2 for Juno distros. When the decision
> was made, it was plainly evident that 4.2 wasn?t going to match 3.8
on 
> any of the quality metrics. IDE users might have been ok with quality
> drop if 4.2 delivered compelling new functionality that you couldn?t
get 
> in 3.8, yet there is no tangible functional delta. The value of 4.x
> platform is for RCP developers and to certain limited extent for IDE
> plugin developers. Certainly not for IDE users. The refreshed 
> look-n-feel has been touted as a big end user feature of 4.2, but
the 
> new look-n-feel itself has numerous issues that leave it looking like
an 
> unfinished project.
>
>
> Sadly, the user reaction that we?ve been seeing over the last several
> months has been entirely predictable.
>
>
> - Konstantin

-- 
Kind regards,
Mit freundlichen Grüßen
Andrey Loskutov

+Andrey: http://plus.google.com/u/0/113794713998126448910
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev