[platform-dev] Performance Testing Update

[Erich Gamma <erich_gamma@xxxxxxxxxx>]

Here is an update on where we are with regard
to performance testing and an outline for moving forward.

The full work item with regard to performance for 3.1 is described
succinctly in bug 71123.

For M1 the goal was that each component implements a performance test
using the org.eclipse.perfmsr plug-in. We were partially successful... and
7 plug-ins are now contributing performance tests.
One of them is the text component which
has invested a lot in both implementing tests and performance testing
infrastructure.
In particular this infrastructure enables to define criteria when a
performance
test should be considered as a failure.

Releng has integrated the performance test execution into the
build process.

Feedback on this part of the effort was collected in bug 71383.

After M1 we have formed a dynamic team and
we had some initial meetings. Here is the plan that came out
of these meetings.

*     implement a replacement for the org.eclipse.perfmsr
plug-in. This plug-in can no longer be maintained by the
original contributor. The home of the replacement will be
a new org.eclipse.test.performance plug-in. The old plug-in
will be left in place and existing tests will not break.

The plug-in will include:
-     an API for writing performance tests
-     infrastructure to collect the performance data results.

The API for writing performance tests is based on and continues the work
from the text team. The infrastructure will be based on
some lightweight data base (initial investigations have
started with Cloudscape).

The API for writing performance tests will be finalized next week.


*     implement an initial set of critical performance tests
for both speed and space:
-     eclipse start-up
-     full build of a large workspace
-     opening a Java editor (in place)
These tests will be used as an initial performance indicator.
More tests will be required to appear during M2 to
cover all critical aspects of eclipse.


*     Once we have performance API and data collection in place
we will work on how to publish and report performance data.

--erich