[eclipse-pmc] 3.5 Retrospective Actions

[Daniel Megert <daniel_megert@xxxxxxxxxx>]

Below I've compiled an initial set of the more important action items out
of the 3.5 retrospective feedback that we discussed in the PMC and arch
calls (transcript: http://wiki.eclipse.org/Eclipse/Galileo/Retrospective).
I suggest we discuss this in the next PMC call and assign owners to each of
them so none of them falls through the cracks.

Dani


PLANNING:
- plan came very late and wasn't updated often
==> make 3.6 plan available earlier and update it more often
- length and dates of milestones were considered good
==> don't change milestone and RC length and dates for the 3.6 plan
- we had less upstream pain in 3.5
==> again declare M5 as freeze for major features and changes that cause
big ripples
- there was not enough time to fix bugs in RC2
==> shift test pass to the end of RC1
==> more actively ask community for help, especially when it comes to
platform testing
- we have very strict rules for RCs but none for maintenance builds; too
many bugs fixed for 3.4.x
==> publish rules of engagement for maintenance builds (there must at least
be a patch with a +1 from another committer)
- having a polish pass was considered goodness
==> plan a polish pass for 3.6 with wiki page (see
http://wiki.eclipse.org/index.php/Polish3.5) to track it
- during RC some PMC members expected to be added to the bugs by the
committer but this was not part of the rules
==> PMC should read the rules of engagement (or we can change the rules if
that's what we want next time)
- some of the additional requirements/bugs that the planning council added
were not well thought through (e.g. capabilities)
==> talk to the planning council to first discuss the items with the
experts in that area and have a clear understanding what projects need to
do exactly before mass-filing bugs

PERFORMANCE:
- Frédéric watching over performance was helpful
==> we need to nominate a person responsible for performance throughout the
3.6 release (Frédéric is moving away from Eclipse soon)
- first 3 milestones were without performance tests
==> make sure to have performance tests early and not just after 3
milestones
- performance degradation comments need to be reset at the beginning of
each release cycle
==> the person responsible for performance must ensure that degradation
comments are reset by pointing to those who failed to do it

BUILD ISSUES:
- often broken I-build and then at least one rebuild; build input quality;
needed rebuilds; milestone builds on Saturday ,...
==> closer track build failures (ask why it happened; check if some
component(s) regularly break the builds)
==> teams must only put code into an I-build that got either verified by an
N-build or by very thorough testing i.e. at least doing a smoke test and
running all component tests
==> teams must not participate in rebuilds except if they asked for it
explaining why they also need a rebuild
- doc changes in RC4 broke the builds
==> make sure doc changes don't break the milestone or RC builds
      ==> PDE who made those tests needs to educate how all other teams can
run them
      ==> no doc changes to be released into map files during milestone and
RC weeks without running those tests

FOUNDATION TOPICS:
- bugzilla is one of our main tools and it slows us down
==> talk to the eclipse foundation about slow bugzilla performance and what
can be done to improve it
- broken builds due to maintenance (e.g. certificate change)
==> talk to the eclipse foundation about service level of build machines,
planned maintenance and planned maintenance windows

ALREADY DONE / COMMUNICATED:
- meeting notes contain too much redundant information
==> people should report those thing that used up majority of their time
and (can be vacation as well) and say which bugs they fixed instead of just
"bug fixing"