Eclipse 3.7 endgame plan

Status

June 6 2011 - RC4 Done - All further work needs PMC approval

Detailed Timeline

May 2011 
  Transition to fix and polish mode      details   
2 Mon 8:00 am EDT   Start 2-day test pass against M7/RC0      details   
12 Thu 8:00 am EDT   Release Candidate 1 build      rules   
16 Mon 8:00 am EDT  Start 1-day test pass against RC1      details   
17 Tue 8:00 am EDT   Start fix pass      rules   
19 Thu 8:00 am EDT   Release Candidate 2 build     goals   
23 Mon 8:00 am EDT   Start 1-day test pass against RC2      details   
24 Tue 8:00 am EDT   Start fix pass      rules   
26 Thu 8:00 am EDT   Release Candidate 3 build      goals   
30 Mon 8:00 am EDT   All-day test pass against RC3     details   
31 Tue 8:00 am EDT   Start fix pass      rules   
June 2011 
2 Thu 8:00 am EDT   Release Candidate 4 build     goals   
Post RC4 builds will only occur if required to meet translation, documentation and Indigo goals.
PMC approval is required for any changes including documentation.
    
22   Release 3.7 available      details   

Useful Links

What is the game plan?

The Eclipse 3.7 release endgame involves a sequence of test/fix passes leading to the official 3.7 release. Even more than at other times, we welcome all the help we can get with testing and fixing the various Eclipse release candidates. To participate effectively everyone needs to track this schedule closely so that we end up testing the latest release candidate and entering bugzilla bug reports in time to be considered for the fix pass that immediately follows, giving rise to the next release candidate. Throughout the process, we are most concerned with "stop ship" (P1) bugs that must be fixed before we can declare that we have a release. If we discover a "stop ship" bug late in the process, we may have to slip the schedule to allow it to be fixed and retested. This is why it is so important to ferret out "stop ship" bugs as early as possible, while there is still time left in the schedule to address them. Most of the bugs that will be uncovered will be less serious.

During the fix passes, we prioritize the less serious bugs and try to fix as many of the important ones as possible without jeopardizing the schedule or the overall stability of the release. We're always on the look out for "regression" type bugs where we somehow manage to break something that had been working fine before. Regressions are an important warning sign that our optimism and enthusiasm is outpacing our understanding and abilities. Calling special attention to regressions helps us to collectively bring our head back in line with our feet, so to speak. With each cycle, we gradually raise the bar on the kinds and numbers of changes that we will consider making, until we reach a point where we would only fix "stop ship" bugs and regressions. (The lesser bugs that we don't end up fixing will be reconsidered for the next release.) Because of this progressive tightening, the windows of opportunity for fixing problems within the schedule are relatively narrow. Things works best if everyone pushes in the right direction on the right things at the right times.

As it is virtually impossible to work out all the details in advance, we will be updating this page regularly to reflect current status and current testing emphasis. If you are participating we suggest you bookmark this page in your browser and check back frequently for updates. General announcements during the endgame are posted to the platform-releng-dev@eclipse.org developer mailing list. Anyone participating in the endgame should be subscribed to this list, and should direct any general questions and comments about the process there as well.

Release Candidate - Release candidate builds are like milestone builds. The main difference is that release candidate builds are usually immediately followed by a rigorous test pass. We test each release candidate to find serious bugs and to increase our confidence in what we have. We then fix the serious bugs in each release candidate to get the next release candidate, which ought to be even better. Each release candidate build is kicked off at the indicated time, with the goal being to have a release candidate available within 24 hours. As the build is ready, all of the teams validate it and declare it either "go" of "no go" for testing. Getting a build that is testable may require a few attempts. These happen in rapid succession, and we continue rebuilding and revalidating until we have our next release candidate. It is critical that we have enough time to do test passes. We will slide the schedule and use weekends as necessary if there are delays of more than 24 hours in getting a viable release candidate. Note that we will also do warm-up builds in the days leading up to each release candidate build to do early integration of fixes.

Test Pass - Once we have a release candidate build in hand, we enter an intensive test pass for a limited period of time. Each component team is responsible for preparing a comprehensive test plan for their component. These component test plans cover all the functionality that requires manual testing, and identifies the operating environments in which the testing needs to be done. Each component team is responsible for staffing and carrying out their test plan each cycle. Each component team is expected to have most of the team testing throughout each test pass (a small subset of the team may be focused on concurrently preparing candidate fixes for "stop ship" bugs or other high priority tasks). Everyone in the Eclipse community is encouraged to participate in test passes and report bugs to bugzilla. Ideally, the bug report should explicitly call attention to regressions and potential "stop ship" problems.

Fix Pass - After each test pass, we analyze and prioritize the set of outstanding bugs and enter an intensive fix pass for a limited period of time where we try to fix the most pressing problems. The bugs that we intend to fix for the next release candidate are tagged accordingly (e.g., a bug tagged as Target 3.7 RC1 should be fixed as of the release candidate 1 build). "stop ship" bugs and regressions are always given first priority; less severe problems are addressed by decreasing priority and as many as possible are fixed in the time available. With each successive release candidate, we also tighten the rules for the kinds of changes will be allowed to the code base and increase the number of people that check the changes before they are released. The rules apply to any and all changes to the code. Any committer for any Eclipse component can perform the checking duties. All committers for a component have the right to veto a change (with an explanation) even after it has been released into the code base. If such a veto occurs, the change automatically comes out until the vetoing committer's concerns are addressed. The committer who releases a given change, the person that checks the change, and the component leads that approved making the change in the first place, are jointly responsible for seeing it through. In other words, we expect a strong commitment to personally help fix any problems caused by changes made in fix passes.

Holidays

Here are some holidays during the freeze period to keep in mind:

  • May 3 - Poland
  • May 23 - Canada
  • June 2 - Switzerland
  • June 13 - Switzerland

Details

Transition to fix and polish

  • All components transition on May 2 to polishing and fixing bugs for remainder of release cycle.
  • PMC approval is required for feature work including API changes being done after M7. Use the eclipse pmc mailing list to request API and feature work approval. If a change is made to API to make it binary compatible with 3.6, technically this is still an API change, and thus it should be treated in the same way as any other API change requests.
  • No changes are to be released without appropriate approval and associated bug report. See the fix pass rules of engagement for specific approval requirements for each release candidate.

RC0/M7 Goals

  • RC0 and milestone M7 are one and the same
  • All components feature complete.
  • Accurate prioritization of all outstanding defects.
  • String externalization complete (including mnemonics).
  • 3.7 test plans posted.

RC1 Goals

  • Accurate prioritization of all outstanding defects.
  • All work on polish items complete.
  • Final API.
  • No outstanding P1 defects.
  • As few P2 defects as possible.

Test pass prior to RC1

Two day test pass involving entire community, using the M7 build (also called RC0). The goal of this test pass is to get broad testing coverage of the workflows, features, and platforms that aren't regularly tested during our day to day development. Committers should take the opportunity to browse through New & Noteworthy documents from past milestones and give all the new features for the release a good workout.

Test pass prior to RC2

Full day test pass involving entire community, using the most recent nightly build to help stabilize HEAD for the upcoming RC2 build. Committers with high priority fixes to make for RC2 can opt out of testing to focus on getting in fixes. However, all committers should be working with the test candidate build.

RC2 Goals

  • Final artwork in place.
  • Accurate prioritization of all outstanding defects.
  • No outstanding P1 defects.
  • As few P2 defects as possible.

Test pass using RC2

Concerted 1-day testing effort on RC2 involving entire community including all component teams. In an effort to mix things up and hold off the onset of "tester fatigue", each component team will be designating one team member that will be assigned to test some other component.

RC3 Goals

  • Accurate prioritization of all outstanding defects.
  • No outstanding P1 defects.
  • As few P2 defects as possible.

Test pass using RC3

Concerted 1-day testing effort on RC3 involving entire community including all component teams, searching for regressions and on the lookout for undiscovered "stop ship" defects.

RC4 Goals

  • Accurate prioritization of all outstanding defects.
  • Stable code base; no outstanding P1 defects.

Release 3.7

Ship Eclipse 3.7 during the last week of June.

There is no formal test pass for RC4 and beyond other than to check for last minute regressions. We will perform sanity checking focused on documentation. Release 3.7 should be complete and available for download as soon as we are satisfied with it.

Fix pass rules of engagement

May 2-12 - contributions to RC1

Focus:(1) P1 defects, (2) performance defects, (3) special "polish" items. Fixing other defects has lower priority.

Fix approval: Another committer must +1 your bug report. All changes must have their associated bug reports tagged 3.7RC1. Ongoing changes to component documentation do not require special approval.
No bugs must appear in this query. When your bug appears there, make sure that it gets a review+ or add the 'Documentation', 'example' or 'test' keyword if appropriate.

API change approval: PMC must approve all API changes. No changes are to be released without prior approval and associated bug report. Send the request for approval to the eclipse pmc mailing list. If a change is made to API to make it binary compatible with 3.6, technically this is still an API change, and thus it should be treated in the same way as any other API change requests.

Notification requirements: None.

Extra checking requirements: One additional committer must check all code changes prior to release.

May 16-19 - contributions to RC2

Focus: (1) P1 defects, (2) performance defects, (3) special "polish" items. Fixing other defects has lower priority.

Fix approval: Two additional committers must +1 your bug report. All changes must have their associated bug reports tagged 3.7RC2. (Ongoing changes to component documentation do not require special approval.)
No bugs must appear in this query. When your bug appears there, make sure that it gets a review+ or add the 'Documentation', 'example' or 'test' keyword if appropriate.

API change approval: PMC must approve all API changes. No changes are to be released without prior approval and associated bug report. Send the request for approval to the eclipse pmc mailing list. If a change is made to API to make it binary compatible with 3.6, technically this is still an API change, and thus it should be treated in the same way as any other API change requests.

Notification requirements: None.

Extra checking requirements: Two additional committers must check all code changes prior to release.

May 23-26 - contributions to RC3

Focus: Serious defects only; documentation.

Fix approval: Two additional committers and a component lead must +1 your bug report (3 people who are not the one making the change). All changes must have their associated bug reports tagged 3.7RC3. (Ongoing changes to component documentation do not require special approval.)
No bugs must appear in this query. When your bug appears there, make sure that it gets a review+ or add the 'Documentation', 'example' or 'test' keyword if appropriate.

API change approval: PMC must approve all API changes. No changes are to be released without prior approval and associated bug report. Send the request for approval to the eclipse pmc mailing list. If a change is made to API to make it binary compatible with 3.6, technically this is still an API change, and thus it should be treated in the same way as any other API change requests.

Notification requirements: Announce bug numbers of intended non-doc changes to platform-releng-dev@eclipse.org mailing list.

Extra checking requirements:Two additional committers must check all code changes prior to release. Person who reported bug should mark the bug as verified once they have retested.

May 30 - June 2 - contributions to RC4

Focus: Serious defects only; documentation.

Fix approval: Component lead plus one other component lead must approve all work on a component. In addition, any component lead can veto a change with cause. No changes are to be released without associated bug report tagged 3.7RC4 including risk assessment and prior approvals. (Ongoing changes to component documentation do not require special approval.)
No bugs must appear in this query. When your bug appears there, make sure that it gets a review+ or add the 'Documentation', 'example' or 'test' keyword if appropriate.

API change approval: PMC must approve all API changes. No changes are to be released without prior approval and associated bug report. Send the request for approval to the eclipse pmc mailing list. If a change is made to API to make it binary compatible with 3.6, technically this is still an API change, and thus it should be treated in the same way as any other API change requests.

Notification requirements:Announce bug numbers of intended non-doc changes to platform-releng-dev@eclipse.org mailing list.

Extra checking requirements:Two additional committers must check all code changes prior to release. Person who reported bug should mark the bug as verified once they have retested.