The Eclipse Simultaneous Release Checklist

Draft - initial working version.


Project:

Date last changed:

This document defines the rules and criteria for participating in the yearly Simultaneous Release. There are more criteria than when releasing at other times partially because there are more projects releasing at once, so the workload needs to streamlined and made more uniform. More important, the extra criteria are included by mutual agreement between projects (via their representatives to Planning Council) so that as a whole, the release will be of better quality, maintainability, and improved consumability by adopters.

The spirit of this document should not be so much as a "contract" of what has to be done to release, but instead as an agreement to make the Yearly Release good, if not great! While each Project does their individual things to make the Release great, this document and process describe how we as a group document the achievement of our agreement. We are always open to better agreements and better documentation of our achievements so feel free to keep track and make suggestions year-to-year (preferably through your Planning Council representative) on how to make our yearly release better. In fact, occasionally changes may be made to this document for clarity or to improve reference links throughout the development cycle, but nothing new that would change work-load will be added after M4.

To foster communication and flexibility where required, any exceptions to these criteria or deadlines will follow the Planning Council Exception Process.

The requirements are divided into three categories:

  1. Requirements to be released as part of the "yearly release", normal release requirements, done earlier than usual.
  2. Requirements to be part of the Common Discovery Site repository and, consequently, the minimum requirements to be part of EPP packages.
  3. Requirements to demonstrate good Eclipse Citizenship, following "the Eclipse Way".

Do the basics ... early

The requirements and conditions stated in this section are the basic minimum required for a project to claim they are part of the yearly Simultaneous Release.

To join a Simultaneous Release, Projects must have stated their intent to do so, and be in a build for the composite site aggregation by M4, at the latest. For projects continuing from previous years, the expectation is they will be in M1, unless they formally withdraw.

The "statement of intent" is done formally by marking the "Simultaneous Release Flag" in the project's Portal meta-data.

In M1 common repo (M4 for new projects)

Portal meta-data updated (By M4, at latest)

Planning

All projects must have their project plan in the Eclipse Foundation standard XML Format (a normal Eclipse requirement). Committing to be in the Simultaneous Release means you commit to having these plans early: M2, for those projects that already know they will be in the Simultaneous Release, M4 will be the latest possible time, for those projects that are new to the Simultaneous Release Train and decide to join after M2. The plans should be updated periodically as things change, or as items are completed.

Initial Planning Document done by M4. URL:

Also, for long term planning, remember that being in a Simultaneous Release also means a commitment to participating the SR1 and SR2 simultaneous maintenance releases.

[New this year.] Once in, always in. Once a project joins one year's Simultaneous Release, it is assumed they will be in the next one, unless they formally withdraw. So, for example, it is required they will meet the subsequent year's Milestone Schedule, have plans done by M2 of following year, etc. Put another way, being part of the Simultaneous Release is not a "one time" activity, covering only the literal release date and not even a "part time" activity covering only part of the yearly development cycle. Instead it is a commitment to stay "simultaneous" on an on-going basis.

Communication

At least one person from each project in a Simultaneous Release must subscribe to cross-project mailing list, since that is the primary communication channel for issues related to the Simultaneous Release. Also, at least one person from each project must subscribe to cross-project bugzilla inbox, as that is the primary bugzilla components for bugs that are truly cross-project, or bugs which are not known to be in one particular component.

People assigned. Names:

Your representative to the Planning Council, either from PMC or Strategic Member, must attend PC meetings and represent you there. Presumably, of course, after meeting or communicating with you and the other projects they represent, so they can fairly bring forward concerns and vote on issue that effect all projects, if required. Put another way, by committing to be in the Simultaneous Release, you agree to abide by all the Planning Council decisions and rules, so be sure your representative understands your project and your situation.

IP Documentation

Projects must have their IP Logs approved (a normal Eclipse requirement) and will follow the Eclipse Legal deadlines to do so. In addition, drafts of the Projects IP Logs must be available every milestone, starting with M5. The ones for M5 and M6 will be understood to be "drafts" but for most projects the IP Logs should be relatively complete by M7. If Projects have changes come in after M7 they can update until the deadline set by the IP staff (usually RC2). The purpose of having these early drafts is so that projects get familiar with what's required, and do not allow work to build up at the end, also to allow questions to come up, and have time to find answers, and also to allow time for issues with automatic IP tools to be addressed.

M5 IP draft URL:

M6 IP draft URL:

M7 IP draft URL:

Final IP URL:

Being in the Simultaneous Release will give your IP some higher priority in getting evaluated, in order to make the date. The higher priority treatment is only for the 5 months or so before the release (after the deadline for CQs). The reason being, of course, is that the rest of the year the IP staff must also get work done for maintenance releases and projects not on the release train. During that part of the year (roughly July to February every year) all CQs are prioritized in a uniform way.

Release Review

The release review archival materials must be complete by the date specified by the EMO, which is usually staged in earlier than for a usual release. (Typically RC2.)

A Project's PMC must approve the projects request for review (a normal Eclipse requirement). In addition, to help organize and streamline the yearly Simultaneous release, a PMC must provide their approval in writing, in the form of a short summary of their projects that are requesting review and summary of the PMC's discussion or method of approving them. (This is meant to be very brief, such as 1/2 to 1 page). The short summary can be documented in a mailing list, PMC Meeting notes, or even a wiki document. A pointer (URL) to the document should be provided to EMO, and will be included in the same notice to the community that provides pointers to the Project Docuware.

URL to PMC's Executive Summary (by RC2):

The public review calls will be organized based on Top Level Project, and at least one PMC member should be on that call to give very brief overview of projects that are requesting the release review (not to exceed 5 minutes, at the very most).

In addition to the ordinarily required Release Review Archival Materials, all Projects participating in yearly Release agree to provide a checklist-with-detail form that describes their compliance (or not) with all of the criteria items described in this document. Note that this checklist-with-detail must be updated every milestone as things are completed, or details added, so progress can be reviewed by Planning Council and potential adopters. The primary report of compliance with the checklist must be provided at least at the level of a Top Level Project. In some cases, such as if sub-projects are very independent of each other, PMCs may decide to document things at a subproject level, and then "roll-up" to a Top Level document, or, if a Top Level Project is known to be uniform and "close knit" then they may provide one summary document that applies to all sub-projects. [This will likely be automated as web-app, possibly with "automatic" roll up].

URL to checklists:

Play well with others ... to be in common repository

The requirements in this section must be met for a project to be on the common, central repository (e.g. /releases/helios) for end users to discover easily and minimum requirements to be included in EPP Packages. The criteria in this section are designed to make sure projects work relatively well, and work well together. This is especially required for adopters who may be using these projects in complicated, interwoven ways so each piece of the puzzle must fit together well and be dependable and be maintainable, as well as being on time and IP clean.

Communication: Build team members (or their designated alternates) from each project may be asked to provide direct communication channels: phone, mail, IM, IRC and at least one build team member must be "on call" during the milestone integration periods.

Releng names and contact data provided. Can send data to Programming Chair ... for privacy reasons we won't publish it publically.

API. Projects should leverage only published APIs of dependencies. All deviations must be documented in bugzillas. These bugzillas may be of the type that a dependent project should provide a required API, or of the type that a consuming project must move to some API that already exists. Note that technically there is no obligation for consumed projects to provide API that is requested ... that depends on many things ... but the main goal of requiring these bugzilla entries is to provide some documentation and measure of the amount of risk associated with non-API use.

Bugzillas open, or can "prove" API clean. Link to API check reports:

List of Bugilla's open for API deviations:

Message Bundles. Projects must use Eclipse message bundles unless there are technical reasons not to.

Message Bundles used throughout.

Message Bundle Exception. Give description of technical reasons, and link to Planning Councils approval.


Version Numbering. Projects must use 4-part version numbers.

Standard Versioning Used.

Exceptions

OSGi bundle format. All plug-ins (bundles) must use the true bundle form. That is, provide a manifest.mf file, and not rely on the plugin.xml file being 'translated' into a manifest.mf file at initial startup. With that, empty plugin.xml files in the presence of a manifest.mf file should not be included in a bundle. (For some old history, see bug 130598.)

Execution Environment. All plug-ins must correctly list their Bundle Required Execution Environment (BREE).

Signing. Projects must use signed plugins using the Eclipse certificate.

Jarred Bundles. Projects must use jarred plug-ins (with unpack=false) unless authorized by the planning council for technical exceptions. Also, nested jars should be avoided if possible since it creates problems for projects that has dependencies to such plug-ins. The OSGi runtime is fine with it but the PDE environment is not able to handle classpaths that contain nested jars.

Re-use and share common third party jars. Any third-party plug-ins that are common between projects must be consumed via Orbit; a Simultaneous Release will not have duplicate third-party libraries (note that this only applies to identical versions of the libraries; thus if project A requires foo.jar 1.6 and project B uses foo.jar 1.7, that's ok, as long as it is required and has a documented reason).

Optimization. Projects must optimize their p2 repositories to reduce bandwidth utilization and provide a better install and update experience for users.

Provide p2 repository. Projects must provide their own project p2 repository for their own project and updates. In addition, they must provide their archives and metadata in a specified format and method to allow at least parts of their repository to be aggregated and mirrored to a common repository. The current process may be modified throughout the year, if improvements can be made.

Capabilities. Each project will provide basic capability/activity definitions to allow for their UI contributions to be hidden. These may be provided in a separate plugins and features to facilitate inclusion and reuse by consumers in product development, or simply well documented so adopters can reuse via copy/paste. Ideally, projects should also provide triggers to facilitate progressive discovery of functionality.

URL to a project's description of their capabilities (even if sample) and/or plugin names adopters can use.

URL to a project's description of their progressive discover strategy or documentation:

Support Translations. All strings must be externalized, and Projects must participate in Babel, meaning it is registered and available for string translation, etc. Projects must freeze the UI sufficiently early to allow the Babel project time to translate strings so there can be simultaneous release of translated versions. The UI should be frozen by M6 (a "freeze" all major changes and additions are done by M6, and changes after that are done in a controlled, well documented fashion, so Babel translators can more easily "keep up" with late changes).

Excel in NL support. The Project must use ICU4J, where appropriate, to excel in NL support. (The latest ICU4J bundles will be in Orbit).

Exceptions:


Branding. Each major project (as determined by participating PMCs) must have an 'About' dialog icon with hover text that displays the provider name. Every plug-in and feature must specify a descriptive provider-name (for features), or Bundle-Vendor header (for plug-ins), as determined by the project's PMC (e.g. "Eclipse Modeling Project" rather than "Eclipse.org"). Also, Projects must contribute to the welcome page when appropriate.

Do No Harm. Projects must work together in any combination of any install. Put another way, this means that users can install any subset of the Helios projects and each of the installed projects will work as well as if it had been loaded independently. If such a problem is identified, the affected projects must track down and fix the problem.

[New this year.] License text consistency. Use standard forms of license documents so it is displayed in the most usable, and concise way during install and update. It is a normal requirement to use a standard Eclipse Foundation "about" template, but where those templates are edited by each project, care must be taken to be sure they are edited in similar ways. You can see an example of the license-consolidating UI in Eclipse Platform Helios M1.

Be a good Eclipse Citizen ... and document it

Projects should exhibit good Eclipse Citizenship, to Release and participate in Common Discovery Site and EPP Packages. These are often "best practices" that some projects have found helpful at Eclipse. These criteria often speak to the quality of the Project, as an Eclipse Project, as opposed to their code or architecture. They are a bit more subjective than some of the other criteria, and the relevancy to any particular project may not be as universal, so there is no set number of items to satisfy. But, it is required that each project document their level of compliance to each item. Especially good Eclipse Citizens will get a gold star, and especially bad ones might get a frowny face.

Engage Community. The Project should actively engage their community to get feedback on milestone builds, and document how they do that. One way to do this is to have a New & Noteworthy for each milestone. New and Noteworthy documents should be something readable and usable not just a static list of all the bugs. Corollary: individual new & noteworthy should be linked in to the collective New & Noteworthy.

If yes, describe, or provide URL:


Usability. Should follow the User Interface Guidelines. The UI Checklist is a good place to start. Also, should participate in a User Interface Best Practices Working Group UI walkthrough.

If yes, describe, or provide URL:

Performance. Project should have measurable performance criteria that are regularly tested against. Projects should devote at least one milestone to performance and scalability improvements.

If yes, describe, or provide URL:

Test Localization. The project should use the Babel Pseudo Translation Test to verify their translatability. [Need better reference link.]

If yes, describe, or provide URL:

Enable Use with All Languages. Should design and test for enabling all languages including bidi, unicode characters, etc. This is different than "translating" the program. For example, while using an English version of Eclipse Web Tools, someone should be able to create a Chinese language web application. [Need "how to" reference link.]

If yes, describe, or provide URL:

Builds. Projects must have a mature, stable build process: documented, scripted, repeatable, and executable by others.

If yes, describe, or provide URL:

Ramp Down Planned and Defined. Projects must have a written ramp down policy by M6, at the latest, and provide link. The plan should describe when the project plans to be feature complete, have API frozen, and similar. See Platform 3.5 Endgame plan as a guideline. See also Helios Final Daze.)

If yes, describe, or provide URL:

Accessibility. Projects should design and test for accessibility compliance, following established guidelines and Eclipse fundamental techniques to achieve accessibility. Projects must document their accessibility work and compliance. Ideally this would be by using a publicly available checklists, such as

but, given the advice of the Accessibility Cross Project Team, for this year's Helios Simultaneous Release, projects can document their work or compliance as a negative, such as "we did not do any accessibility work or testing and do not know the degree of our compliance". But its important to document, so adopters know. If possible, and appropriate, accessibility testing tools can be leveraged such as NVDA. The main accessibility article at Eclipse Corner has been made current (thanks goes to Todd Creasey).

If yes, describe, or provide URL:

[New this year.]. Unit Tests. Projects must have some unit tests that can verify at least basic functionality of a build or distribution. The steps to build and run the tests must be documented and executable by others.

If yes, describe, or provide URL:

[New this year.]. API Policy Defined and Documented. Typically would include how 'APIs' are distinguished from non-API and 'provisional' API, if any. It is recommended that non-API be marked with x-internal in the bundles manifest. Also, should include what the commitment is to API, how long maintained after deprecated, etc. As one example, see WTP API Policy.

If yes, describe, or provide URL:

[New this year.]. Retention Policy. Projects should define and document their retention policy. This should include both zip distributions and repositories. For examples, see WTP Retention Policy and Eclipse Project Update Sites

If yes, describe, or provide URL:

[New this year.]. Project Metrics. Projects should provide some summary metrics, such as number of bundles, number of committers, lines of code, number of bugs opened and fixed. This is so some statements can be made and tracked year-to-year about the size of the simultaneous release.

If yes, describe, or provide URL: