[eclipse-pmc] p2 integration into the SDK

[Pascal Rapicault <Pascal_Rapicault@xxxxxxxxxx>]

Hello,

Following feedback/questions provided by Jeff, please find a description of
how p2 will be integrated into the SDK

PaScaL

The SDK 3.4 will ship with p2 enabled by default. It means that "Help >
Software updates" will open p2 UI. That said, for obvious compatibility
reasons, update manager will still be shipped with the SDK and will be
fully functional. However, by default its UI will not be reachable from a
menu to avoid confusions.

Compatibility
p2 is backward compatible with p2 and also allows for a co-existence mode.

- End-user level compatibility
 - p2 users will be able to install transparently from existing update
sites. At the moment, features using install handlers [1] can *not* be
installable by this mean (note that RAD has zero install handlers). Time
permitting we may provide something redirecting the user to the old UM UI
when install handlers are encountered.

 - Plugins producers will keep on authoring features and publishing those
on update sites. At build time they will be able to indicate whether or not
p2 repositories should be generated. Therefore, update sites produced from
3.4 will be working both on p2 and UM not forcing migration or more work to
ensure usability from the two technologies. Should producers decide to only
ship p2 repositories, they can delete the site.xml.

- Technology compatibility/co-existence
Now that we have two technologies, there are three ways to configure
eclipse. These configuration choices have to be made when the product is
being built. Moving from any of the configuration to another one will not
be supported.
1) UM manager only. Update manager is left unmodified and will therefore
work the way it used to. UM will not no be able install from p2
repositories.

2) p2 / UM co-existence. p2 and update manager are both shipped and fully
functional. Depending on the configuration of the system, either one UI or
both could be made visible to the end-user.
When an operation is done by update manager, p2 picks up the change by
monitoring the platform.xml [2] and applies them in p2. Conversely when p2
installs something, the platform.xml is being kept up-to-date. It is to be
noted that since UM works in term of features and platform.xml, p2 is
forced to deliver features.

3) p2 only. p2 is used as the provisioning technology. p2 users will be
able to install transparently from existing update sites and of course
leverage p2 specific content. Because of PDE inner workings, even in this
configuration p2 is still required to deliver features and maintain a
platform.xml. We may explore a "pure" mode (where all the update manager
artifacts are absent) to deliver RCP applications.

Install shape
p2 supports a variety of install shapes, however for the 3.4 release we
focus on the two ones below

- The shape of the eclipse SDK becomes:
      eclipse/
            plugins/
            features/
            artifacts.xml
            configuration/
                  org.eclipse.equinox.simpleconfigurator/
                        bundles.txt
            p2/
                  org.eclipse.equinox.p2.engine/
                        profileRegistry.xml
                        installRegistry.xml
                  org.eclipse.equinox.p2.director/
                        rollback.xml
                  org.eclipse.equinox.p2.core/
                        downloadCache/
                              artifacts.xml


- In case of an installation where the bundles are pooled (this is obtained
by using an installer)
      rad/
            configuration/
                  org.eclipse.equinox.simpleconfigurator/
                        bundles.txt
                  org.eclipse.update/
                        platform.xml
            eclipse.exe
            eclipse.ini
            eclipsec.exe

      eclipse/
            configuration/
                  org.eclipse.equinox.simpleconfigurator/
                        bundles.txt
                  org.eclipse.update/
                        platform.xml
            eclipse.exe
            eclipse.ini
            eclipsec.exe

      <...>/p2
                  <same content as above>

      <bundlePool>/
                  artifacts.xml
                  plugins/
                  features/


Here is a description of the files contained in the p2 folder. To better
understand the nature of each file it is important to remember that p2 is
designed to be a "management system" capable of dealing with multiple
install at a time. To that end, the profileRegistry lists all the eclipse
installs managed by this "agent". It stored things like the OS/WS/Arch of
the platform. UM had no equivalent since these values were obtained from
the running system. Then the installRegistry lists on a per profile basis
all the Installable Units [3] installed into it. For a given profile this
is equivalent to the UM platform.xml.
The rollback.xml captures the former state of all the profiles managed by
an "agent". UM had a similar concept in the form of a platform.xml.
The downloadCache/ is the folder where things that needs to be downloaded
before being installed will be stored.
The bundlepool is the folder where eclipse installs managed by one "agent"
can share plugins. The artifacts.xml is a TOC of the folder and allows to
see the folder as a repository.

The bundles.txt lists the bundles that must be run by OSGi. The UM
equivalent is the platform.xml.

Now, coming back to the install shape, the "self-contained" shape is an
installation where the p2 folder is located under the eclipse folder (and
thus only managing this system), and where the bundle pool is stored in the
install location. folder. The "shared install" shows a case where the p2
folder is stored in a wellknown place on the system and where each
installation contain only the strict minimal number of files. This latter
shape does allow for moving the installation on disk.

None of these file is API and we do not expect those to be manually
modified. They are all manipulated through tools provided with p2 or result
from provisioning operations.


Dropin support
What is referred to as dropin support is the ability to drop plug-ins in an
eclipse install and have them automatically discovered on startup. With p2
we are are trying to move away from that model by providing better
metaphors to install.
- No more unzip and run with -clean. Users don't drop plugins into the
plugins folder, instead they drop plugins in managed dropin areas. These
are folders being monitored for addition / removal of plugins. The
frequency at which these folders are polled can be set, thus giving us the
ability to poll on startup, or at a given interval while the system is
running thus giving the ability to install plug-ins by unzipping without
restarting.
- The dropin support is handled uniformely by p2. All plug-ins and features
appearing in a dropin folder are going through a complete install
life-cycle, like if they had been install through the p2 UI. As such it
would be possible to provision missing prerequisites of the plug-ins added
to the dropins folder (not in 1.0).


Layering UM on p2.
The layering of UM on p2 has been considered. However we did not go that
route because from our user base it appeared that a non trivial amount of
teams had short-circuited UM by directly editing the platform.xml (i.e.
Rational Install Manager) and thus layering with removal of such
fundamental files would have broken people.


[1] install handlers: a mechanism to execute user code as part of the
installation process.
[2] platform.xml: describe the system in UM terms. It lists features and
plug-ins.
[3] Installable units: p2 representation of what is installable. It
contains things such as dependencies and configuration information.