Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cross-project-issues-dev] Producing downloadable p2 repositories

The long term goal (post 3.5) is for everything to have p2 metadata. This particular recommendation is one step along the way to help us solve the issue of metadata uniqueness where one should not regenerate an IU for a bundle when the IU already exists in a repository (e.g. a product should not have to recreate the metadata for org.eclipse.equinox.registry).
Therefore, the intent for these repositories is mostly to target the releng crowd and potentially ppl creating their target (even though for those they are probably much better off using the PDE target provisioner). As such we have enhanced PDE Build to consume those repos straight into a target (http://download.eclipse.org/eclipse/downloads/drops/S-3.5M5-200902021535/eclipse-news-M5.html).
We effectively do not expect regular users to use those repos in the dropins esp if they contain packed plug-ins, but simply use them to install from them. Remember unzipping is not installing :)

That said, there is still the possibility to create zips that could be extracted in the dropins if all the bundles were in their runnable form (e.g. folder or jar when appropriate). However I'm not a fan of this approach if we can not get consistency across what the user can and cannot do with those repos (e.g. GEF can be used in the dropins and EMF can not). It just makes the end users life harder.
The other thing here is that we tried to keep things as simple as possible and zipping up repos was by far the simplest recommendation. One could surely create one repository to put on a remote site and another one to put in the zip so it can be unzipped to run, but this sounded like more work and would scare more ppl away :)

HTH

PaScaL

> What does p2 do when in encounters an unpacked p2 repo containing packed
> jars, dropped into the dropins folder? Does it unpack those jars,
> effectively doubling the disk footprint on the client' machine as it
> installs them into the eclipse root folder? Or does it *replace* the
> packed jars w/ normal jars, and keep them in dropins?

>
> The last time I checked the numbers pack200 (using Java 5.0) reduced the
> size of sites' update jars by 30-40%. So, sure, you're only downloading
> 7M instead of 10M, but then on disk you've eaten up 17M instead of 10M.
> Save 30% bandwidth, but incur 170% disk usage.
>
> I can see the benefit for mirrors, but this sounds like teh suck for the
> end user, especially since some of those unpacked jars still need to be
> unpacked again to be useful (eg., feature jars) - which means we've not
> got the same content three times on disk.
>
> Or am I off-base here?
>
> Nick
>
> John Arthorne wrote:
> >
> > Hi Anthony,
> >
> > We're unfortunately not in a position to be able to tell people what
> > they must do ;). We were hoping to provide sufficiently convincing
> > arguments that you would *want* to provide zips of your repositories,
> > since this makes it much easier for downstream components and products
> > to consume your project's binaries and metadata (and a happier Eclipse
> > Foundation due to reduced disk footprint and bandwidth use). And making
> > life easier for your consumers is typically in projects' best interests...
> >
> > John
> >
> >
> >
> > *Anthony Hunter/Ottawa/IBM@IBMCA*
> > Sent by: cross-project-issues-dev-bounces@xxxxxxxxxxx
> >
> > 02/10/2009 12:16 PM
> > Please respond to
> > Cross project issues <cross-project-issues-dev@xxxxxxxxxxx>
> >
> >
> >    
> > To
> >    Cross project issues <cross-project-issues-dev@xxxxxxxxxxx>
> > cc
> >    
> > Subject
> >    Re: [cross-project-issues-dev] Producing downloadable p2 repositories
> >
> >
> >    
> >
> >
> >
> >
> >
> > Hi John,
> >
> > Your email says "The p2 team is now recommending". It is not clear to me
> > what you are asking of projects for Galileo, if anything.
> >
> > Cheers...
> > Anthony
> > --
> > Anthony Hunter _mailto:anthonyh@xxxxxx.com_
> > Software Development Manager: Eclipse Open Source Components
> > IBM Rational Software: Aurora / GEF / GMF / Modeling Tools
> > Phone: 613-270-4613
> >
> >
> > Inactive hide details for John Arthorne---02/10/2009 09:49:48 AM---If
> > you are responsible for your project's builds and packagiJohn
> > Arthorne---02/10/2009 09:49:48 AM---If you are responsible for your
> > project's builds and packaging, please read on!
> >
> > From:    
> > John Arthorne/Ottawa/IBM@IBMCA
> >
> > To:    
> > cross-project-issues-dev@xxxxxxxxxxx
> >
> > Date:    
> > 02/10/2009 09:49 AM
> >
> > Subject:    
> > [cross-project-issues-dev] Producing downloadable p2 repositories
> >
> >
> > ------------------------------------------------------------------------
> >
> >
> >
> >
> > If you are responsible for your project's builds and packaging, please
> > read on!
> >
> > As you all know, providing update sites that include p2 metadata is a
> > "must do" for the Galileo release. Now that everyone is producing this
> > metadata, the obvious question that arises is how do people building
> > products and packages consume this metadata? Everyone is accustomed to
> > consuming Eclipse project output in the convenient form of zip files:
> >
> > * zips are easily replicated to company mirrors, which reduces bandwidth
> > costs for both producers and consumers
> > * zips are a reliable and consistent input for builds. If you keep the
> > input zips around, you can reproduce an old build easily and reliably
> > * power users can hack together applications by unzipping various zips
> > into the Eclipse dropins folder
> >
> > However, there are also numerous advantages to consuming project output
> > in the form of p2 repositories:
> >
> > * Repositories can use pack200 to drastically reduce transfer costs and
> > disk footprint
> > * Repositories contain p2 metadata that would otherwise need to be
> > generated on the fly by p2 on first startup
> > * As projects start to produce and exploit more advanced p2 metadata, it
> > can no longer be generated on the fly (think chmod and sym-link metadata
> > for example)
> > * A project can produce a single repository containing all of their
> > project's output. Consumers then have the flexibility to install only
> > the portions they need. In the past this consumer flexibility was only
> > possible by having the producer provide numerous zip files containing
> > the different permutations of their project output. These large
> > collections of zips are a maintenance nightmare for producers, and lead
> > to slower builds and higher disk usage.
> >
> > On the other hand, remote repositories don't make for a reliable build
> > input. They expose builds to transient communication failures, they may
> > change or be deleted, they are difficult to mirror, they add to
> > bandwidth costs if they are consumed remotely on every build, etc.
> >
> > So, how do projects offer all of the advantages of both zips and p2
> > repositories? The answer: zipped p2 repositories. Simply take the p2
> > repositories you are producing today, zip them up, and make them
> > available on your project download page. Consumers can then download
> > these repositories, and use them offline in all the same ways they use
> > either zips or remote repositories today. The p2 team is now
> > recommending this zipped repository format as the downloadable format of
> > the future. For more details, see this wiki page:
> > _
> > __http://wiki.eclipse.org/Equinox/p2/Metadata_Consumption_
> >
> > Thanks,
> > The p2 team_______________________________________________
> > cross-project-issues-dev mailing list
> > cross-project-issues-dev@eclipse.org_
> > __https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
> >
> > _______________________________________________
> > cross-project-issues-dev mailing list
> > cross-project-issues-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > cross-project-issues-dev mailing list
> > cross-project-issues-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
>
> --
> Nick Boldt :: http://wiki.eclipse.org/User:Nickb
> Release Engineer :: Eclipse Modeling & Dash CBI
> _______________________________________________
> cross-project-issues-dev mailing list
> cross-project-issues-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev


Back to the top