Eclipse Community Forums: B3 » Aggregated repository

Home » Archived » B3 » Aggregated repository - allow updates of mapped features

Aggregated repository - allow updates of mapped features [message #884672]

Mon, 11 June 2012 11:34

Eclipse User

Hi all!

I have difficulties configuring aggregated repository that should serve as a central Eclipse update mirror for our team. We are trying to mirror ~25 external Eclipse update sites (standard set, findbugs, etc.) and also want to aggregate few our internal update sites where our developers are publishing Eclipse extensions they are working on.

The goal is to avoid mirroring of internal update sites and have a single aggregate repository that will "redirect" clients to internal update sites. Any update (new plug-in version, etc.) on the internal update sites should be instantly and automatically available through the aggregated repository.

So far, it doesn't work. I have to manually open b3 model and re-build aggregated repository to make updates from internal update sites avaialble in the aggregated repo.

Here's details about my current setup:
* All features from internal update sites are mapped to a custom category
* "mirror = false" for "validated repository" entries associated with internal update sites
* Generated final/aggregate/ directory doesn't contain mirrored artifacts from internal sites
* compositeArtifacts.xml contains <child/> entry for each referenced internal update site
* content.xml contains two kinds of information about our custom plug-ins and features
** fixed versions that were available at time of aggregate repository generation
** version ranges for feature declared in custom category

I tried both to specify a "permissive" OSGI version range for features and keep this version range open-ended (default setting) but no luck. Updated version of features are perfectly avaialbe when you chek internal update site directly but not from aggregated repository.

I'd be very grateful if someone could give me a hint what is wrong with my current configuration.

Re: Aggregated repository - allow updates of mapped features [message #884723 is a reply to message #884672]

Mon, 11 June 2012 13:22

Eclipse User

Hi Pavel,

It's unclear to me what "it doesn't work" means. What exactly are the problems you're facing? What is it that doesn't
work? And does everything work when you open the model and re-build?

- thomas

On 06/11/2012 05:34 PM, Pavel Tarasenko wrote:
> Hi all!
>
> I have difficulties configuring aggregated repository that should serve as a central Eclipse update mirror for our team.
> We are trying to mirror ~25 external Eclipse update sites (standard set, findbugs, etc.) and also want to aggregate few
> our internal update sites where our developers are publishing Eclipse extensions they are working on.
>
> The goal is to avoid mirroring of internal update sites and have a single aggregate repository that will "redirect"
> clients to internal update sites. Any update (new plug-in version, etc.) on the internal update sites should be
> instantly and automatically available through the aggregated repository.
>
> So far, it doesn't work. I have to manually open b3 model and re-build aggregated repository to make updates from
> internal update sites avaialble in the aggregated repo.
>
> Here's details about my current setup:
> * All features from internal update sites are mapped to a custom category
> * "mirror = false" for "validated repository" entries associated with internal update sites
> * Generated final/aggregate/ directory doesn't contain mirrored artifacts from internal sites
> * compositeArtifacts.xml contains <child/> entry for each referenced internal update site
> * content.xml contains two kinds of information about our custom plug-ins and features
> ** fixed versions that were available at time of aggregate repository generation
> ** version ranges for feature declared in custom category
>
> I tried both to specify a "permissive" OSGI version range for features and keep this version range open-ended (default
> setting) but no luck. Updated version of features are perfectly avaialbe when you chek internal update site directly but
> not from aggregated repository.
>
> I'd be very grateful if someone could give me a hint what is wrong with my current configuration.

Re: Aggregated repository - allow updates of mapped features [message #884949 is a reply to message #884723]

Tue, 12 June 2012 02:49

Eclipse User

Hi Thomas,

Thank you for your answer! I'll try to clarify my question.

By "doesn't work" I mean that updated version of Eclipse feature / plug-in isn't available from aggregated repository until I re-build the aggregated repo with b3.

Let's assume we have INTERNAL_REPO (Eclipse update site for MY_PLUGIN v1.0.0) and AGGREGATED_REPO. Now, imagine a new version of MY_PLUGIN v1.1.0 being released and published to the INTERNAL_REPO. But AGGREGATED_REPO still has v1.0.0 despite the fact that mirroring for INTERNAL_REPO was disabled.

My goal is to have MY_PLUGIN v1.1.0 instantly visible through AGGREGATED_REPO.

Re: Aggregated repository - allow updates of mapped features [message #884996 is a reply to message #884672]

Tue, 12 June 2012 04:39

Eclipse User

Privet Pavel,
Pavel Tarasenko wrote on Mon, 11 June 2012 17:34

* content.xml contains two kinds of information about our custom plug-ins and features
** fixed versions that were available at time of aggregate repository generation

That's the problem. You have to rerun b3 every time you rebuild your internal repository.
A good approach would be to mirror eclipse.org stuff with b3 and then create a composite repository which points to your b3 mirror and to your internal repository.

Regards,
Dennis.

http://wiki.eclipse.org/Equinox/p2/Composite_Repositories_%28new%29

[Updated on: Tue, 12 June 2012 04:46] by Moderator

Re: Aggregated repository - allow updates of mapped features [message #885111 is a reply to message #884996]

Tue, 12 June 2012 09:37

Eclipse User

Thanks for the reply!

Do you know if it is a design goal to have "fixed" plug-in versions in aggregated repository generated by b3 or it's simply a shortcoming of current b3 version?

I really like UI, automatic validation and management capabilities that b3 provides, and it would be very sad if it will be necessary to step back and use different approach to manage our in-house plug-ins.

Re: Aggregated repository - allow updates of mapped features [message #885137 is a reply to message #885111]

Tue, 12 June 2012 10:14

Eclipse User

On 2012-12-06 15:37, Pavel Tarasenko wrote:
> Thanks for the reply!
>
> Do you know if it is a design goal to have "fixed" plug-in versions in
> aggregated repository generated by b3 or it's simply a shortcoming of
> current b3 version?
>
> I really like UI, automatic validation and management capabilities that
> b3 provides, and it would be very sad if it will be necessary to step
> back and use different approach to manage our in-house plug-ins.

One approach as Dennis suggests is to use an aggregation of the repo
from Eclipse and your internal repo. If your internal repo in turn needs
to be aggregated (say multiple groups are responsible for producing
their part), it suggest using an approach similar to what is used at
Eclipse. Each group contributes their repository, and the b3 aggregator
is used to produce the overall result. This result represents a
"release" or some kind and it contains what is considered the latest.
You can then compose repositories to suit your needs - your release
repository would be an composite repository with:

- Eclipse
- Our Composite Releases
- Release 1
- Release 2
- ... etc

You can now very easily modify the repository by adding new releases
(containing only the latest). Pruning is easily done (simply unlink the
repository (only negative effect is that users can not easily revert to
that release).

The cons with this setup is that very large composites with many indexes
are somewhat slower to load by p2 and you may have multiple copies of
artifacts. Unless your repo is huge this probably does not matter...

With this setup it is easy to make other combinations involving faster
moving dev/nightly/integration builds - they are simply linked from
other composite repositories.

In this scenario, b3 helps you check the consistency and correctness of
each release produced by your organization. The only thing it does not
do is help you publish/maintain the last step of linking/managing the
composite repo.

If your organization is smaller, and your build always produce the
release repository, you simply make a composite repo out of the blessed
"released" builds - b3 does not have to be involved in those as you are
building one consistent thing (or it would not build).

Does that help?
- henrik

Re: Aggregated repository - allow updates of mapped features [message #885185 is a reply to message #885137]

Tue, 12 June 2012 11:13

Eclipse User

Thank you for the detailed explanation.
Using a b3 for each team in our organization probably will be a bit an overkill for us.

I will consider a simple aggregation (as suggested by Dennis) that will link together content from "external" Eclipse update sites aggregated with b3 and our internal update sites with in-house plug-ins.

Of course, it would be much more convenient to avoid extra aggregation level and have all configuration managed from b3 model.

Re: Aggregated repository - allow updates of mapped features [message #885354 is a reply to message #885185]

Tue, 12 June 2012 17:25

Eclipse User

On 2012-12-06 17:13, Pavel Tarasenko wrote:
> Thank you for the detailed explanation. Using a b3 for each team in our
> organization probably will be a bit an overkill for us.
>
> I will consider a simple aggregation (as suggested by Dennis) that will
> link together content from "external" Eclipse update sites aggregated
> with b3 and our internal update sites with in-house plug-ins.
>
> Of course, it would be much more convenient to avoid extra aggregation
> level and have all configuration managed from b3 model.

Not sure it was clear, I meant - do one aggregation with b3 of your
different team's results/repositories in order to produce a "release"
(milestone/build/whatever) - compose the resulting repositories (over
time) into one. Prune depending on type or resulting repo.

Teams does not have to use b3, but you may want to delegate the
responsibility to the teams regarding what they contribute (like at
Eclipse), as opposed to using a centralized model. Works just as fine
both ways.

- henrik

Previous Topic:	Dont aggregate to a 'final' catalog
Next Topic:	Maven2 Repositories

Goto Forum:

-=] Back to Top [=-

Current Time: Sat Mar 22 16:08:23 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter