Eclipse Community Forums: Oomph » How to add a mirror to Oomph?

Help

Home

Home » Eclipse Projects » Oomph » How to add a mirror to Oomph?

Show: Today's Messages :: Show Polls :: Message Navigator

How to add a mirror to Oomph? [message #1725969]

Tue, 08 March 2016 16:50

Silvestre Martins

Messages: 84
Registered: July 2009

Member

I have a P2 mirror provided by Nexus (in-house) for some update sites such as http://download.eclipse.org/eclipse/releases/mars.

I was able to force Oomph to use this P2 mirror by adding directly the Nexus URLs to the setup file: org.eclipse.products.setup

However this will cause the installation to fail if this mirror is not accessible,
Can I simply add multiple URLs in the same setup file where some URLs are mirror from other ones? And if yes, how does Oomph prioritizes which URL to use?
And doesn't it make the installation slower if I add more URLs since it has more URLs to contact?

I was expecting a different configuration, where I could explicitly add mirror URLs and Oomph would decide if it should use the mirror or the eclipse download site.

[Updated on: Tue, 08 March 2016 16:50]

Report message to a moderator

Re: How to add a mirror to Oomph? [message #1725989 is a reply to message #1725969]

Tue, 08 March 2016 20:55

Ed Merks

Messages: 33139
Registered: July 2009

Senior Member

Silvestre,

Comments below.

On 08.03.2016 17:50, Silvestre Martins wrote:
> I have a P2 mirror provided by Nexus for some update sites such as
> http://download.eclipse.org/eclipse/releases/mars.
>
> I was able to force Oomph to use this P2 mirror by adding directly the
> Nexus URLs to the setup file: org.eclipse.products.setup
You can also use a Redirection Task or use
-Doomph.redirection.foo=http://download.eclipse.org/eclipse/releases/mars->http://foo/bar

>
> However this will cause the installation to fail if this mirror is not
> accessible,
Yes, all approaches I can think of are like that. A proper "mirror" is
something specified by the repository itself. In this case the Mars
repository does specify a mirror request to Eclipse.org which returns a
list of mirrors which are then used (and can be broken without causing
an overall failure).
> Can I simply add multiple URLs in the same setup file where some URLs
> are mirror from other ones?
Yes.
> And if yes, how does Oomph prioritizes which URL to use?
Oomph, doesn't. P2 will use them all and I don't know if there is any
intelligence in which it chooses to use in the end.
> And doesn't it make the installation slower if I add more URLs since
> it has more URLs to contact?
Yes, that's a problem too.
>
> I was expecting a different configuration, where I could explicitly
> add mirror URLs and Oomph would decide if it should use the mirror or
> the eclipse download site.
That would be nice, but it's not how p2 works. In principle we could
augment p2 to add "externally specified" mirrors rather then the
built-in p2 approach where the update site itself must specify its mirrors

Ed Merks
Professional Support: https://www.macromodeling.com/

Report message to a moderator

Re: How to add a mirror to Oomph? [message #1725991 is a reply to message #1725989]

Tue, 08 March 2016 21:41

Silvestre Martins

Messages: 84
Registered: July 2009

Member

Thanks Ed.

Quote:

> I have a P2 mirror provided by Nexus for some update sites such as
> http://download.eclipse.org/eclipse/releases/mars.
>
> I was able to force Oomph to use this P2 mirror by adding directly the
> Nexus URLs to the setup file: org.eclipse.products.setup
You can also use a Redirection Task or use
-Doomph.redirection.foo=http://download.eclipse.org/eclipse/releases/mars->http://foo/bar

Sorry, but I don't understand the syntax of:
-Doomph.redirection.foo=http://download.eclipse.org/eclipse/releases/mars-http://foo/bar

what does "foo" refer to? can it be any name? how does Oomph uses that "foo" token? Does it have any relationship with the "target URL" or can I have this actually in the file:
-Doomph.redirection.foo=http://download.eclipse.org/eclipse/releases/mars-http://intra.my-company.com/nexus/repository/mars ?

Quote:

> I was expecting a different configuration, where I could explicitly
> add mirror URLs and Oomph would decide if it should use the mirror or
> the eclipse download site.
That would be nice, but it's not how p2 works. In principle we could
augment p2 to add "externally specified" mirrors rather then the
built-in p2 approach where the update site itself must specify its mirrors

Do you think it makes sense to submit a request for this?

Report message to a moderator

Re: How to add a mirror to Oomph? [message #1726078 is a reply to message #1725991]

Wed, 09 March 2016 12:52

Ed Merks

Messages: 33139
Registered: July 2009

Senior Member

Silvestre,

Comments below.

On 08.03.2016 22:41, Silvestre Martins wrote:
> Thanks Ed.
>
> Quote:
>> > I have a P2 mirror provided by Nexus for some update sites such as
>> > http://download.eclipse.org/eclipse/releases/mars.
>> >
>> > I was able to force Oomph to use this P2 mirror by adding directly
>> the > Nexus URLs to the setup file: org.eclipse.products.setup
>> You can also use a Redirection Task or use
>> -Doomph.redirection.foo=http://download.eclipse.org/eclipse/releases/mars->http://foo/bar
>
> Sorry, but I don't understand the syntax of:
> -Doomph.redirection.foo=http://download.eclipse.org/eclipse/releases/mars-http://foo/bar
>
The "pattern" is
-Doomph.redirection.{any-name}={any-uri}->{any-other-uri} to define
what's essentially a rewrite rule to redirect any URI to any other URI.
It uses a EMF's URIMap and so "folder" redirections are possible as
well, e.g.,
-Doomph.redirection.eclipse.download=http://download.eclipse.org/->http://my.com/downloads/
will redirect all p2 sites hosted by Eclipse.org to the my.com domain's
downloads area.
> what does "foo" refer to? can it be any name? how does Oomph uses that
> "foo" token? Does it have any relationship with the "target URL" or
> can I have this actually in the file:
> -Doomph.redirection.foo=http://download.eclipse.org/eclipse/releases/mars-http://intra.my-company.com/nexus/repository/mars
> ?
Yes, like that only with -> to separate the two URIs.
>
> Quote:
>> > I was expecting a different configuration, where I could explicitly
>> > add mirror URLs and Oomph would decide if it should use the mirror
>> or > the eclipse download site.
>> That would be nice, but it's not how p2 works. In principle we could
>> augment p2 to add "externally specified" mirrors rather then the
>> built-in p2 approach where the update site itself must specify its
>> mirrors
>
> Do you think it makes sense to submit a request for this?
Yes, I think so. I think it's very common for companies to host mirrors
inside their firewall, so p2's ability to use mirrors is badly limited
by requiring the original p2 host repository to specify a way to
determine all appropriate mirrors.

Ed Merks
Professional Support: https://www.macromodeling.com/

Report message to a moderator

Re: How to add a mirror to Oomph? [message #1726085 is a reply to message #1726078]

Wed, 09 March 2016 13:48

Silvestre Martins

Messages: 84
Registered: July 2009

Member

Ed Merks wrote on Wed, 09 March 2016 07:52

Silvestre,
>> > I was expecting a different configuration, where I could explicitly
>> > add mirror URLs and Oomph would decide if it should use the mirror
>> or > the eclipse download site.
>> That would be nice, but it's not how p2 works. In principle we could
>> augment p2 to add "externally specified" mirrors rather then the
>> built-in p2 approach where the update site itself must specify its
>> mirrors
>
> Do you think it makes sense to submit a request for this?
Yes, I think so. I think it's very common for companies to host mirrors
inside their firewall, so p2's ability to use mirrors is badly limited
by requiring the original p2 host repository to specify a way to
determine all appropriate mirrors.

Should it be an enhancement for the p2 component or specific for Oomph configuration?

Report message to a moderator

Re: How to add a mirror to Oomph? [message #1726122 is a reply to message #1726085]

Wed, 09 March 2016 20:22

Ed Merks

Messages: 33139
Registered: July 2009

Senior Member

Silvestre,

It's not likely p2 will implement any new features so the p2 component
of the Oomph product would be the bugzilla target.

On 09.03.2016 14:48, Silvestre Martins wrote:
> Ed Merks wrote on Wed, 09 March 2016 07:52
>> Silvestre,
>> >> > I was expecting a different configuration, where I could
>> explicitly >> > add mirror URLs and Oomph would decide if it should
>> use the mirror >> or > the eclipse download site.
>> >> That would be nice, but it's not how p2 works. In principle we
>> could >> augment p2 to add "externally specified" mirrors rather then
>> the >> built-in p2 approach where the update site itself must specify
>> its >> mirrors
>> >
>> > Do you think it makes sense to submit a request for this? Yes, I
>> think so. I think it's very common for companies to host mirrors
>> inside their firewall, so p2's ability to use mirrors is badly
>> limited by requiring the original p2 host repository to specify a way
>> to determine all appropriate mirrors.
>
> Should it be an enhancement for the p2 component or specific for Oomph
> configuration?
>

Ed Merks
Professional Support: https://www.macromodeling.com/

Report message to a moderator

Re: How to add a mirror to Oomph? [message #1726128 is a reply to message #1726122]

Wed, 09 March 2016 22:13

Silvestre Martins

Messages: 84
Registered: July 2009

Member

Request created:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=489322

Report message to a moderator

Re: How to add a mirror to Oomph? [message #1852613 is a reply to message #1726078]

Fri, 27 May 2022 09:28

Elie Richa

Messages: 72
Registered: February 2016

Member

Hello Ed,

Ed Merks wrote on Wed, 09 March 2016 13:52

Silvestre,

The "pattern" is
-Doomph.redirection.{any-name}={any-uri}->{any-other-uri} to define
what's essentially a rewrite rule to redirect any URI to any other URI.
It uses a EMF's URIMap and so "folder" redirections are possible as
well, e.g.,
-Doomph.redirection.eclipse.download=http://download.eclipse.org/->http://my.com/downloads/
will redirect all p2 sites hosted by Eclipse.org to the my.com domain's
downloads area.

Does the "folder" redirection mentioned above also apply to RedirectionTask?

I'm trying to create a product setup that would only use my company's P2 servers and mirrors and I would like to redirect all http[s]://download.eclipse.org/* URLs to our servers.

I created 2 Redirection Tasks from http://download.eclipse.org/ and https://download.eclipse.org/ but I still find that ultimately the following 2 repos get added by the installer:

- https://download.eclipse.org/oomph/updates/milestone/latest
- https://download.eclipse.org/egit/updates

I can work around it by adding Redirection Tasks from the above exact URLs, but I was hoping for something that would cover all http[s]://download.eclipse.org/*, or even better, a global redirection of all P2 URLs (regardless of origin) to the company mirror.

Is there a way to do that?

Also a side question, do I understand correctly that Oomph automatically adds the above P2 repos even though they are not part of my P2 Director repo list because my setup contains Git and other Oomph tasks?

Many thanks!

Elie Richa, Ph.D
Software Engineer, AdaCore
https://www.adacore.com

Report message to a moderator

Re: How to add a mirror to Oomph? [message #1852618 is a reply to message #1852613]

Fri, 27 May 2022 13:10

Ed Merks

Messages: 33139
Registered: July 2009

Senior Member

It seemed to work for me. I added these to the product catalog:

<?xml version="1.0" encoding="UTF-8"?>
<xmi:XMI xmi:version="2.0"
    xmlns:xmi="http://www.omg.org/XMI"
    xmlns:setup="http://www.eclipse.org/oomph/setup/1.0">
  <setup:RedirectionTask
      sourceURL="http://download.eclipse.org/"
      targetURL="http://bogus/"/>
  <setup:RedirectionTask
      sourceURL="https://download.eclipse.org/"
      targetURL="https://bogus/"/>
</xmi:XMI>

And the p2 task looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<setup.p2:P2Task
    xmi:version="2.0"
    xmlns:xmi="http://www.omg.org/XMI"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:setup="http://www.eclipse.org/oomph/setup/1.0"
    xmlns:setup.p2="http://www.eclipse.org/oomph/setup/p2/1.0"
    label="Eclipse IDE for Eclipse Committers (2022-06) + Platform + EMF Tools + E4 Tools">
  <repository
      url="https://bogus/modeling/emf/emf/builds/nightly/latest"/>
  <repository
      url="https://bogus/technology/epp/packages/2022-06/202205261200"/>
  <repository
      url="https://bogus/releases/2022-06/202205271000">
    <annotation
        source="http://www.eclipse.org/oomph/setup/ReleaseTrain"/>
  </repository>
  <repository
      url="https://bogus/oomph/updates/milestone/latest"/>
</setup.p2:P2Task>

The dynamic *.ecore models do have annotations with URL like https://download.eclipse.org/egit/updates. The annotation for Oomph is like ${oomph.update.url} so I can see that this variable is likely expanded after the redirections are applied and this property might be in your installer's *.ini. Note that some repos are annotated "ReleaseTrain" to avoid adding the discovery URLs (like the one for EGit) when that's present...

Ed Merks
Professional Support: https://www.macromodeling.com/

Report message to a moderator

Re: How to add a mirror to Oomph? [message #1852623 is a reply to message #1852618]

Fri, 27 May 2022 15:21

Elie Richa

Messages: 72
Registered: February 2016

Member

Thanks for the explanations, Ed.

In my case I'm specifying the RedirectionTasks in a product setup rather than in a catalog.

Could it be a scoping subtlety whereby my redirection tasks only apply within my product, and since the other URLs are getting added from outside (I'm not familiar with the term but I think those are the discovery URLs) they don't undergo redirection?

Elie Richa, Ph.D
Software Engineer, AdaCore
https://www.adacore.com

Report message to a moderator

Re: How to add a mirror to Oomph? [message #1852631 is a reply to message #1852623]

Sat, 28 May 2022 06:07

Ed Merks

Messages: 33139
Registered: July 2009

Senior Member

I tried it even in a Product Version and it still worked and also in a Project and including that in the wizard. Perhaps you can make a small sample that reproduces the problem, or a set of steps I can reproduce locally...

Ed Merks
Professional Support: https://www.macromodeling.com/

Report message to a moderator

Previous Topic:	Thanks Oomph!
Next Topic:	Updating setup files from Nexus hosting

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Sat Apr 20 16:39:13 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter