[tycho-user] Mirror p2 artifacts from zipped archive deployed as a Maven

["Stewart Francis" <stewartfrancis@xxxxxxxxxx>]

Hi,

 

I'm working for an IBM team using Tycho to build a variety of different Eclipse plugin and RCP based products.  Our products have interdependencies, we're currently controlling compatible versions of those dependencies by using a corporate master pom.xml which has the versions set as properties.  We're using Artifactory as our artifact repository, and publishing entire eclipse-repositories, rather than individual plugins and features, which are excluded.  We take advantage of Artifactory's ability to reference files within zip archives to reference those zip files as p2 repositories in our maven builds with the support that Tycho provides for p2 repository layout.  We build our target platform by composing combinations of p2 repositories, which is working really well for us.

 

However, a significant portion of our build time is spent mirroring p2 artifacts.  My working assumption is that this is because p2 mirroring is single threaded, and negotiating the transfer of each file adds time too.  Additionally, due to the large number of network connections involved (I guess) we're also exposed to network irregularities, resulting in occational SocketTimeoutExceptions whilst mirroring p2 artifacts.  I've not dug into these too far.  We're running our builds in clean Docker environments, so we're hitting these kind of issues in most of our builds.  As a result of some experimentation, I've found it's significantly faster for us to use Maven's dependency resolution mechanism to mirror entire p2 repositories, rather than p2's mirroring support to mirror individual artifacts.

 

I wanted to ask your opinions on whether it'd be possible to extend Tycho's capability for p2 repository layouts with support for zipped p2 repositories, identified by Maven co-ordinates?  Is that sort of thing sensible / feasible?  I've some experience with developing Maven plugins (though nothing as sophisticated as Tycho!), and have had a look at the Tycho source, and it seems like it might be feasible, but could potentially have wide-reaching consequences, depending on implementation.  I'm happy to have a go at prototyping it, if it's not a crazy idea.  I was thinking something like:

 

<repository>

  <id>id</id>

  <url>mvn://groupId/artifactId/version</url>

  <layout>p2-maven</layout>

</repository>

 

I'd be interested to hear your thoughts.

 

Thanks,

 

Stew Francis

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU