Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Virgo » Deployment of Virgo runtime dependencies: Maven-based builds without Eclipse IDE(Options for deploying dependencies; p2 repository, or other, with or without tycho.)
Deployment of Virgo runtime dependencies: Maven-based builds without Eclipse IDE [message #1150941] Wed, 23 October 2013 03:35 Go to next message
Douglas Beattie is currently offline Douglas Beattie
Messages: 6
Registered: October 2013
Junior Member
As implied in my comments of a recent discussion in this thread, there is a trick to provisioning a Virgo deployment, based on the dependencies ('Import-Package:') of each bundle, and the sometimes "nested" dependencies of those imports.

As GianMaria mentioned, a 'p2' repository can be used to hold these dependencies, and Eclipse Virgo can be configured to pull its dependencies as needed from the p2 repository.

There are at least 2 tricks here I'd like to learn:

  1. how to create the repository, with proper naming conventions or hierarchy if needed, and
  2. how to gather -- and filter the dependencies destined for the repository.


For example, using the Maven enforcer plugin and several declarations in the 'properties' section of my POM file, it seems that I'm reasonably able to control which version of Spring components are used. So, at build time, I limit the Spring versions of my application to 3.1.x and above, which show up as "[3.1,4)" in OSGi bundle manifests.

But, since most (but not all) of these are provided by Eclipse Virgo, they should not be included in the list of files which are dependencies which need to be gathered and made available to Virgo before my bundles can be expected to load and resolve properly.

At the moment, I'm using a combination of the maven-dependency-plugin, and a Python program (about 150 lines of code). I run the plugin from the command line as

$ mvn dependency:copy-dependencies -DoutputDirectory=/tmp/deps -DexcludeScope=provided


This pulls a bunch of files from the local Maven build repository
and places them in the output directory. Notice a small sample of the
directory listing:
  spring-aop-3.1.0.RELEASE.jar
  spring-aop-3.1.4.RELEASE.jar
  spring-asm-3.1.0.RELEASE.jar
  spring-core-3.1.0.RELEASE.jar
  spring-core-3.1.2.RELEASE.jar
  spring-core-3.1.4.RELEASE.jar
  spring-expression-3.1.4.RELEASE.jar
  spring-orm-3.1.2.RELEASE.jar
  spring-tx-3.1.2.RELEASE.jar
  spring-tx-3.1.4.RELEASE.jar
  spring-data-commons-1.5.2.RELEASE.jar
  spring-jdbc-3.1.2.RELEASE.jar

The Python program post-processes this set of files by performing lookups based on "major.minor" version, against existing libraries held in the release at 'repository/ext/' and omits most of the spring "dependencies" completely from the final list which is then simply copied into 'repository/usr/', renaming all valid bundles according to naming convention {bundle-symbolic-name}_{bundle-version}. Once all those dependencies are in the 'usr' repository, I'm good. Dropping my bundles into 'repository/usr/' and then dropping my Virgo 'plan' file into the 'pickup' folder has been working great.

So, what of a strategy involving a 'p2 repository'? For a purpose-built virtual appliance (my "application"), there can be no remote p2 repo, so the proper dependencies must exist locally, and this appliance runs standalone. While I suspect there may be a way to start the system at least once and have it "pull" all its required dependencies from a remote location within the corporate LAN, I think this may not be acceptable for bundles which are located/loaded "on demand." And the final product is not necessarily meant to be booted before being deployed. (Think "shrink-wrap.")

Note: I'm not using the Eclipse IDE for development (I switched to IntelliJ IDEA, because Eclipse IDE for almost 10 years still gives me no option for a non-blinking cursor) and the objective is to maintain our Maven projects so they can be built entirely by automated build tools.


. . .
Douglas Beattie
www.linkedin.com/in/beattidp

[Updated on: Wed, 23 October 2013 03:47]

Report message to a moderator

Re: Deployment of Virgo runtime dependencies: Maven-based builds without Eclipse IDE [message #1151162 is a reply to message #1150941] Wed, 23 October 2013 06:57 Go to previous messageGo to next message
Florian Waibel is currently offline Florian Waibel
Messages: 96
Registered: June 2010
Member
Thanks for sharing your way how to deploy Virgo runtime dependencies with Maven. Didn't know about the excludeScope option so far. It's a great convenience.
Re: Deployment of Virgo runtime dependencies: Maven-based builds without Eclipse IDE [message #1151652 is a reply to message #1150941] Wed, 23 October 2013 13:56 Go to previous messageGo to next message
GianMaria Romanato is currently offline GianMaria Romanato
Messages: 61
Registered: July 2009
Member
Altough we had a misunderstanding in your previous thread, I asked a colleague of mine to join this discussion, hoping that he will be able to answer at least partially some of your questions.

Douglas Beattie wrote on Tue, 22 October 2013 23:35

Note: I'm not using the Eclipse IDE for development (I switched to IntelliJ IDEA, because Eclipse IDE for almost 10 years still gives me no option for a non-blinking cursor) and the objective is to maintain our Maven projects so they can be built entirely by automated build tools.


This is a bit off topic, but did you know that cursor blinking can be changed globally at the Operating System level? At least in Windows and Linux this is possible. In Windows it is as simple as opening Control Panale -> Keyboard Properties -> Speed (tab) -> Cursor blink rate.

GianMaria.




Developing for Virgo using Eclipse PDE: http://bit.ly/1w0tTit
Global JNDI support in Virgo: http://bit.ly/1to42mn
How to use Hyperic to monitor Virgo Server: http://bit.ly/W1Fst9
Re: Deployment of Virgo runtime dependencies: Maven-based builds without Eclipse IDE [message #1151697 is a reply to message #1150941] Wed, 23 October 2013 14:36 Go to previous messageGo to next message
Nikolas Falco is currently offline Nikolas Falco
Messages: 7
Registered: July 2009
Junior Member
Our build system is based on tycho and we build plugins developed in eclipse IDE.

We develop plugins against a target platform different from eclipse default (Running Platform) that is a sum of virgo plugins plus other third party bundles (orbit, springsource, etc. etc.).

Our job in Jenkins is composed by two steps:

  1. An Ant pre-build step collects all bundles in the virgo distribution (folders: plugins and repository/ext) and specific third party bundle from our git "3rd_party" repository. After that create a p2 repository (see attached ant file where we download eclipse archive from an FTP within LAN);
  2. A Maven 3.0 step (and tycho 0.18.0) where newest created P2 folder was linked in the parent pom.xml repositories section (for now only URL link is supported). See attached pom.xml file


If your builder has problem to access WAN and if you are conversant with developing with PDE eclipse this link can help you to define a "target platform".
This lists all needed bundles/feature.
Export it in the parent project root folder and then link it in the parent pom file (remember that the only "location" supported is of kind "Software Site").
  • Attachment: pom.xml
    (Size: 2.74KB, Downloaded 36 times)
  • Attachment: eclipse.xml
    (Size: 8.01KB, Downloaded 26 times)

[Updated on: Wed, 23 October 2013 15:52]

Report message to a moderator

Re: Deployment of Virgo runtime dependencies: Maven-based builds without Eclipse IDE [message #1155065 is a reply to message #1151697] Fri, 25 October 2013 17:15 Go to previous message
Douglas Beattie is currently offline Douglas Beattie
Messages: 6
Registered: October 2013
Junior Member
Nikolas -- very interesting, and good info! (for the build, I assume) and I'll look closer into this, and study the tycho plugins for Maven.

So, for the provisioning and deployment:

Here for me 'provisioning' means putting all the pieces and configuration -- plan(s), bundles, dependencies -- into an application server installation (Eclipse Virgo of course) on the virtual appliance image. (Other steps to complete the full provisioning might include defaults for DNS resolution, devices/storage, network interfaces or VLAN, etc.)

And, 'deployment' means installing a virtual appliance into a hypervisor, and powering it up, which subsequently would automatically start the application server and associated services for a custom app or set of apps. At the initial point of deployment the 'first boot' home page would typically allow an administrator to further configure application-related end points, host names, credentials, and the like, for the target environment.

At the point of deployment, the purpose-built virtual appliance must be assumed to run inside a secure network environment with possibly an extreme level of firewall and restricted access. So, everything which is required to run the appliance (in particular all bundle dependencies) must exist on the virtual media, and be readily available for Eclipse Virgo for Apache Tomcat. We can't assume any Internet or external access will be allowed or available. (In the experience of a previous employer, if internet access is even attempted during an initial evaluation, their security officers may not like it very much. That is, they should not have to create new routes or deliberately open ports to the outside, to effectively utilize an embedded set of core services and functionality contained within the virtual appliance.)

Anyway, perhaps if I properly create a p2 repository, can I copy it as-is, exactly onto the appliance's virtual media as part of the provisioning process? And then, what would I need to tell Virgo? Can I simply "overlay" it directly into the existing 'p2' folder in the root of the Virgo release, or should I take a copy of the release 'p2' folder and use that as a starting point and destination for (Maven-driven) populating the p2 repo at build time? (And then perhaps copy the final repo contents back into place.)



. . .
Douglas Beattie
www.linkedin.com/in/beattidp
Previous Topic:javax.management error in virgo
Next Topic:Spring MVC with blueprint
Goto Forum:
  


Current Time: Tue Sep 23 04:23:24 GMT 2014

Powered by FUDForum. Page generated in 0.02778 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software