Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Virgo » STS and PAR deployment
STS and PAR deployment [message #545750] Thu, 08 July 2010 22:01 Go to next message
Kristof Taveirne is currently offline Kristof Taveirne
Messages: 6
Registered: July 2010
Junior Member
Hi,

I'm quite new to Virgo and spring Dm in general.

There is something I don't understand using the STS IDE to deploy my par.

Here is my folder structure.

myproject.toplevel
+ myproject.parent
+ myproject.par
+ myproject.jpa
+ myproject.db

I based this on the greenpages directory structure.

My toplevel pom, has just a bunch of <module> element to include the bundles.

My parent project basicly holds dependencyManagement configuration in the pom.

When I maven package the toplevel project, the PAR gets build under myproject.parent/target

But I can not find a way to deploy this directly to Virgo from STS. When I use Add/Remove the toplevel project does not list. The par bundle does list, but the jpa and db projects are not included.

When I do mvn package on the par, I get dependency not found error on the jpa and db bundles.

Is there a way to Add the toplevel project to the Virgo runtime so the PAR would get deployed automatically?

Thanks!
Re: STS and PAR deployment [message #545800 is a reply to message #545750] Fri, 09 July 2010 08:27 Go to previous messageGo to next message
Glyn Normington is currently offline Glyn Normington
Messages: 1222
Registered: July 2009
Senior Member
A PAR file packages a number of bundles. In this example, the PAR should package the jpa and db bundles. Deploying the PAR to Virgo will cause the bundles inside the PAR to be deployed too, but scoped to the PAR.

You may wish to step through the "getting started guide" up to and including chapter 3 to see this working with the (solution form of the) greenpages sample in STS.
Re: STS and PAR deployment [message #546972 is a reply to message #545800] Wed, 14 July 2010 18:58 Go to previous messageGo to next message
Kristof Taveirne is currently offline Kristof Taveirne
Messages: 6
Registered: July 2010
Junior Member
Hi,

Thanks for your answer.

The problem however is not the PAR... I understand how it more or less works. Everything works when I use command line tools (mvn package, cp to deploy to the hotdeploy folder, ...)

I just don't find a streamlined way to do everything from within STS.

This is more of a STS question then it is a Virgo/OSGi question really.

The things I want to be able to do is:
- run unit test on single bundles
- run integration tests across multiple bundles
- deploy the par to Virgo from within STS
- update a single bundle within the par without updating the complete application (par)

Anyway, I'll experiment some more. Hopefully I can start being productive very soon.

I'm installing the new STS 2.3.3 M1 as we speak ... I'll see how that goes
Re: STS and PAR deployment [message #547093 is a reply to message #546972] Thu, 15 July 2010 14:52 Go to previous messageGo to next message
Glyn Normington is currently offline Glyn Normington
Messages: 1222
Registered: July 2009
Senior Member
Please ensure you install the latest dm Server tools (which include the Virgo tools) as described in this blog.

Until the tools are donated to Eclipse, you can use the dm Server tooling forum to discuss the tooling.
Re: STS and PAR deployment [message #547526 is a reply to message #547093] Sun, 18 July 2010 10:34 Go to previous messageGo to next message
Kristof Taveirne is currently offline Kristof Taveirne
Messages: 6
Registered: July 2010
Junior Member
Thank you for your response!

I think I am a few steps further now with the new STS and Virgo plugins.

however, each time I rebuild my application the Virgo console gives me the following message:


[2010-07-18 11:19:24.950] TCP Connection(53)-127.0.0.1 <DE0050W> Cannot refresh par 'myproject' version '0.0.1' as refresh of par artifacts is not supported.

I'm trying to understand why this is, but I can't find anything in the documentation.

Any ideas?

Also, I don't fully understand yet how the dependency management works.

So first I have to make sure the pom is OK in order to compile, test and package everything.
Then I also need to make sure my template.mf file is correct so the osgi framework can resolve all dependencies and this reflects my pom configuration. (template.mf is not autogenerated from the pom, so this is a manual process).

Then the manifest.mf gets generated from the template.mf using bundlor.

But then when I deploy to Virgo, I need to make sure that all dependencies are available on the server.

I would think that, since maven needs those bundles to compile and test, they would also be deployed to Virgo if the bundle is not yet available in the runtime.

I tried to deploy greenpages but that also get an 'unable to satisfy dependencies' because it is missing com.springsource.org.eclipse.persistence

I think it would make sense if all those dependencies would get packaged in the par by mvn unless the scope of the dependency in the pom would be set 'provided'.

I hope I'll get something working soon, because I've been unproductive for way too long on this project struggling with these issues. I would be nice if I could get a working project set-up and test/deploy process in place soon.

All help is welcome!

Thanks!

Greetings,

Kristof
Re: STS and PAR deployment [message #547971 is a reply to message #547526] Tue, 20 July 2010 09:57 Go to previous messageGo to next message
Glyn Normington is currently offline Glyn Normington
Messages: 1222
Registered: July 2009
Senior Member
Kristof Taveirne wrote on Sun, 18 July 2010 11:34
Thank you for your response!

I think I am a few steps further now with the new STS and Virgo plugins.

however, each time I rebuild my application the Virgo console gives me the following message:


[2010-07-18 11:19:24.950] TCP Connection(53)-127.0.0.1 <DE0050W> Cannot refresh par 'myproject' version '0.0.1' as refresh of par artifacts is not supported.

I'm trying to understand why this is, but I can't find anything in the documentation.

Any ideas?

Virgo does not support refresh of PARs because there is not much benefit over and above redeploying the PAR. I suspect the PAR is being redeployed and the message is just warning that refresh didn't work as may have been expected.
Quote:


Also, I don't fully understand yet how the dependency management works.

So first I have to make sure the pom is OK in order to compile, test and package everything.
Then I also need to make sure my template.mf file is correct so the osgi framework can resolve all dependencies and this reflects my pom configuration. (template.mf is not autogenerated from the pom, so this is a manual process).

Then the manifest.mf gets generated from the template.mf using bundlor.

But then when I deploy to Virgo, I need to make sure that all dependencies are available on the server.

I would think that, since maven needs those bundles to compile and test, they would also be deployed to Virgo if the bundle is not yet available in the runtime.


The Virgo model is that dependencies are placed into a repository and are then available when artifacts are deployed which need those dependencies. This reduces the footprint by avoiding installing dependencies which are not used.
Quote:


I tried to deploy greenpages but that also get an 'unable to satisfy dependencies' because it is missing com.springsource.org.eclipse.persistence

I think it would make sense if all those dependencies would get packaged in the par by mvn unless the scope of the dependency in the pom would be set 'provided'.


Packaging all the dependencies in a PAR would be a bit like packaging all the dependencies of a web app in WEB-INF/lib. This is the traditional Java EE approach which minimises sharing and the ability to update pieces of an application. The Virgo model is to keep the dependencies separate.
Quote:


I hope I'll get something working soon, because I've been unproductive for way too long on this project struggling with these issues. I would be nice if I could get a working project set-up and test/deploy process in place soon.

All help is welcome!

Thanks!

Greetings,

Kristof


Hope that's some help. There is also a SpringSource forum for dm Server tooling which you may find helpful as it covers Virgo tooling too until that is donated to Eclipse.
Re: STS and PAR deployment [message #549584 is a reply to message #547526] Tue, 27 July 2010 09:34 Go to previous messageGo to next message
Ric Klaren is currently offline Ric Klaren
Messages: 13
Registered: July 2010
Junior Member
Kristof Taveirne wrote on Wed, 14 July 2010 14:58
Hi,
The problem however is not the PAR... I understand how it more or less works. Everything works when I use command line tools (mvn package, cp to deploy to the hotdeploy folder, ...)


Command line is usually not an issue with spring dm server and virgo too I guess. For the tooling you have a load of 'dances' and hoops to jump through.

Quote:
- run unit test on single bundles
- run integration tests across multiple bundles


In container testing is a bit of a problem you probably have to build your own things around the test stuff in spring dynamic modules. It's a bit of a maintenance pain to synchronize versions of bundles between test cases and your project (at least we found that to be a pain)

Quote:
- deploy the par to Virgo from within STS
- update a single bundle within the par without updating the complete application (par)


Advice: don't try to deploy the par from within STS (and also using a plan will not work properly, refreshing one bundle will redeploy your whole plan at least that's what we saw in our project).

Get used to dragging all your open projects into the server in the right deploy order (and place binary dependencies in in the repository of your dm server). (You will come to loathe this procedure, I certainly do, there's no easy way to tell the tooling this is the startup order and always use that)

Usual setup: have a par project in your workspace connected to the right runtime, configure the runtime in all your bundles, add all your bundles to the par project. (Make scripts for that if you have many bundles) Drag (or use the add wizard on the server, that's more reliable the dragging hardly ever works)

Quote:
I'm trying to understand why this is, but I can't find anything in the documentation.
Any ideas?


See above.

Quote:
Also, I don't fully understand yet how the dependency management works.

So first I have to make sure the pom is OK in order to compile, test and package everything.
Then I also need to make sure my template.mf file is correct so the osgi framework can resolve all dependencies and this reflects my pom configuration. (template.mf is not autogenerated from the pom, so this is a manual process).

Then the manifest.mf gets generated from the template.mf using bundlor.

But then when I deploy to Virgo, I need to make sure that all dependencies are available on the server.

I would think that, since maven needs those bundles to compile and test, they would also be deployed to Virgo if the bundle is not yet available in the runtime.


There's no direct interaction between maven build and STS. What we do is that we manage our runtime (e.g. spring dm server with all it's binary dependencies) from maven. We made maven the boss of all version information.

E.g. we have a target that will unpack the right spring dm server zip, modify the repository configuration of this dm server, then will fill the repository/usr directory with the runtime dependencies from the maven build. Then inside eclipse we use this maven installed runtime as target runtime. During development you want to keep this as stable as possible bundle repository rescans in STS are quite expensive and depending on your hardware you get a coffee break.

Generating your manifests via maven is a good way to go I think, inside the tooling you'll end up hand editing generated manifests to fix all kinds of bundlor/sts weirdness.

Actually at the moment I would not use bundlor anymore (e.g. use bnd) you can't put comments in the template.mf, so when you end up merging between branches you'll soon run into problems figuring out why some import is in the template.mf... (you may have less problems with that if you use the Spring DM specific headers for importing bundles) We don't use them since they feel too much like require bundle which is inherently evil (and we didn't want to tie ourselves to Spring DM Server).

Switching between maven build artifacts and having a project open in STS seems not to be the recommended way (you'll see weird things, especially generating manifests inside STS, depending on having a binary artifact or having the project open you may see different imports or sometimes bundlor completely loses it). It can be done but expect glitches. Don't forget to clean your server if you switch things between open in eclipse or binary artifact in a repository. (and don't forget the dances to delete the stage and work directory contents if you start to suspect that things are not up to date)

For test dependencies you can use a User library in eclipse (and always delete the TEST.MF's generated by bundlor (not an issue if you generate via maven)) Or use test bundles and a import via TEST.MF (you need the most recent STS for this to work somewhat reliably, This does not work well up to and including to STS 2.3.2. (You can break your classpath container if you import different packages from a bundle in manifest and test.mf, work around it with a fake import in manifest.mf or hand edit test.mf everytime, note that the TEST.MF is not generated by the maven bundlor plugin)

We have used both approaches, the User library is a pain to maintain. The test.mf stuff is not perfect. Choose your poison Smile

I hope I didn't scare you with this Smile Background: we've been using Spring DM Server since it was released, we also have it in production, our project is now 500K LoC, about 66 bundles (of our own, many more dependencies). Things are smoother in eclipse if your project is smaller. We didn't find anything better for OSGi development yet, and I must say the tooling is a breeze to use compared to say half a year ago.

Hope this helps a bit to get started.

Ric
Re: STS and PAR deployment [message #590477 is a reply to message #546972] Thu, 15 July 2010 14:52 Go to previous messageGo to next message
Glyn Normington is currently offline Glyn Normington
Messages: 1222
Registered: July 2009
Senior Member
Please ensure you install the latest dm Server tools (which include the Virgo tools) as described in http://blog.springsource.com/2010/06/17/first-virgo-mileston e-ships/

Until the tools are donated to Eclipse, you can use the http://forum.springsource.org/forumdisplay.php?f=56 to discuss the tooling.
Re: STS and PAR deployment [message #590573 is a reply to message #547093] Sun, 18 July 2010 10:34 Go to previous messageGo to next message
Kristof Taveirne is currently offline Kristof Taveirne
Messages: 6
Registered: July 2010
Junior Member
Thank you for your response!

I think I am a few steps further now with the new STS and Virgo plugins.

however, each time I rebuild my application the Virgo console gives me the following message:


[2010-07-18 11:19:24.950] TCP Connection(53)-127.0.0.1 <DE0050W> Cannot refresh par 'myproject' version '0.0.1' as refresh of par artifacts is not supported.

I'm trying to understand why this is, but I can't find anything in the documentation.

Any ideas?

Also, I don't fully understand yet how the dependency management works.

So first I have to make sure the pom is OK in order to compile, test and package everything.
Then I also need to make sure my template.mf file is correct so the osgi framework can resolve all dependencies and this reflects my pom configuration. (template.mf is not autogenerated from the pom, so this is a manual process).

Then the manifest.mf gets generated from the template.mf using bundlor.

But then when I deploy to Virgo, I need to make sure that all dependencies are available on the server.

I would think that, since maven needs those bundles to compile and test, they would also be deployed to Virgo if the bundle is not yet available in the runtime.

I tried to deploy greenpages but that also get an 'unable to satisfy dependencies' because it is missing com.springsource.org.eclipse.persistence

I think it would make sense if all those dependencies would get packaged in the par by mvn unless the scope of the dependency in the pom would be set 'provided'.

I hope I'll get something working soon, because I've been unproductive for way too long on this project struggling with these issues. I would be nice if I could get a working project set-up and test/deploy process in place soon.

All help is welcome!

Thanks!

Greetings,

Kristof
Re: STS and PAR deployment [message #591356 is a reply to message #590573] Tue, 20 July 2010 09:58 Go to previous messageGo to next message
Glyn Normington is currently offline Glyn Normington
Messages: 1222
Registered: July 2009
Senior Member
Kristof Taveirne wrote on Sun, 18 July 2010 11:34
> Thank you for your response!
>
> I think I am a few steps further now with the new STS and Virgo plugins.
>
> however, each time I rebuild my application the Virgo console gives me the following message:
>
>
> [2010-07-18 11:19:24.950] TCP Connection(53)-127.0.0.1 <DE0050W> Cannot refresh par 'myproject' version '0.0.1' as refresh of par artifacts is not supported.
>
> I'm trying to understand why this is, but I can't find anything in the documentation.
>
> Any ideas?

Virgo does not support refresh of PARs because there is not much benefit over and above redeploying the PAR. I suspect the PAR is being redeployed and the message is just warning that refresh didn't work as may have been expected.
Quote:
> Also, I don't fully understand yet how the dependency management works.
>
> So first I have to make sure the pom is OK in order to compile, test and package everything.
> Then I also need to make sure my template.mf file is correct so the osgi framework can resolve all dependencies and this reflects my pom configuration. (template.mf is not autogenerated from the pom, so this is a manual process).
>
> Then the manifest.mf gets generated from the template.mf using bundlor.
>
> But then when I deploy to Virgo, I need to make sure that all dependencies are available on the server.
>
> I would think that, since maven needs those bundles to compile and test, they would also be deployed to Virgo if the bundle is not yet available in the runtime.

The Virgo model is that dependencies are placed into a repository and are then available when artifacts are deployed which need those dependencies. This reduces the footprint by avoiding installing dependencies which are not used.
Quote:
> I tried to deploy greenpages but that also get an 'unable to satisfy dependencies' because it is missing com.springsource.org.eclipse.persistence
>
> I think it would make sense if all those dependencies would get packaged in the par by mvn unless the scope of the dependency in the pom would be set 'provided'.

Packaging all the dependencies in a PAR would be a bit like packaging all the dependencies of a web app in WEB-INF/lib. This is the traditional Java EE approach which minimises sharing and the ability to update pieces of an application. The Virgo model is to keep the dependencies separate.
Quote:
> I hope I'll get something working soon, because I've been unproductive for way too long on this project struggling with these issues. I would be nice if I could get a working project set-up and test/deploy process in place soon.
>
> All help is welcome!
>
> Thanks!
>
> Greetings,
>
> Kristof


Hope that's some help. There is also a SpringSource forum for http://forum.springsource.org/forumdisplay.php?f=56 which you may find helpful as it covers Virgo tooling too until that is donated to Eclipse.
Re: STS and PAR deployment [message #592799 is a reply to message #590573] Tue, 27 July 2010 09:34 Go to previous message
Ric Klaren is currently offline Ric Klaren
Messages: 13
Registered: July 2010
Junior Member
Kristof Taveirne wrote on Wed, 14 July 2010 14:58
> Hi,
> The problem however is not the PAR... I understand how it more or less works. Everything works when I use command line tools (mvn package, cp to deploy to the hotdeploy folder, ...)


Command line is usually not an issue with spring dm server and virgo too I guess. For the tooling you have a load of 'dances' and hoops to jump through.

Quote:
> - run unit test on single bundles
> - run integration tests across multiple bundles


In container testing is a bit of a problem you probably have to build your own things around the test stuff in spring dynamic modules. It's a bit of a maintenance pain to synchronize versions of bundles between test cases and your project (at least we found that to be a pain)

Quote:
> - deploy the par to Virgo from within STS
> - update a single bundle within the par without updating the complete application (par)


Advice: don't try to deploy the par from within STS (and also using a plan will not work properly, refreshing one bundle will redeploy your whole plan at least that's what we saw in our project).

Get used to dragging all your open projects into the server in the right deploy order (and place binary dependencies in in the repository of your dm server). (You will come to loathe this procedure, I certainly do, there's no easy way to tell the tooling this is the startup order and always use that)

Usual setup: have a par project in your workspace connected to the right runtime, configure the runtime in all your bundles, add all your bundles to the par project. (Make scripts for that if you have many bundles) Drag (or use the add wizard on the server, that's more reliable the dragging hardly ever works)

Quote:
> I'm trying to understand why this is, but I can't find anything in the documentation.
> Any ideas?


See above.

Quote:
> Also, I don't fully understand yet how the dependency management works.
>
> So first I have to make sure the pom is OK in order to compile, test and package everything.
> Then I also need to make sure my template.mf file is correct so the osgi framework can resolve all dependencies and this reflects my pom configuration. (template.mf is not autogenerated from the pom, so this is a manual process).
>
> Then the manifest.mf gets generated from the template.mf using bundlor.
>
> But then when I deploy to Virgo, I need to make sure that all dependencies are available on the server.
>
> I would think that, since maven needs those bundles to compile and test, they would also be deployed to Virgo if the bundle is not yet available in the runtime.


There's no direct interaction between maven build and STS. What we do is that we manage our runtime (e.g. spring dm server with all it's binary dependencies) from maven. We made maven the boss of all version information.

E.g. we have a target that will unpack the right spring dm server zip, modify the repository configuration of this dm server, then will fill the repository/usr directory with the runtime dependencies from the maven build. Then inside eclipse we use this maven installed runtime as target runtime. During development you want to keep this as stable as possible bundle repository rescans in STS are quite expensive and depending on your hardware you get a coffee break.

Generating your manifests via maven is a good way to go I think, inside the tooling you'll end up hand editing generated manifests to fix all kinds of bundlor/sts weirdness.

Actually at the moment I would not use bundlor anymore (e.g. use bnd) you can't put comments in the template.mf, so when you end up merging between branches you'll soon run into problems figuring out why some import is in the template.mf... (you may have less problems with that if you use the Spring DM specific headers for importing bundles) We don't use them since they feel too much like require bundle which is inherently evil (and we didn't want to tie ourselves to Spring DM Server).

Switching between maven build artifacts and having a project open in STS seems not to be the recommended way (you'll see weird things, especially generating manifests inside STS, depending on having a binary artifact or having the project open you may see different imports or sometimes bundlor completely loses it). It can be done but expect glitches. Don't forget to clean your server if you switch things between open in eclipse or binary artifact in a repository. (and don't forget the dances to delete the stage and work directory contents if you start to suspect that things are not up to date)

For test dependencies you can use a User library in eclipse (and always delete the TEST.MF's generated by bundlor (not an issue if you generate via maven)) Or use test bundles and a import via TEST.MF (you need the most recent STS for this to work somewhat reliably, This does not work well up to and including to STS 2.3.2. (You can break your classpath container if you import different packages from a bundle in manifest and test.mf, work around it with a fake import in manifest.mf or hand edit test.mf everytime, note that the TEST.MF is not generated by the maven bundlor plugin)

We have used both approaches, the User library is a pain to maintain. The test.mf stuff is not perfect. Choose your poison :)

I hope I didn't scare you with this :) Background: we've been using Spring DM Server since it was released, we also have it in production, our project is now 500K LoC, about 66 bundles (of our own, many more dependencies). Things are smoother in eclipse if your project is smaller. We didn't find anything better for OSGi development yet, and I must say the tooling is a breeze to use compared to say half a year ago.

Hope this helps a bit to get started.

Ric
Previous Topic:Expected timeframe of 'real' Virgo release
Next Topic:Uses constraint when adding org.springframework.spring
Goto Forum:
  


Current Time: Sat Oct 25 00:56:10 GMT 2014

Powered by FUDForum. Page generated in 0.02125 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software