Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Buckminster » Kepler Resolve & Materialize Performance-Issue(Kepler Resolve & Materialize Performance-Issue)
Kepler Resolve & Materialize Performance-Issue [message #1235013] Thu, 23 January 2014 05:53 Go to next message
Adrian Gaberell is currently offline Adrian Gaberell
Messages: 9
Registered: July 2011
Junior Member
hi,

when we update our environment from juno (eclipse project sdk 4.2.2/buckminster 1.5.0) to kepler (eclipse project sdk 4.3.1/buckminster 1.6.0)
our resolve & materialize process will take much longer than before.
juno: 1 minute 12 seconds
kepler: 8 minute 46 seconds

this resolve job contacts 300 p2 repositories via http.

does anyone notice similar behaviour?

thanks, adrian

[Updated on: Thu, 23 January 2014 06:35]

Report message to a moderator

Re: Kepler Resolve & Materialize Performace-Issue [message #1236516 is a reply to message #1235013] Mon, 27 January 2014 06:57 Go to previous messageGo to next message
Lorenzo Bettini is currently offline Lorenzo Bettini
Messages: 1307
Registered: July 2009
Senior Member
On 23/01/2014 11:53, Adrian Gaberell wrote:
> hi,
>
> when we update our environment from juno (eclipse project sdk
> 4.2.2/buckminster 1.5.0) to kepler (eclipse project sdk
> 4.3.1/buckminster 1.6.0)
> our resolve & materialize process will take much longer than before.
> juno: 1 minute 12 seconds
> kepler: 8 minute 46 seconds
>
> this resolve job contacts 300 p2 repositories via http.
>
> does anyone notice similar behaviour?
>
> thanks, adrian

This is only a guess: probably the kepler update site aggregates much
more features than juno...

cheers
Lorenzo

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Xtext Book:
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book
Re: Kepler Resolve & Materialize Performace-Issue [message #1239044 is a reply to message #1236516] Mon, 03 February 2014 02:40 Go to previous messageGo to next message
Adrian Gaberell is currently offline Adrian Gaberell
Messages: 9
Registered: July 2011
Junior Member
hi lorenzo,

thanks for your help. but unfortunately this can't be the problem because there is no juno or kepler p2-repository involved.
in those p2 repositories are only our "own" artifacts stored and for both "resolve and materialize" actions, exactly the same p2 repositories are contacted and the same artifacts are downloaded.

cheers, adrian
Re: Kepler Resolve & Materialize Performace-Issue [message #1246015 is a reply to message #1239044] Fri, 14 February 2014 10:54 Go to previous messageGo to next message
Adrian Gaberell is currently offline Adrian Gaberell
Messages: 9
Registered: July 2011
Junior Member
dear bucky development team,

did anyone notice similar behavior?
first I had the suspicion, that the change in the ecf-project from org.apche.commons.httpclient to org.apache.httpcomponents is responsible for this behavior,
but then I made some measurements inside the HttpClientRetrieveFileTransfer class of the ecf-framework and there was no difference between Juno and Kepler.
so I guess the leak must be inside buckminster where the information of the downloaded content.jar from a p2-repository is analyzed and processed...

regards, adrian

[Updated on: Fri, 14 February 2014 10:55]

Report message to a moderator

Re: Kepler Resolve & Materialize Performace-Issue [message #1248777 is a reply to message #1246015] Mon, 17 February 2014 11:51 Go to previous messageGo to next message
Lorenzo Bettini is currently offline Lorenzo Bettini
Messages: 1307
Registered: July 2009
Senior Member
On 14/02/2014 16:54, Adrian Gaberell wrote:
> dear bucky development team,
>
> did anyone notice similar behavior?
> first I had the suspicion, that the change in the ecf-project from
> org.apche.commons.httpclient to org.apache.httpcomponents is responsible
> for this behavior,
> but then I made some measurements inside the
> HttpClientRetrieveFileTransfer class of the ecf-framework and there was
> no difference between Juno and Kepler.
> so I guess the leek must be inside buckminster where the information of
> the downloaded content.jar from a p2-repository is analyzed and
> processed...

Hi

I did not notice this behavior... can you share your cquery/rmap?


--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Xtext Book:
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book
Re: Kepler Resolve & Materialize Performace-Issue [message #1259546 is a reply to message #1248777] Fri, 28 February 2014 09:12 Go to previous messageGo to next message
Adrian Gaberell is currently offline Adrian Gaberell
Messages: 9
Registered: July 2011
Junior Member
hi,

i think it isn't enough to just share the cquery/rmap... therefore i made an example cquery with an example p2-repository.
to test this example, just install the p2 repository in apache and run the cqery against it.
for repeatable tests, i set up an local apache in a vm, set the maximum number of resolver threads to 1 and the network settings to direct.
before every testrun i cleaned the .metadata and .buckminster directory in the workspace because of caching stuff.

the example is simple: there is a bundle named com.test.resolve, this bundle has a dependency to bundle com.test.bundle9, this bundle is located in p2 repository com.test.feature9.
the bundle com.test.bundle9 has a dependency to bundle com.test.bundle8 witch is located in p2 repository com.test.feature8... and so on until bundle1 in feature1.

test results (seconds):
juno
resolve to wizard:
test1: 2,3
test2: 1,7

resolve and materialize:
test1: 10,3
test2: 10,5

kepler:
resolve to wizard:
test1: 5,8
test2: 5,8

resolve and materialize:
test1: 26,5
test2: 26,4

regards, adrian
Re: Kepler Resolve & Materialize Performace-Issue [message #1268598 is a reply to message #1259546] Mon, 10 March 2014 14:17 Go to previous messageGo to next message
Lorenzo Bettini is currently offline Lorenzo Bettini
Messages: 1307
Registered: July 2009
Senior Member
Hi Adrian

I cannot test it right now, but I see that your cquery contains

useMaterialization="false"
useRemoteResolution="false"
useTargetPlatform="false"

this might cause it to check the repositories each time over and over...
could you please try to remove them and see if something changes? If I
understand correctly the meaning of those flags from the documentation...

cheers
Lorenzo

On 28/02/2014 15:12, Adrian Gaberell wrote:
> hi,
>
> i think it isn't enough to just share the cquery/rmap... therefore i made an example cquery with an example p2-repository.
> to test this example, just install the p2 repository in apache and run the cqery against it.
> for repeatable tests, i set up an local apache in a vm, set the maximum number of resolver threads to 1 and the network settings to direct.
> before every testrun i cleaned the .metadata and .buckminster directory in the workspace because of caching stuff.
>
> the example is simple: there is a bundle named com.test.resolve, this bundle has a dependency to bundle com.test.bundle9, this bundle is located in p2 repository com.test.feature9.
> the bundle com.test.bundle9 has a dependency to bundle com.test.bundle8 witch is located in p2 repository com.test.feature8... and so on until bundle1 in feature1.
>
> test results (seconds):
> juno
> resolve to wizard:
> test1: 2,3
> test2: 1,7
>
> resolve and materialize:
> test1: 10,3
> test2: 10,5
>
> kepler:
> resolve to wizard:
> test1: 5,8
> test2: 5,8
>
> resolve and materialize:
> test1: 26,5
> test2: 26,4
>
> regards, adrian
>


--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Xtext Book:
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book
Re: Kepler Resolve & Materialize Performace-Issue [message #1269175 is a reply to message #1268598] Tue, 11 March 2014 09:35 Go to previous messageGo to next message
Adrian Gaberell is currently offline Adrian Gaberell
Messages: 9
Registered: July 2011
Junior Member
hi lorenzo,

those flags are only used for the bundle witch is desired as source:
<cq:advisorNode namePattern="com.test.resolve" mutableLevel="REQUIRE" sourceLevel="REQUIRE" useMaterialization="false" useRemoteResolution="false" useTargetPlatform="false"/>

not for the ones desired as binary (com.test.(bundle|feature):
<cq:advisorNode namePattern=".*" mutableLevel="REJECT" sourceLevel="REJECT" useWorkspace="false"/>

I have chosen this pattern, because I don't want to install a source-repository for this simple sample where I had to expand the rmap with unnecessary stuff.

the problem is, if the resolve & materialize task need to download the information from a p2 repository I takes much longer with the new version.
if you have a product with a lot of different features and plugins, this will happens every time you run resolve & materialize, because the bundles in the p2 repository may change with every nightly-build.

regards, adrian

Re: Kepler Resolve &amp;amp;amp;amp; Materialize Performace-Issue [message #1281708 is a reply to message #1269175] Tue, 01 April 2014 04:38 Go to previous message
Lorenzo Bettini is currently offline Lorenzo Bettini
Messages: 1307
Registered: July 2009
Senior Member
On 11/03/2014 14:35, Adrian Gaberell wrote:
> hi lorenzo,
>
> those flags are only used for the bundle witch is desired as source:
> <cq:advisorNode namePattern="com.test.resolve" mutableLevel="REQUIRE"
> sourceLevel="REQUIRE" useMaterialization="false"
> useRemoteResolution="false" useTargetPlatform="false"/>
> not for the ones desired as binary (com.test.(bundle|feature):
> <cq:advisorNode namePattern=".*" mutableLevel="REJECT"
> sourceLevel="REJECT" useWorkspace="false"/>
> I have chosen this pattern, because I don't want to install a
> source-repository for this simple sample where I had to expand the rmap
> with unnecessary stuff.
>
> the problem is, if the resolve & materialize task need to download the
> information from a p2 repository I takes much longer with the new version.
> if you have a product with a lot of different features and plugins, this
> will happens every time you run resolve & materialize, because the
> bundles in the p2 repository may change with every nightly-build.
>
> regards, adrian

Hi

in this case I wouldn't know... I guess only Thomas can say something
useful :)

cheers
Lorenzo


--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Xtext Book:
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book
Previous Topic:Re: [buckminster-dev] Buckminster for Luna
Next Topic:Suppressing component request conflict warning
Goto Forum:
  


Current Time: Wed Jul 30 23:28:22 EDT 2014

Powered by FUDForum. Page generated in 0.08203 seconds