Eclipse Community Forums: Buckminster » Hudson/Buckminster: Separate Build and Test Job with Emma

Help

Home

Home » Archived » Buckminster » Hudson/Buckminster: Separate Build and Test Job with Emma

Show: Today's Messages :: Show Polls :: Message Navigator

Hudson/Buckminster: Separate Build and Test Job with Emma [message #630968]

Tue, 05 October 2010 16:02

Marcel Hoetter

Messages: 28
Registered: July 2009

Junior Member

Hi,

I know that this is a pure Buckminster newsgroup. My question is more buckminster related, though
and I guess that a lot of people here have Hudson knowledge, too, so:

I have a Hudson/Buckminster setup with separated Jobs for building and testing my projects.
Right now, i do the following:

-build and archive the projects (not the generated plug-ins) in the build job
-fetch the zips into my test job workpace via the URLSCM plug-in in
-extract the zips
-i refer to the unzip-folder in my rmap via the "local" reader type
-build the tests (which makes use of the unzipped projects from the build-job)
-run the tests via emma

This works pretty well, although i have the following problem with this approach:
the projects fetched from the build job are rebuild in the test job again.

My original intention was to use the projects/plug-ins from the build-job and test and compile
coverage for it in the test job without rebuilding them again.

The problem is that emma will not instrument libraries (jars) that i get when using the p2 or
eclipse.import reader.

Is there a better approach to "reuse" the build plug-ins when emma is to be used?
Is there a way to prevent the rebuild?

Thanks,

Marcel

Report message to a moderator

Re: Hudson/Buckminster: Separate Build and Test Job with Emma [message #631103 is a reply to message #630968]

Wed, 06 October 2010 08:20

Johannes Utzig

Messages: 329
Registered: July 2009

Senior Member

Hi Marcel,

your setup sounds rather complicated to me.
Why do you want to build first and then test instead of the other way
around?
If this is about integration tests and you want to test the final
product, then code coverage is not quite as useful as a metric anyway as
it would be for actual unit tests.

Best regards,
Johannes

Report message to a moderator

Re: Hudson/Buckminster: Separate Build and Test Job with Emma [message #631130 is a reply to message #631103]

Wed, 06 October 2010 09:41

Marcel Hoetter

Messages: 28
Registered: July 2009

Junior Member

Hi Johannes,

separating a build in multiple smaller build and test jobs is a pretty common setup. See:

http://wiki.eclipse.org/Teneo/Teneo_Build_Setup#Understandin g_Buckminster
http://wiki.hudson-ci.org/display/HUDSON/Splitting+a+big+job +into+smaller+jobs

In our project we manage a hand full of subsystems. Each of them has to be unit/component tested to
certain coverage, then integrated and integration-tested.

I am talking about the unit/component tests here. Reusing the build-artifacts is straightforward in
a simple JUnit test approach:

-Create a p2 update site in the build job.
-materialize the plug-ins from the p2 site in the test job.
-build & run Junit tests

With emma, though, the to-be-tested plug-ins have to be available as projects in the buckminster
workspace. And, as far as i know, the sources have to be there, too. At least, in the Eclipse IDE,
one cannot select packages to generate coverage for if the sources are not present. And Emma wont
compile coverage for binary-jars in an OSGi environment (class files have to be instrumented in place).

My question is, does anyone know about a clean and simple approach to reuse the build artifacts of
the build job without the need for a rebuild in the Test Job?

Mybe by using the eclipse.import reader to import the project with sources into the workspace?

Thanks,

Marcel

Am 06.10.2010 10:20, schrieb Johannes Utzig:
> Hi Marcel,
>
> your setup sounds rather complicated to me.
> Why do you want to build first and then test instead of the other way around?
> If this is about integration tests and you want to test the final product, then code coverage is not
> quite as useful as a metric anyway as it would be for actual unit tests.
>
> Best regards,
> Johannes
>

Report message to a moderator

Re: Hudson/Buckminster: Separate Build and Test Job with Emma [message #631131 is a reply to message #631130]

Wed, 06 October 2010 09:48

Johannes Utzig

Messages: 329
Registered: July 2009

Senior Member

Am 06.10.2010 11:41, schrieb Marcel Hoetter:
> Hi Johannes,
>
> separating a build in multiple smaller build and test jobs is a pretty
> common setup. See:
>
> http://wiki.eclipse.org/Teneo/Teneo_Build_Setup#Understandin g_Buckminster
> http://wiki.hudson-ci.org/display/HUDSON/Splitting+a+big+job +into+smaller+jobs
>

I agree that it makes sense to split a job in several subtasks, but I
would say the other way is the easier setup since it gives you out of
the box what you're looking for:
1. Job - run unit tests + code coverage without building anything
2. Job - Build the artifacts
3. Job (optionally) - use the artifacts of 2. to do integration testing
(code coverage is probably less important here)

Best regards,
Johannes

Report message to a moderator

Re: Hudson/Buckminster: Separate Build and Test Job with Emma [message #631142 is a reply to message #631131]

Wed, 06 October 2010 10:29

Marcel Hoetter

Messages: 28
Registered: July 2009

Junior Member

Hi, thanks for your answer.

The test-objects (that is, the to-be-tested plug-ins) have to be present (build) before the tests
can be build and run.
This can be done explicitly in the same or an upstream job; or it is done implicitly when the tests
themselves are build (which, of course, depend on the test-objects).

Cheers,

Marcel

Am 06.10.2010 11:48, schrieb Johannes Utzig:
> Am 06.10.2010 11:41, schrieb Marcel Hoetter:
>> Hi Johannes,
>>
>> separating a build in multiple smaller build and test jobs is a pretty
>> common setup. See:
>>
>> http://wiki.eclipse.org/Teneo/Teneo_Build_Setup#Understandin g_Buckminster
>> http://wiki.hudson-ci.org/display/HUDSON/Splitting+a+big+job +into+smaller+jobs
>>
>
> I agree that it makes sense to split a job in several subtasks, but I would say the other way is the
> easier setup since it gives you out of the box what you're looking for:
> 1. Job - run unit tests + code coverage without building anything
> 2. Job - Build the artifacts
> 3. Job (optionally) - use the artifacts of 2. to do integration testing (code coverage is probably
> less important here)
>
> Best regards,
> Johannes

Report message to a moderator

Re: Hudson/Buckminster: Separate Build and Test Job with Emma [message #631152 is a reply to message #631142]

Wed, 06 October 2010 11:05

Johannes Utzig

Messages: 329
Registered: July 2009

Senior Member

Hi Marcel,

Am 06.10.2010 12:29, schrieb Marcel Hoetter:
>
> The test-objects (that is, the to-be-tested plug-ins) have to be present
> (build) before the tests can be build and run.
> This can be done explicitly in the same or an upstream job; or it is
> done implicitly when the tests themselves are build (which, of course,
> depend on the test-objects).

I don't know, maybe I am simply misunderstand you, but:
to execute unit tests with buckminster you do not have to build the
actual plugins. Unit testing with Buckminster works exactly the same way
you unit test in your IDE. You start a unit test launch config just like
you do it in the IDE and that's it. That works for both 'plain' junit
tests and 'plug-in unit tests', the launch configuration decides.

That is the way you usually use unit tests with buckminster and it
makes it very easy to integrate code coverage. If however you still want
to test on binary bundles instead and compute code coverage for these
binaries, then I fear you will have to do the instrumentation by hand
(maybe somebody knows a better way).

Best regards,
Johannes

Report message to a moderator

Re: Hudson/Buckminster: Separate Build and Test Job with Emma [message #632051 is a reply to message #631152]

Mon, 11 October 2010 11:16

Daniel Weber

Messages: 51
Registered: July 2009

Member

Hi there,

if you run "Emma-enabled" tests and then create the plugin artifacts
(jars/p2 sites), won't they contain the Emma instrumentation?

I also created a two job setup:

- Job 1 (triggered by SCM changes):
resolve, generate & build, test

- Job 2 (downstream project of Job 1, uses the same workspace):
clean (to get rid of Emma instrumentation), build site.p2 & product zip and publish them

This way, I can easily tell Hudson to *not* create the update site when
one of the tests fails (which makes Job 1 unstable). I did not find a
way to do this in a single job.

Regards,
Daniel

Report message to a moderator

Previous Topic:	Tutorial: Provisioning your Target Platform as local p2 site
Next Topic:	Problems getting create.product to work

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Thu Apr 18 04:44:26 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter