Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Buckminster » problems when running emma in Jenkins
problems when running emma in Jenkins [message #1059464] Sun, 19 May 2013 09:05 Go to next message
Lorenzo Bettini is currently offline Lorenzo Bettini
Messages: 1355
Registered: July 2009
Senior Member
Hi

I've just started using emma in my buckminster builds in Jenkins.

First of all, I run emma as follows

emma -l "mylaunch.launch" --output
"${buckminster.output.root}/test.results/mylaunch.launch.xml" --xml
"${buckminster.output.root}/test.results/coverage.mylaunch.launch.xml"
--flatXML --stdout --stderr

and I run buckminster from ant (I tested this both with Buckminster
headless 4.2 and 4.3).

When I run the ant script locally on my machine, everything works file
and coverage.mylaunch.launch.xml is created.

When I run the same ant script in Jenkins, I get this error when the
junit test suite ends

[java] INFO: Starting test session TraitJTypingUtilTest
[java] INFO: Running test...
....
[java] INFO: ...OK [0.201s]
[java] INFO: Tests finished.
[java] INFO: Elapsed time: 1.556 seconds.
[java] INFO: Total number of tests executed: 2
[java] INFO: Successful tests: 2
[java] INFO: Failed tests: 0
[java] INFO: Errors: 0
[java] INFO: Ignored tests: 0
[java] INFO: Overall status: OK
[java] ERROR: No coverage has been collected during coverage launch
(code 5101).

the strange thing is that most of the time the
coverage.mylaunch.launch.xml is actually created (in spite of the ERROR
above)...

when the job executes the postbuild action of the Emma plugin in Jenkins
"Record Emma coverage report", the coverage.mylaunch.launch.xml is
found, but I always get this error

hudson.util.IOException2: Failed to read
/usr/share/tomcat6/.jenkins/workspace/my.project/buildroot/buckminster.output/test.results/coverage.mylaunch.launch.xml
at hudson.tasks.junit.TestResult.parse(TestResult.java:284)
at hudson.tasks.junit.TestResult.parsePossiblyEmpty(TestResult.java:223)
....
Caused by: org.dom4j.DocumentException:
/usr/share/tomcat6/.jenkins/workspace/my.project/buildroot/buckminster.output/test.results/report.dtd
(No such file or directory) Nested exception:
/usr/share/tomcat6/.jenkins/workspace/xtext-traitj-gerrit-ant/buildroot/buckminster.output/test.results/report.dtd
(No such file or directory)
....

which seems to imply that it's trying to read the report.dtd...

anyone else experiencing this?

thanks in advance
Lorenzo

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Re: problems when running emma in Jenkins [message #1059840 is a reply to message #1059464] Tue, 21 May 2013 20:19 Go to previous messageGo to next message
Thomas Hallgren is currently offline Thomas Hallgren
Messages: 3229
Registered: July 2009
Senior Member
Hi Lorenzo,

There's been lots of changes in this area when we switched to a more
recent version of Emma. Your case is fairly special though, and I don't
know how to help. If you do further findings, and/or if you can attach
something that can be easily reproduced to a bug report, then I'll look
into it.

- thomas

On 2013-05-19 11:05, Lorenzo Bettini wrote:
> Hi
>
> I've just started using emma in my buckminster builds in Jenkins.
>
> First of all, I run emma as follows
>
> emma -l "mylaunch.launch" --output
> "${buckminster.output.root}/test.results/mylaunch.launch.xml" --xml
> "${buckminster.output.root}/test.results/coverage.mylaunch.launch.xml"
> --flatXML --stdout --stderr
>
> and I run buckminster from ant (I tested this both with Buckminster
> headless 4.2 and 4.3).
>
> When I run the ant script locally on my machine, everything works file
> and coverage.mylaunch.launch.xml is created.
>
> When I run the same ant script in Jenkins, I get this error when the
> junit test suite ends
>
> [java] INFO: Starting test session TraitJTypingUtilTest
> [java] INFO: Running test...
> ....
> [java] INFO: ...OK [0.201s]
> [java] INFO: Tests finished.
> [java] INFO: Elapsed time: 1.556 seconds.
> [java] INFO: Total number of tests executed: 2
> [java] INFO: Successful tests: 2
> [java] INFO: Failed tests: 0
> [java] INFO: Errors: 0
> [java] INFO: Ignored tests: 0
> [java] INFO: Overall status: OK
> [java] ERROR: No coverage has been collected during coverage launch
> (code 5101).
>
> the strange thing is that most of the time the
> coverage.mylaunch.launch.xml is actually created (in spite of the ERROR
> above)...
>
> when the job executes the postbuild action of the Emma plugin in Jenkins
> "Record Emma coverage report", the coverage.mylaunch.launch.xml is
> found, but I always get this error
>
> hudson.util.IOException2: Failed to read
> /usr/share/tomcat6/.jenkins/workspace/my.project/buildroot/buckminster.output/test.results/coverage.mylaunch.launch.xml
> at hudson.tasks.junit.TestResult.parse(TestResult.java:284)
> at hudson.tasks.junit.TestResult.parsePossiblyEmpty(TestResult.java:223)
> ....
> Caused by: org.dom4j.DocumentException:
> /usr/share/tomcat6/.jenkins/workspace/my.project/buildroot/buckminster.output/test.results/report.dtd
> (No such file or directory) Nested exception:
> /usr/share/tomcat6/.jenkins/workspace/xtext-traitj-gerrit-ant/buildroot/buckminster.output/test.results/report.dtd
> (No such file or directory)
> ....
>
> which seems to imply that it's trying to read the report.dtd...
>
> anyone else experiencing this?
>
> thanks in advance
> Lorenzo
>
Re: problems when running emma in Jenkins [message #1059850 is a reply to message #1059840] Tue, 21 May 2013 22:33 Go to previous messageGo to next message
Lorenzo Bettini is currently offline Lorenzo Bettini
Messages: 1355
Registered: July 2009
Senior Member
Hi Thomas

I made some progresses: first of all, I had not realized that EclEmma
(on which Buckminster relies for the 'emma', as far as I understand) is
now based on Jacoco, thus, in Jenkins, I need to use the Jacoco plugin,
and not the Emma plugin; the Jacoco plugin, in order to generate the
report, requires .exec files instead of .xml, thus I need to use the command

emma -l "mylaunch.launch" --output
"${buckminster.output.root}/test.results/mylaunch.launch.xml" --exec
"${buckminster.output.root}/test.results/coverage.mylaunch.launch.exec"

and then the Jacoco plugin can deal with the generated exec files :)

the problem which remains is the

[java] ERROR: No coverage has been collected during coverage launch
(code 5101).

as I said, the strange thing is that most of the time the
coverage.mylaunch.launch.exec is actually created (in spite of the ERROR
above)... sometimes though, the whole build fails due to this error...

I run the emma command twice in my build: the first command deals with
plain Junit tests, the second command deals with Plug-in Junit tests...
the second command does not report the above error... I'll try to
investigate further, possibly with other projects, and keep you posted.

Concerning the "report.dtd" problem it was my fault: I was trying to
archive the coverage.xml as a Junit report ;)

In any case, the .xml generated by jacoco is simply not compatible with
Jenkins Emma Plugin: when using the new EclEmma, you need to use Jenking
Jacoco Plugin anyway.

cheers
Lorenzo

On 05/21/2013 10:19 PM, Thomas Hallgren wrote:
> Hi Lorenzo,
>
> There's been lots of changes in this area when we switched to a more
> recent version of Emma. Your case is fairly special though, and I don't
> know how to help. If you do further findings, and/or if you can attach
> something that can be easily reproduced to a bug report, then I'll look
> into it.
>
> - thomas
>
> On 2013-05-19 11:05, Lorenzo Bettini wrote:
>> Hi
>>
>> I've just started using emma in my buckminster builds in Jenkins.
>>
>> First of all, I run emma as follows
>>
>> emma -l "mylaunch.launch" --output
>> "${buckminster.output.root}/test.results/mylaunch.launch.xml" --xml
>> "${buckminster.output.root}/test.results/coverage.mylaunch.launch.xml"
>> --flatXML --stdout --stderr
>>
>> and I run buckminster from ant (I tested this both with Buckminster
>> headless 4.2 and 4.3).
>>
>> When I run the ant script locally on my machine, everything works file
>> and coverage.mylaunch.launch.xml is created.
>>
>> When I run the same ant script in Jenkins, I get this error when the
>> junit test suite ends
>>
>> [java] INFO: Starting test session TraitJTypingUtilTest
>> [java] INFO: Running test...
>> ....
>> [java] INFO: ...OK [0.201s]
>> [java] INFO: Tests finished.
>> [java] INFO: Elapsed time: 1.556 seconds.
>> [java] INFO: Total number of tests executed: 2
>> [java] INFO: Successful tests: 2
>> [java] INFO: Failed tests: 0
>> [java] INFO: Errors: 0
>> [java] INFO: Ignored tests: 0
>> [java] INFO: Overall status: OK
>> [java] ERROR: No coverage has been collected during coverage launch
>> (code 5101).
>>
>> the strange thing is that most of the time the
>> coverage.mylaunch.launch.xml is actually created (in spite of the ERROR
>> above)...
>>
>> when the job executes the postbuild action of the Emma plugin in Jenkins
>> "Record Emma coverage report", the coverage.mylaunch.launch.xml is
>> found, but I always get this error
>>
>> hudson.util.IOException2: Failed to read
>> /usr/share/tomcat6/.jenkins/workspace/my.project/buildroot/buckminster.output/test.results/coverage.mylaunch.launch.xml
>>
>> at hudson.tasks.junit.TestResult.parse(TestResult.java:284)
>> at
>> hudson.tasks.junit.TestResult.parsePossiblyEmpty(TestResult.java:223)
>> ....
>> Caused by: org.dom4j.DocumentException:
>> /usr/share/tomcat6/.jenkins/workspace/my.project/buildroot/buckminster.output/test.results/report.dtd
>>
>> (No such file or directory) Nested exception:
>> /usr/share/tomcat6/.jenkins/workspace/xtext-traitj-gerrit-ant/buildroot/buckminster.output/test.results/report.dtd
>>
>> (No such file or directory)
>> ....
>>
>> which seems to imply that it's trying to read the report.dtd...
>>
>> anyone else experiencing this?
>>
>> thanks in advance
>> Lorenzo
>>
>


--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Re: problems when running emma in Jenkins [message #1060440 is a reply to message #1059850] Fri, 24 May 2013 16:07 Go to previous messageGo to next message
Lorenzo Bettini is currently offline Lorenzo Bettini
Messages: 1355
Registered: July 2009
Senior Member
Hi Thomas,

here's some other collected info

On 05/22/2013 12:33 AM, Lorenzo Bettini wrote:

> the problem which remains is the
>
> [java] ERROR: No coverage has been collected during coverage launch
> (code 5101).
>
> as I said, the strange thing is that most of the time the
> coverage.mylaunch.launch.exec is actually created (in spite of the ERROR
> above)... sometimes though, the whole build fails due to this error...
>

by looking at buckminster code, Emma.java, I see that in case there is
no coverage session, buckminster throws an exception with message "No
coverage sessions found", while the above error is issued by eclemma
itself, and the faq (http://www.eclemma.org/faq.html#trouble01) says

"This happens when the Java process did not terminate properly, e.g. has
been manually killed with the Terminate button. Make sure your program
terminates on its own. "

indeed, the buckminster's emma command terminates, in fact, by using
DEBUG I see

INFO: Tests finished.
INFO: Elapsed time: 69.02 seconds.
INFO: Total number of tests executed: 113
INFO: Successful tests: 113
INFO: Failed tests: 0
INFO: Errors: 0
INFO: Ignored tests: 0
INFO: Overall status: OK
ERROR: No coverage has been collected during coverage launch (code 5101).
Doing full workspace refresh
Waiting for jobs to end

thus, I assume the buckminster command terminates correctly, but
internally EclEmma issues that error because its Java process is
terminated before it has finished?

If I run plain junit test suite and the plug-in junit test suite, the
first suite reports this error

If I run plug-in junit test suite first and then plain junit test suite,
none of them reports this error

when the error is reported, the coverage.xml file is generated anyway
(most of time, when it's not generated, buckminster itself fails).

Can it be due to some synchronization issues when buckminster runs
eclemma and does not wait for it to terminate?

The above problem happens in a Jenkins server, which is almost 10 years
old, thus not very powerful...

shall I file a bug report?

cheers
Lorenzo

>
> On 05/21/2013 10:19 PM, Thomas Hallgren wrote:
>> Hi Lorenzo,
>>
>> There's been lots of changes in this area when we switched to a more
>> recent version of Emma. Your case is fairly special though, and I don't
>> know how to help. If you do further findings, and/or if you can attach
>> something that can be easily reproduced to a bug report, then I'll look
>> into it.
>>
>> - thomas
>>
>> On 2013-05-19 11:05, Lorenzo Bettini wrote:
>>> Hi
>>>
>>> I've just started using emma in my buckminster builds in Jenkins.
>>>
>>> First of all, I run emma as follows
>>>
>>> emma -l "mylaunch.launch" --output
>>> "${buckminster.output.root}/test.results/mylaunch.launch.xml" --xml
>>> "${buckminster.output.root}/test.results/coverage.mylaunch.launch.xml"
>>> --flatXML --stdout --stderr
>>>
>>> and I run buckminster from ant (I tested this both with Buckminster
>>> headless 4.2 and 4.3).
>>>
>>> When I run the ant script locally on my machine, everything works file
>>> and coverage.mylaunch.launch.xml is created.
>>>
>>> When I run the same ant script in Jenkins, I get this error when the
>>> junit test suite ends
>>>
>>> [java] INFO: Starting test session TraitJTypingUtilTest
>>> [java] INFO: Running test...
>>> ....
>>> [java] INFO: ...OK [0.201s]
>>> [java] INFO: Tests finished.
>>> [java] INFO: Elapsed time: 1.556 seconds.
>>> [java] INFO: Total number of tests executed: 2
>>> [java] INFO: Successful tests: 2
>>> [java] INFO: Failed tests: 0
>>> [java] INFO: Errors: 0
>>> [java] INFO: Ignored tests: 0
>>> [java] INFO: Overall status: OK
>>> [java] ERROR: No coverage has been collected during coverage launch
>>> (code 5101).
>>>
>>> the strange thing is that most of the time the
>>> coverage.mylaunch.launch.xml is actually created (in spite of the ERROR
>>> above)...
>>>
>>> when the job executes the postbuild action of the Emma plugin in Jenkins
>>> "Record Emma coverage report", the coverage.mylaunch.launch.xml is
>>> found, but I always get this error
>>>
>>> hudson.util.IOException2: Failed to read
>>> /usr/share/tomcat6/.jenkins/workspace/my.project/buildroot/buckminster.output/test.results/coverage.mylaunch.launch.xml
>>>
>>> at hudson.tasks.junit.TestResult.parse(TestResult.java:284)
>>> at
>>> hudson.tasks.junit.TestResult.parsePossiblyEmpty(TestResult.java:223)
>>> ....
>>> Caused by: org.dom4j.DocumentException:
>>> /usr/share/tomcat6/.jenkins/workspace/my.project/buildroot/buckminster.output/test.results/report.dtd
>>>
>>> (No such file or directory) Nested exception:
>>> /usr/share/tomcat6/.jenkins/workspace/xtext-traitj-gerrit-ant/buildroot/buckminster.output/test.results/report.dtd
>>>
>>> (No such file or directory)
>>> ....
>>>
>>> which seems to imply that it's trying to read the report.dtd...
>>>
>>> anyone else experiencing this?
>>>
>>> thanks in advance
>>> Lorenzo
>>>
>>
>
>


--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Re: problems when running emma in Jenkins [message #1060447 is a reply to message #1060440] Fri, 24 May 2013 16:59 Go to previous messageGo to next message
Thomas Hallgren is currently offline Thomas Hallgren
Messages: 3229
Registered: July 2009
Senior Member
On 2013-05-24 18:07, Lorenzo Bettini wrote:
>
> shall I file a bug report?
>
Yes please. Slow equipment is often a good indicator of less frequent
(and thus very hard to find) problems. Some kind of join point seems to
be needed here. Thanks for analyzing this.

- thomas
Re: problems when running emma in Jenkins [message #1060510 is a reply to message #1060447] Sat, 25 May 2013 17:37 Go to previous message
Lorenzo Bettini is currently offline Lorenzo Bettini
Messages: 1355
Registered: July 2009
Senior Member
On 05/24/2013 06:59 PM, Thomas Hallgren wrote:
> On 2013-05-24 18:07, Lorenzo Bettini wrote:
>>
>> shall I file a bug report?
>>
> Yes please. Slow equipment is often a good indicator of less frequent
> (and thus very hard to find) problems. Some kind of join point seems to
> be needed here. Thanks for analyzing this.

Done: https://bugs.eclipse.org/bugs/show_bug.cgi?id=409068

thanks
Lorenzo


--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Previous Topic:ERROR [0002] : Artifact is a folder but the repository is an archive or remote location.
Next Topic:Avoid full workspace refresh during JUnit tests
Goto Forum:
  


Current Time: Sat Sep 20 03:51:47 GMT 2014

Powered by FUDForum. Page generated in 0.06370 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software