Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Mylyn » (no subject)
(no subject) [message #715780] Mon, 15 August 2011 14:30 Go to next message
Carsten Reckord is currently offline Carsten Reckord
Messages: 139
Registered: July 2009
Senior Member
Hi all,

Does anybody here have a working build setup with Hudson/Jenkins and
Buckminster, in which the JUnit test reports work with the Mylyn Build
Connector for Hudson?

Our problem with our current setup using Mylyn 3.6.1 is that
- failed tests in the build's "Test Results" section cannot be opened(Failed
to locate test in workspace)
- opening the report in the JUnit view works, but again failed tests are not
openable (Test class not found in selected project)

The reason for this is the format of our test reports, which contains nested
test classnames concatenated by '$' which Mylyn doesn't seem to understand.


Some background:

We are running Buckminster builds of Eclipse projects using the Hudson
Buckminster plug-ins. Those builds contain tests that are run through
Buckminster's junit action that triggers standard Eclipse JUnit runs
configured through a launch config.

This works great, mostly. However, there are some differences between
Eclipse's JUnit XML report and the format understood by Hudson/Jenkins,
especially concerning nested test suites. Since we make use of those, we run
our tests using the --flatXML flag to Bucky's junit command, creating a
report that's somewhat garbled but understood by Hudson.

The resulting reports look like this:

<testsuite name="test.name.from.launch.config" tests="231" errors="0"
failures="13" ignored="2">
<testcase name="testMethod1"
classname="org.example.FQNSuite$org.example.FQNTestClass" time="21.288"/>
<testcase name="testMethod2"
classname="org.example.FQNSuite$org.example.FQNTestClass" time="2.799"/>
<testcase name="testParameterizedFoo[0]"
classname="org.example.FQNSuite$org.example.FQNParameterizedTestClass$[0]$org.example.FQNParameterizedTestClass"
time="1.302"/>
<testcase name="testParameterizedBar[0]"
classname="org.example.FQNSuite$org.example.FQNParameterizedTestClass$[0]$org.example.FQNParameterizedTestClass"
time="1.302"/>
<testcase name="testParameterizedFoo[1]"
classname="org.example.FQNSuite$org.example.FQNParameterizedTestClass$[1]$org.example.FQNParameterizedTestClass"
time="0.98"/>
<!-- ... -->
</testsuite>


Does anybody have any experience with setting up something similar with both
Hudson and Mylyn understanding the test reports correctly? Is there a better
alternative than using --flatXML that's understood by both maybe?


Best regards,
Carsten
(no subject) [message #717390 is a reply to message #715780] Sat, 20 August 2011 10:41 Go to previous message
Steffen Pingel is currently offline Steffen Pingel
Messages: 706
Registered: July 2009
Senior Member
Hi Carsten,

the Hudson integration regonizes the standard Tycho/surefire output without
any problems but as far as I know that does not have information about
nested test suites. You can try one of the Mylyn build jobs as an example:
https://hudson.eclipse.org/hudson/job/mylyn-integration-e4.1/

Please file a bug for enhancing the the JUnit support for the Buckminster
format. A sample report like the one below or a small Maven sample project
and pointers to documentation about the format (if you have any) would be
very helpful. It looks like it shouldn't be overly difficult to add that.

Steffen


Carsten Reckord wrote:

> Hi all,
>
> Does anybody here have a working build setup with Hudson/Jenkins and
> Buckminster, in which the JUnit test reports work with the Mylyn Build
> Connector for Hudson?
>
> Our problem with our current setup using Mylyn 3.6.1 is that
> - failed tests in the build's "Test Results" section cannot be
> opened(Failed to locate test in workspace)
> - opening the report in the JUnit view works, but again failed tests are
> not openable (Test class not found in selected project)
>
> The reason for this is the format of our test reports, which contains
> nested test classnames concatenated by '$' which Mylyn doesn't seem to
> understand.
>
>
> Some background:
>
> We are running Buckminster builds of Eclipse projects using the Hudson
> Buckminster plug-ins. Those builds contain tests that are run through
> Buckminster's junit action that triggers standard Eclipse JUnit runs
> configured through a launch config.
>
> This works great, mostly. However, there are some differences between
> Eclipse's JUnit XML report and the format understood by Hudson/Jenkins,
> especially concerning nested test suites. Since we make use of those, we
> run our tests using the --flatXML flag to Bucky's junit command, creating
> a report that's somewhat garbled but understood by Hudson.
>
> The resulting reports look like this:
>
> <testsuite name="test.name.from.launch.config" tests="231" errors="0"
> failures="13" ignored="2">
> <testcase name="testMethod1"
> classname="org.example.FQNSuite$org.example.FQNTestClass" time="21.288"/>
> <testcase name="testMethod2"
> classname="org.example.FQNSuite$org.example.FQNTestClass" time="2.799"/>
> <testcase name="testParameterizedFoo[0]"
>
classname="org.example.FQNSuite$org.example.FQNParameterizedTestClass$[0]$org.example.FQNParameterizedTestClass"
> time="1.302"/>
> <testcase name="testParameterizedBar[0]"
>
classname="org.example.FQNSuite$org.example.FQNParameterizedTestClass$[0]$org.example.FQNParameterizedTestClass"
> time="1.302"/>
> <testcase name="testParameterizedFoo[1]"
>
classname="org.example.FQNSuite$org.example.FQNParameterizedTestClass$[1]$org.example.FQNParameterizedTestClass"
> time="0.98"/>
> <!-- ... -->
> </testsuite>
>
>
> Does anybody have any experience with setting up something similar with
> both Hudson and Mylyn understanding the test reports correctly? Is there a
> better alternative than using --flatXML that's understood by both maybe?
>
>
> Best regards,
> Carsten

--
Steffen Pingel
Committer, http://eclipse.org/mylyn
Senior Developer, http://tasktop.com
Previous Topic:Package explorer broken with Working Sets as Top Level Elements
Next Topic:Wish to freeze context & open editors status
Goto Forum:
  


Current Time: Sun Sep 21 02:10:30 GMT 2014

Powered by FUDForum. Page generated in 0.08729 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software