Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Buckminster » JUnit command runs but reports no tests and no errors
JUnit command runs but reports no tests and no errors [message #637515] Fri, 05 November 2010 18:56 Go to next message
Daniel Ford is currently offline Daniel Ford
Messages: 138
Registered: July 2009
Location: New York
Senior Member
I'm having an interesting time trying to get JUnit4 tests to be run by
Buckminster under Hudson. I've got things to the point where the test
is run, but nothing happens, good or bad. There is a single test to be
run and everything works in the IDE from the same launch configuration.

From the output below, it appears that no tests were run, nor ignored,
and there were no errors. The test case must not be found.

Any suggestions on what might be wrong and how to proceed?

Thanks

Dan

------------------------------------------------------------ ------------


Waiting for jobs to end
junit '-l' com.foo.bar.test/FooServiceTest.launch' '--stderr'
'c:/output/junit_stderr' '--stdout' 'c:/output/junit_stdout' '-o'
'c:/output/test-reports/junit_result.xml'

INFO: Tests finished.
INFO: Elapsed time: ? seconds.
INFO: Total number of tests executed: 0
INFO: Successful tests: 0
INFO: Failed tests: 0
INFO: Errors: 0
INFO: Ignored tests: 0
INFO: Overall status: Failure
Doing full workspace refresh
Waiting for jobs to end
Recording test results
No test report files were found. Configuration error?
Re: JUnit command runs but reports no tests and no errors [message #637656 is a reply to message #637515] Mon, 08 November 2010 04:06 Go to previous messageGo to next message
Philipp Nanz is currently offline Philipp Nanz
Messages: 46
Registered: February 2010
Member
Hi Dan, maybe this will help.

[Updated on: Mon, 08 November 2010 04:07]

Report message to a moderator

Re: JUnit command runs but reports no tests and no errors [message #637774 is a reply to message #637656] Mon, 08 November 2010 14:18 Go to previous messageGo to next message
Daniel Ford is currently offline Daniel Ford
Messages: 138
Registered: July 2009
Location: New York
Senior Member
Thanks, I gave it try, but got the same result. I then thought that
maybe it was an undocumented difference between JUnit3 and JUnit4 test
case handling, so I used the Junit4TestAdapter to see if adapting my
JUnit4 test case would get it recognized. That didn't work either, same
result, no failures, no errors, no output, no error messages, and no
ignored tests reported.

Anybody have any other ideas?

Dan


On 11/8/2010 4:07 AM, Philipp Nanz wrote:
> Hi Dan, maybe this will help:
> http://www.eclipse.org/forums/index.php?t=msg&th=171582& amp;start=0&S=1f0bfc3c80e6f0b5d67d63ed48fe5cba
>
Re: JUnit command runs but reports no tests and no errors [message #637790 is a reply to message #637774] Mon, 08 November 2010 15:25 Go to previous messageGo to next message
Daniel Ford is currently offline Daniel Ford
Messages: 138
Registered: July 2009
Location: New York
Senior Member
I got everything to work finally, but I'm not sure exactly what was
going wrong. However, Buckminster is still reporting that no tests were
run and that none were ignored. This is confusing to say the least. I
was having trouble getting Hudson to find the test output at the same
time, so I'm wondering if things weren't actually working to begin with,
but that I didn't examine the output file because Buckminster was
reporting that no tests were found, passed, failed, or ignored (the
output file is buried on a different build machine). Perhaps the test
case accounting is only correct for JUnit3? That would explain why my
JUnit4 test "didn't count."

One thing I did differently was in the launch configuration, I changed
from executing all of the tests in the package, to just a specific test.
Can't see why this made a difference given that there was only one
test in the package.

Dan


On 11/8/2010 2:18 PM, Daniel Ford wrote:
> Thanks, I gave it try, but got the same result. I then thought that
> maybe it was an undocumented difference between JUnit3 and JUnit4 test
> case handling, so I used the Junit4TestAdapter to see if adapting my
> JUnit4 test case would get it recognized. That didn't work either, same
> result, no failures, no errors, no output, no error messages, and no
> ignored tests reported.
>
> Anybody have any other ideas?
>
> Dan
>
>
> On 11/8/2010 4:07 AM, Philipp Nanz wrote:
>> Hi Dan, maybe this will help:
>> http://www.eclipse.org/forums/index.php?t=msg&th=171582& amp;start=0&S=1f0bfc3c80e6f0b5d67d63ed48fe5cba
>>
>>
>
Re: JUnit command runs but reports no tests and no errors [message #638110 is a reply to message #637790] Tue, 09 November 2010 18:10 Go to previous messageGo to next message
Daniel Ford is currently offline Daniel Ford
Messages: 138
Registered: July 2009
Location: New York
Senior Member
I finally figured out the problem.

My JUnit4 test case was failing (correctly) in a method annotated with
@BeforeClass. This method is run before the actual test methods
(annotated with @Test) are run. A failure at this point of the test
process seems to be ignored by Buckminster. This would explain why
Buckminster reported no tests passing, failing or being ignored as none
of the methods annotated with @Test were actually entered. The entire
test process failed in the initialization code before it got to those
methods.

This problem compounds itself when running in Hudson as it seems that
Hudson ignores this type of failure as well, even though the JUnit test
report output clearly contains the failure. The problem is that the
failure is associated not with the individual tests, but with the suite
of tests; the individual tests are listed in the report with no failures
associated with them specifically. The net result is that Hudson shows
all tests as having passed, when, in fact, they ALL FAILED!

Dan


On 11/8/2010 3:25 PM, Daniel Ford wrote:
> I got everything to work finally, but I'm not sure exactly what was
> going wrong. However, Buckminster is still reporting that no tests were
> run and that none were ignored. This is confusing to say the least. I
> was having trouble getting Hudson to find the test output at the same
> time, so I'm wondering if things weren't actually working to begin with,
> but that I didn't examine the output file because Buckminster was
> reporting that no tests were found, passed, failed, or ignored (the
> output file is buried on a different build machine). Perhaps the test
> case accounting is only correct for JUnit3? That would explain why my
> JUnit4 test "didn't count."
>
> One thing I did differently was in the launch configuration, I changed
> from executing all of the tests in the package, to just a specific test.
> Can't see why this made a difference given that there was only one test
> in the package.
>
> Dan
>
>
> On 11/8/2010 2:18 PM, Daniel Ford wrote:
>> Thanks, I gave it try, but got the same result. I then thought that
>> maybe it was an undocumented difference between JUnit3 and JUnit4 test
>> case handling, so I used the Junit4TestAdapter to see if adapting my
>> JUnit4 test case would get it recognized. That didn't work either, same
>> result, no failures, no errors, no output, no error messages, and no
>> ignored tests reported.
>>
>> Anybody have any other ideas?
>>
>> Dan
>>
>>
>> On 11/8/2010 4:07 AM, Philipp Nanz wrote:
>>> Hi Dan, maybe this will help:
>>> http://www.eclipse.org/forums/index.php?t=msg&th=171582& amp;start=0&S=1f0bfc3c80e6f0b5d67d63ed48fe5cba
>>>
>>>
>>>
>>
>
Re: JUnit command runs but reports no tests and no errors [message #638252 is a reply to message #638110] Wed, 10 November 2010 07:58 Go to previous messageGo to next message
Thomas Hallgren is currently offline Thomas Hallgren
Messages: 3228
Registered: July 2009
Senior Member
On 2010-11-10 00:10, Daniel Ford wrote:
> ...The net result is that Hudson shows
> all tests as having passed, when, in fact, they ALL FAILED!
>
Jikes! Sounds like a bugzilla would be in order.

- thomas
Re: JUnit command runs but reports no tests and no errors [message #638296 is a reply to message #638252] Wed, 10 November 2010 13:11 Go to previous message
Daniel Ford is currently offline Daniel Ford
Messages: 138
Registered: July 2009
Location: New York
Senior Member
The real bug is in how the test report reports that there are no
failures. This is picked up by Buckminster and Hudson (quite
reasonably) and used as the truth. Fortunately, the report also
contains a sub-element that has the failure report. Maybe this could be
detected as a work around.

I created https://bugs.eclipse.org/bugs/show_bug.cgi?id=329933 with the
test report showing the test report and a code snippet for a test that
should produce the problem.

Dan


On 11/10/2010 7:58 AM, Thomas Hallgren wrote:
> On 2010-11-10 00:10, Daniel Ford wrote:
>> ...The net result is that Hudson shows
>> all tests as having passed, when, in fact, they ALL FAILED!
>>
> Jikes! Sounds like a bugzilla would be in order.
>
> - thomas
Previous Topic:version qualifier replacement in product configuration
Next Topic:Problem with CSpec/RMAP for org.antlr and org.apache
Goto Forum:
  


Current Time: Sat Aug 23 11:24:55 EDT 2014

Powered by FUDForum. Page generated in 0.02299 seconds