Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Jubula » Testexec - report only failed tests
Testexec - report only failed tests [message #1721705] Fri, 29 January 2016 10:24 Go to next message
Alexander Kolesov is currently offline Alexander KolesovFriend
Messages: 43
Registered: November 2012
Member
Hi guys,
Is there an option like in Jubula UI/Preferences to create reports that include only failed cases when running tests with testexec?

Thanks,
Alex.
Re: Testexec - report only failed tests [message #1722043 is a reply to message #1721705] Tue, 02 February 2016 13:13 Go to previous messageGo to next message
Alexandra Schladebeck is currently offline Alexandra SchladebeckFriend
Messages: 1613
Registered: July 2009
Senior Member
Hi Alex,

Thanks for your question. There isn't a way of only reporting failed tests via test exec I'm afraid. You can only opt to write reports or not write them.

Best regards,
Alex
Re: Testexec - report only failed tests [message #1722045 is a reply to message #1722043] Tue, 02 February 2016 13:23 Go to previous messageGo to next message
Alexander Kolesov is currently offline Alexander KolesovFriend
Messages: 43
Registered: November 2012
Member
Hi Alex,
In case of a big amount of test cases I fall in two issues:
1. Reports are really large and are hard to be opened.
2. On big reports I observe Jubula to fail with OOM when generating test results.

Both issues can be resolved by splitting testset in smaller sets, however it would be great to have an option to handle results reporting via some option.

Thanks,
Alex.
Re: Testexec - report only failed tests [message #1722053 is a reply to message #1722045] Tue, 02 February 2016 13:58 Go to previous messageGo to next message
Alexandra Schladebeck is currently offline Alexandra SchladebeckFriend
Messages: 1613
Registered: July 2009
Senior Member
Thanks for the feedback. Which version of Jubula are you using, and are you on 32- or 64-bit systems?
Re: Testexec - report only failed tests [message #1722055 is a reply to message #1722053] Tue, 02 February 2016 14:10 Go to previous messageGo to next message
Alexander Kolesov is currently offline Alexander KolesovFriend
Messages: 43
Registered: November 2012
Member
Jubula
Version: 8.2.0.021
x64
Re: Testexec - report only failed tests [message #1722153 is a reply to message #1722055] Wed, 03 February 2016 06:49 Go to previous messageGo to next message
Alexandra Schladebeck is currently offline Alexandra SchladebeckFriend
Messages: 1613
Registered: July 2009
Senior Member
Thanks - how large are the tests? We execute some pretty large test suites, but don't run into problems with out of memory. Do the tests have lots of errors (screenshots?). It would be great if you could provide some details on the length of time the tests run for / how many steps they contain, if you are running code coverage as well, and when exactly the OOM occurs (writing results to the database?) and the stack trace for when it happens.

Thanks,
Alex
Re: Testexec - report only failed tests [message #1722211 is a reply to message #1722153] Wed, 03 February 2016 12:15 Go to previous messageGo to next message
Alexander Kolesov is currently offline Alexander KolesovFriend
Messages: 43
Registered: November 2012
Member
Stack trace from client.log:
index.php/fa/24873/0/

Here is statistic of testset (please note, it does not fail with OOM now. When it failed there were more executed CAPs (run time was one hour more) and lots of screenshots since many tests were failing. When OOM occurred xml report was created (~130Mb) and HTML report with zero length):
↦ Start Time Tue Feb 02 11:36:24 PST 2016
⇥ End Time Tue Feb 02 15:04:29 PST 2016
↹ Duration (hh:mm:ss) 03:28:04
∑ expected CAPs 28121
∑ executed CAPs 56284
∅ CAP execution (millis) 221
⤳ Event Handler CAPs 28725
↯ failed CAPs 4
Re: Testexec - report only failed tests [message #1722335 is a reply to message #1722211] Thu, 04 February 2016 08:02 Go to previous messageGo to next message
Alexandra Schladebeck is currently offline Alexandra SchladebeckFriend
Messages: 1613
Registered: July 2009
Senior Member
Thanks for the information. I'll have someone check it out - it will be next week due to some holidays at the moment.
Best regards,
Alex
Re: Testexec - report only failed tests [message #1722699 is a reply to message #1721705] Mon, 08 February 2016 11:49 Go to previous messageGo to next message
Marvin Mueller is currently offline Marvin MuellerFriend
Messages: 115
Registered: March 2012
Senior Member
Hi Alexander,

There is a way to raise the memory for the testexec if you are using a 64 Bit version. If you are having a 32 Bit version of jubula it is already at a very high level.
If you want raise it for your 64 Bit Jubula, you must raise the -Xmx value in the testexec.vmoptions

Also we have the options for the testexec not to do Screenshots at all("-s") or not to write Screenshots to XML/HTML since this is sometimes using a lot of space ("-sx").

Best regards,
Marvin
Re: Testexec - report only failed tests [message #1722809 is a reply to message #1722699] Tue, 09 February 2016 10:09 Go to previous message
Alexander Kolesov is currently offline Alexander KolesovFriend
Messages: 43
Registered: November 2012
Member
Hi Marvin,
I have already increased Xmx for testexec, however large html reports are hard to view in browsers.
As for screenshots - reports of UI testing are much less informative without screenshots of a problems Smile

Thanks,
Alex.
Previous Topic:Working workspace for Jubula extension
Next Topic:Click succes - wrong
Goto Forum:
  


Current Time: Fri Oct 19 12:51:59 GMT 2018

Powered by FUDForum. Page generated in 0.02055 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top