Home » Modeling » Epsilon » How to keep the EUnit test running to find all failures at on time and show what are those failures?
How to keep the EUnit test running to find all failures at on time and show what are those failures? [message #1843367] |
Mon, 26 July 2021 11:52 |
健 林 Messages: 15 Registered: July 2021 |
Junior Member |
|
|
I want to show all the failures at a time, but the EUnit test will stop when find the first bug. Could you show me how to keep the test running, even if the bug has been found? I saw that in the XML file which is generated by default, there are only the number of failures and errors. I want to output what is the bug just like the "compare" button in the eclipse. I am using command line to run the EUnit, so the "compare" button is not available. Is there a way to output the what is the failure?
[Updated on: Tue, 10 August 2021 15:42] Report message to a moderator
|
|
|
Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843421 is a reply to message #1843367] |
Wed, 28 July 2021 17:00 |
|
EUnit has similar semantics to JUnit - when an assertion fails, it will immediately fail that test. It will still run the rest of the tests in the suite.
This is not customizable at the moment, as most tests would simply crash regardless. For instance, if a test expected an array to have 1 element and then it didn't, going on even after that assertion failed would probably just result in the test crashing with an ArrayIndexOutOfBoundsException.
Regarding placing the "Compare" results in the XML file, this is not currently supported as the exact serialisation format for those differences would depend on the comparator being used in assertEqualModels, and it would result in the XML file no longer following the JUnit XML format (breaking things like CI integration of EUnit tests).
We could look into extending the EMF Compare-based EUnit model comparator so it produced an XMI serialisation of the found differences, however. Would that work for you?
|
|
|
Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843479 is a reply to message #1843421] |
Mon, 02 August 2021 04:55 |
健 林 Messages: 15 Registered: July 2021 |
Junior Member |
|
|
I am working on a project which is an ETL-grader. The ETL-grader will automatically evaluate the ETL script provided by students and give the mark and feedback. I use the EUnit test to achieve this function. I want to get all the failures in the ETL script which would be useful to calculate the mark and output the feedback. As I know, in Junit, this can be achieved by using
org.junit.runner.JUnitCore;
org.junit.runner.Result;
org.junit.runner.Description;
org.junit.runner.notification.Failure;
org.junit.runner.notification.RunListener; packages.
However, I am not so familiar with EUnit. The EUnit test is written by EOL and is executed by ant tasks. Therefore, I have no idea about how to do similar things just like in the Junit. The following is a JAVA file which do the similar work. Could you please give me some suggestions or solutions?
[Updated on: Mon, 02 August 2021 13:27] Report message to a moderator
|
|
| |
Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843495 is a reply to message #1843489] |
Mon, 02 August 2021 13:26 |
健 林 Messages: 15 Registered: July 2021 |
Junior Member |
|
|
Dear Dimitris,
I am still learning about Eunit so I am just use the eunit.etl example [1] as my project now. The only difference is that I use a build.xml namely ant task to execute only one of the tests on Linux by using command line. The output I want:
1. The number of failures and total points which have been tested. Then, I can calculate the percentage of failures which can be used to calculate the score of the ETL script by percentage = ((total points - failures) / total points). For example, in my current project, when a tree convert to a graph, if the label and the parent of a node are incorrect in the new graph, I want the program output there are two failures and five places (for example) of the new graph have been checked. Then, I can evaluate the ETL script which get (5-2) / 5 * 100% = 60% of the total score.
2.The detail of bugs. I want to make a feedback, after an ETL script is assessed, to convince students. For example, in this project , if I change "n.name = t.label" to "n.name = t.label + "bug"" in ETL script, there will be a failures and I want the information of what is the failure could be printed just like the information I can see when I click "compare" button in the eclipse. If there are several bugs, can I output all of them at a time?
The version of Epsilon:
1.5.1.201809302027
The attachment is the project. You can execute it by type "ant" in the command line under the directory.
Best wishes,
Jian
[1] https://git.eclipse.org/c/epsilon/org.eclipse.epsilon.git/plain/examples/org.eclipse.epsilon.eunit.examples.etl/
-
Attachment: test.zip
(Size: 15.89MB, Downloaded 65 times)
[Updated on: Mon, 02 August 2021 13:48] Report message to a moderator
|
|
|
Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843511 is a reply to message #1843495] |
Tue, 03 August 2021 10:58 |
|
Hi Jian,
I think that you'll need to run EUnit from its API instead of ANT to achieve this. See a minimal example below:
import java.io.File;
import java.util.List;
import org.eclipse.epsilon.eunit.EUnitModule;
import org.eclipse.epsilon.eunit.EUnitTest;
import org.eclipse.epsilon.eunit.EUnitTestResultType;
public class EUnitApp {
public static void main(String[] args) throws Exception {
EUnitModule module = new EUnitModule();
String eunit = String.join("\n",
"@test",
"operation t1() { assertTrue(false); }",
"@test",
"operation t2() { assertTrue(false); }"
);
module.parse(eunit, new File("sample.eunit"));
module.execute();
List<?> selectedOperations = module.getSelectedOperations();
List<EUnitTest> testsWithFailures = module.getSuiteRoot().collectLeafTests(
selectedOperations, EUnitTestResultType.FAILURE);
System.out.println(testsWithFailures.size()); // Prints 2 as both t1 and t2 fail
}
}
Best,
Dimitris
|
|
| | | | | | |
Goto Forum:
Current Time: Fri Mar 29 01:48:04 GMT 2024
Powered by FUDForum. Page generated in 0.04735 seconds
|