Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » Epsilon » How to keep the EUnit test running to find all failures at on time and show what are those failures?
How to keep the EUnit test running to find all failures at on time and show what are those failures? [message #1843367] Mon, 26 July 2021 11:52 Go to next message
健 林 is currently offline 健 林Friend
Messages: 15
Registered: July 2021
Junior Member
I want to show all the failures at a time, but the EUnit test will stop when find the first bug. Could you show me how to keep the test running, even if the bug has been found? I saw that in the XML file which is generated by default, there are only the number of failures and errors. I want to output what is the bug just like the "compare" button in the eclipse. I am using command line to run the EUnit, so the "compare" button is not available. Is there a way to output the what is the failure?

[Updated on: Tue, 10 August 2021 15:42]

Report message to a moderator

Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843421 is a reply to message #1843367] Wed, 28 July 2021 17:00 Go to previous messageGo to next message
Antonio Garcia-Dominguez is currently offline Antonio Garcia-DominguezFriend
Messages: 594
Registered: January 2010
Location: Birmingham, UK
Senior Member

EUnit has similar semantics to JUnit - when an assertion fails, it will immediately fail that test. It will still run the rest of the tests in the suite.

This is not customizable at the moment, as most tests would simply crash regardless. For instance, if a test expected an array to have 1 element and then it didn't, going on even after that assertion failed would probably just result in the test crashing with an ArrayIndexOutOfBoundsException.

Regarding placing the "Compare" results in the XML file, this is not currently supported as the exact serialisation format for those differences would depend on the comparator being used in assertEqualModels, and it would result in the XML file no longer following the JUnit XML format (breaking things like CI integration of EUnit tests).

We could look into extending the EMF Compare-based EUnit model comparator so it produced an XMI serialisation of the found differences, however. Would that work for you?
Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843479 is a reply to message #1843421] Mon, 02 August 2021 04:55 Go to previous messageGo to next message
健 林 is currently offline 健 林Friend
Messages: 15
Registered: July 2021
Junior Member
I am working on a project which is an ETL-grader. The ETL-grader will automatically evaluate the ETL script provided by students and give the mark and feedback. I use the EUnit test to achieve this function. I want to get all the failures in the ETL script which would be useful to calculate the mark and output the feedback. As I know, in Junit, this can be achieved by using
org.junit.runner.JUnitCore;
org.junit.runner.Result;
org.junit.runner.Description;
org.junit.runner.notification.Failure;
org.junit.runner.notification.RunListener; packages.
However, I am not so familiar with EUnit. The EUnit test is written by EOL and is executed by ant tasks. Therefore, I have no idea about how to do similar things just like in the Junit. The following is a JAVA file which do the similar work. Could you please give me some suggestions or solutions?

[Updated on: Mon, 02 August 2021 13:27]

Report message to a moderator

Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843489 is a reply to message #1843479] Mon, 02 August 2021 09:41 Go to previous messageGo to next message
Dimitris Kolovos is currently offline Dimitris KolovosFriend
Messages: 2154
Registered: July 2009
Location: York, UK
Senior Member

Hi,

Could you please provide a minimal example [1] containing an ETL transformation, relevant models, metamodels and EUnit tests, as well as the output you want your grader implementation to produce from them so that we can advise?

Best,
Dimitris

[1] https://www.eclipse.org/epsilon/doc/articles/minimal-examples/
Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843495 is a reply to message #1843489] Mon, 02 August 2021 13:26 Go to previous messageGo to next message
健 林 is currently offline 健 林Friend
Messages: 15
Registered: July 2021
Junior Member
Dear Dimitris,

I am still learning about Eunit so I am just use the eunit.etl example [1] as my project now. The only difference is that I use a build.xml namely ant task to execute only one of the tests on Linux by using command line. The output I want:
1. The number of failures and total points which have been tested. Then, I can calculate the percentage of failures which can be used to calculate the score of the ETL script by percentage = ((total points - failures) / total points). For example, in my current project, when a tree convert to a graph, if the label and the parent of a node are incorrect in the new graph, I want the program output there are two failures and five places (for example) of the new graph have been checked. Then, I can evaluate the ETL script which get (5-2) / 5 * 100% = 60% of the total score.
2.The detail of bugs. I want to make a feedback, after an ETL script is assessed, to convince students. For example, in this project , if I change "n.name = t.label" to "n.name = t.label + "bug"" in ETL script, there will be a failures and I want the information of what is the failure could be printed just like the information I can see when I click "compare" button in the eclipse. If there are several bugs, can I output all of them at a time?

The version of Epsilon:
1.5.1.201809302027
The attachment is the project. You can execute it by type "ant" in the command line under the directory.

Best wishes,
Jian

[1] https://git.eclipse.org/c/epsilon/org.eclipse.epsilon.git/plain/examples/org.eclipse.epsilon.eunit.examples.etl/
  • Attachment: test.zip
    (Size: 15.89MB, Downloaded 65 times)

[Updated on: Mon, 02 August 2021 13:48]

Report message to a moderator

Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843511 is a reply to message #1843495] Tue, 03 August 2021 10:58 Go to previous messageGo to next message
Dimitris Kolovos is currently offline Dimitris KolovosFriend
Messages: 2154
Registered: July 2009
Location: York, UK
Senior Member

Hi Jian,

I think that you'll need to run EUnit from its API instead of ANT to achieve this. See a minimal example below:

import java.io.File;
import java.util.List;

import org.eclipse.epsilon.eunit.EUnitModule;
import org.eclipse.epsilon.eunit.EUnitTest;
import org.eclipse.epsilon.eunit.EUnitTestResultType;

public class EUnitApp {
	
	public static void main(String[] args) throws Exception {
		
		EUnitModule module = new EUnitModule();
		String eunit = String.join("\n",
				"@test",
				"operation t1() { assertTrue(false); }", 
				"@test",
				"operation t2() { assertTrue(false); }" 
				);
		module.parse(eunit, new File("sample.eunit"));
		module.execute();
		
		List<?> selectedOperations = module.getSelectedOperations();
		List<EUnitTest> testsWithFailures = module.getSuiteRoot().collectLeafTests(
				selectedOperations, EUnitTestResultType.FAILURE);
		
		System.out.println(testsWithFailures.size()); // Prints 2 as both t1 and t2 fail
	}
	
}


Best,
Dimitris
Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843512 is a reply to message #1843511] Tue, 03 August 2021 13:37 Go to previous messageGo to next message
健 林 is currently offline 健 林Friend
Messages: 15
Registered: July 2021
Junior Member
Dear Dimitris,
Could you please also share the document of API?
How to run the JAVA file you provided? I use "javac" to compile it, but the error shows packages does not exist.
Thank you,
Jian

[Updated on: Tue, 03 August 2021 15:03]

Report message to a moderator

Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843519 is a reply to message #1843512] Tue, 03 August 2021 18:30 Go to previous messageGo to next message
Dimitris Kolovos is currently offline Dimitris KolovosFriend
Messages: 2154
Registered: July 2009
Location: York, UK
Senior Member

Hi Jian,

The JavaDoc of EUnit is here: https://download.eclipse.org/epsilon/interim-javadoc/org/eclipse/epsilon/eunit/package-summary.html

I'm not aware of any additional documentation (beyond the JavaDoc and https://www.eclipse.org/epsilon/doc/eunit/) I'm afraid.

Best,
Dimitris
Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843541 is a reply to message #1843511] Thu, 05 August 2021 09:50 Go to previous messageGo to next message
健 林 is currently offline 健 林Friend
Messages: 15
Registered: July 2021
Junior Member
Dear Dimitris,
Could you share the whole project of the minimal example?
Best wishes,
Jian
Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843543 is a reply to message #1843541] Thu, 05 August 2021 11:31 Go to previous messageGo to next message
Dimitris Kolovos is currently offline Dimitris KolovosFriend
Messages: 2154
Registered: July 2009
Location: York, UK
Senior Member

Hi Jian,

The project you shared earlier in this thread already contains all the dependencies you need; if you put the EUnitApp class next to TransformationTest.java it should work fine.

Best,
Dimitris
Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843572 is a reply to message #1843543] Sat, 07 August 2021 14:15 Go to previous messageGo to next message
健 林 is currently offline 健 林Friend
Messages: 15
Registered: July 2021
Junior Member
Dear Dimitris,
As I introduced earlier, I am building a grader for ETL script. Hence, what I want is printing all the bugs in the output model. Is it possible to get all failures? For example, if the label of a node and the source of a edge are wrong, I want the program could output these two failures at a time.
Best wishes,
Jian
Re: How to keep the EUnit test running to find all failures at on time and show what are those failu [message #1843707 is a reply to message #1843572] Sat, 14 August 2021 17:59 Go to previous message
Antonio Garcia-Dominguez is currently offline Antonio Garcia-DominguezFriend
Messages: 594
Registered: January 2010
Location: Birmingham, UK
Senior Member

I have a first implementation of the required functionality here:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=575409

Let's continue the discussion there :-).
Previous Topic:Microsoft Excel sheets in Epsilon
Next Topic:How to compare two models to find the difference between them??
Goto Forum:
  


Current Time: Fri Mar 29 01:48:04 GMT 2024

Powered by FUDForum. Page generated in 0.04735 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top