Home » Language IDEs » Java Development Tools (JDT) » Collecting results of JUnit test runs
| Collecting results of JUnit test runs [message #244670] |
Tue, 19 June 2007 09:50  |
Eclipse User |
|
|
|
Originally posted by: monikakrug.expires-2007-06-30.arcornews.de
I have serious difficulties with collecting the results of JUnit test
runs. The ITestRunListener seems to get notified randomly - or not. When
there is just one test case, it works fine. When there are four, the
listener only gets called for two of them.
Pasting all the code here would be hundreds of lines, so I am just going
to describe most of it in general. My plug-in works like this: It
contributes to the pop-up menu for JUnit testcases (ICompilationUnits
that fit the "*Test.java" filter). It supports test-driven development
or rather "extreme harvesting".
1. It parses the selected test case and figures out what methods (their
names, parameters and return types) the tested (not yet written) class
is to have.
2. It generates a query and sends it to Merobase (merobase.com, the
software component search engine). A view is shown with the query and it
can be edited before it is sent.
3. It downloads the classes Merobase has found. The results are also
shown in a table in the view.
4. It creates a Java project for each, puts the downloaded class into it
and creates a copy of the test case the developer wrote in each project
(and adds import statements, moves the downloaded class if it was in the
default package, creates an adapter class if the downloaded class does
not have the right name and such things).
5. If there are no compilation errors (which happen mostly if the
downloaded class refers to other classes that were not downloaded), a
launch configuration is created and run:
ILaunchManager manager = DebugPlugin.getDefault().getLaunchManager();
ILaunchConfigurationType type =
manager.getLaunchConfigurationType(JUnitLaunchConfiguration. ID_JUNIT_APPLICATION);
ILaunchConfigurationWorkingCopy launchConfigurationWc =
type.newInstance(null/*container*/, "Testcase in " + projectName);
launchConfigurationWc.setAttribute(IJavaLaunchConfigurationC onstants.ATTR_PROJECT_NAME,
projectName);
launchConfigurationWc.setAttribute(JUnitBaseLaunchConfigurat ion.TESTTYPE_ATTR,
testCaseCopy.findPrimaryType().getFullyQualifiedName());
launchConfigurationWc.setAttribute(IJavaLaunchConfigurationC onstants.ATTR_MAIN_TYPE_NAME,
testCaseCopy.findPrimaryType().getFullyQualifiedName());
ILaunchConfiguration configuration = launchConfigurationWc.doSave();
DebugUITools.launch(configuration, ILaunchManager.RUN_MODE);
JUnitLaunchConfiguration.ID_JUNIT_APPLICATION and
JUnitBaseLaunchConfiguration.TESTTYPE_ATTR are in internal packages, but
I didn't find another way of running the test cases.
DebugUITools.launch runs the test cases asynchronously, they usually
don't even start to run before all the projects have been created already.
The problem is how to collect the results. I implemented an
ITestRunListener that sets a boolean testFailed to false in
testRunStarted(), to true in testFailed() and changes a field in the row
in the table in the view to green or red in testRunEnded(). I add it
with JUnitCore. Now how to tell it which row of the table to change to
red or green:
Because DebugUITools.launch runs the tests asynchronously, it's not
possible to add an instance of the listener, call launch, remove it:
ITestRunListener listener = new MyTestRunListener(row);
JUnitCore.addTestRunListener(listener);
DebugUITools.launch(configuration, ILaunchManager.RUN_MODE);
JUnitCore.removeTestRunListener(listener); // would remove the listener
before the test even begins to run
So I used only one instance of my listener and wanted to add a guarded
block so that the next test run would only start when the previous one
has finished. Before launch is called, the thread waits until notified
and checks if the variable is true. Then sets it to false. In
testRunEnded (and in testRunStopped and testRunTerminated) the variable
is set to true again and notifyAll() is called.
private volatile boolean allowContinue = true;
private ITestRunListener listener;
public synchronized void runTests()
{
if (listener==null)
{
listener = new MyTestRunListener();
JUnitCore.addTestRunListener(listener);
}
// create Java project
// download class into it
// copy testcase into it
// ...
// create launch configuration (see above)
while (!allowContinue)
{
try
{
wait();
}
catch(InterruptedException ex)
{
}
}
allowContinue = false;
listener.setRow(row);
DebugUITools.launch(configuration, ILaunchManager.RUN_MODE);
// ...
}
private class MyTestRunListener implements ITestRunListener
{
private TableItem tableRow;
public void setRow(TableItem row) { this.tableRow = row; }
private boolean testFailed;
public void testRunStarted(int testCount)
{
testFailed = false;
System.out.println(".testRunStarted()");
}
public void testFailed(int status, String testId, String testName,
String trace)
{
testFailed = true;
System.out.println(".testFailed()");
}
public synchronized void testRunEnded(long elapsedTime)
{
try
{
System.out.println(".testRunEnded()");
// need to make UI changes in the UI thread:
Display display = tableRow.getDisplay();
display.asyncExec(new Runnable()
{
public void run()
{
if (testFailed)
{
setResultColumn(tableRow, "failed");
setResultColumn(tableRow, RED);
}
else
{
setResultColumn(tableRow, "passed");
setResultColumn(tableRow, GREEN);
}
}
});
}
finally
{
allowContinue = true;
notifyAll();
}
}
// remaining methods with debug statements ...
}
As soon as the second project with no compile errors is reached, the
program hangs up, waiting forever. So evidently allowContinue is never
set to true again. Actually not even testRunStarted() gets called!
So I removed all the synchronization code for debugging purposes and
expected the testRunStarted testStarted testFailed testEnded
testRunEnded messages of different tests to get mixed and the last table
row to show the result of the last test, the others showing nothing.
(Unless there is only one test case, when it should work.)
That's not quite what happens: It works with one test case (as one would
expect). But when there are four compilable test cases (out of 34
downloaded classes), testRunStarted etc. get called for two of them. No
exceptions thrown for the others or any error messages. The test cases
did get run though: The launch configurations are seen in the list, and
in the list of terminated consoles all four are to be found, and if I
put debug statements into the tested classes, these get printed for each
project. As one would expect, the last of the table rows with no
compilation problems shows a red or green field while the others stay
white - but sometimes even that last one stays white.
I have tried going through it step by step in debug mode, but new
threads get started all the time and I get lost eventually. I haven't
found anything so far that would explain this behavior.
So, how can it be that only 2 out of 4 test cases send notifications to
the ITestRunListener? How can I collect the results of the test runs
properly?
Monika.
|
|
| | | |
Goto Forum:
Current Time: Sun Jun 08 14:20:11 EDT 2025
Powered by FUDForum. Page generated in 0.03131 seconds |
Home