Using TPTP's API Recorder to record application behavior and generate a TPTP JUnit test suite
Author: Borna Safabakhsh (borna@us.ibm.com)
Reviewer: Paul Slauenwhite (paules@ca.ibm.com)
Last Updated: November 30, 2006

TPTP API Recorder and Test Generation Tutorial

The following document provides an overview of the new API Recorder Technology Preview in TPTP, and how it can be used to record application behavior and generate a TPTP JUnit test suite.

Table of Contents:
1.0 Introduction
1.1 Obtaining TPTP API Recorder plug-ins
2.0 Step by Step Tutorial by Example
2.1 Recording Application Behavior
2.2 Generating and Running a JUnit Test Suite based on a Recording
Appendix: Recording Target Specification Filters

1.0 Introduction

The API Recorder is an agent that records the behavior of an application at specified points. This API Recorder is designed for Java applications, and is capable of recording the execution of a Java application and capturing the entry and exit of methods, the arguments provided to them, and the values returned by them. For example, invocations of an application's methods may be recorded, and arguments and return values captured. The recording data may then be used for analysis, testing, or the generation of a test harness to further test the application's methods. Using the API Recorder begins with a user launching a Java application in a TPTP test and profiling environment. By declaring Java package, class, and method targets through the API Recorder tooling, the user can specify the method invocations to capture for recording and test suite generation. Launching the user application with the API Recorder agent attached, and executing the targeted code paths, will allow the API Recorder to capture the method invocations and the associated parameters and return value. The user can then use standard TPTP tools, such as the variety of views in the Profiling and Logging perspective, to analyze the trace data and can also use the API recorder's JUnit test generation facilities to generate a JUnit test suite mimicking the recorded behavior. The generated JUnit test suite will build the JUnit test class that invokes the recorded method using the equivalent input parameters to those recorded and verifies the equivalency of the return value.

The reader is assumed to have an intermediate knowledge of TPTP. The features leveraged in this document are based on TPTP 4.3. Ensure that a correct driver is downloaded and setup. See TPTP's installation guide for more details on installing TPTP.

The source code of the SimpleStringTest plug-in created can be downloaded here.


1.1 Obtaining TPTP API Recorder plug-ins

Using a browser, navigate to the TPTP downloads page (http://www.eclipse.org/tptp/ >> Downloads >> 4.3) and download the TPTP driver. Within the download page, go to the Technology Previews >> API Recorder Framework and Test Tools section and download the API Recorder feature and plug-in. Install the downloaded feature and plug-in into a TPTP 4.3-enabled Eclipse environment. After installing the feature and plug-in, launch the Eclipse environment with the "-clean" command line option.


2.0 Step by Step Tutorial by Example

In this example, we will be recording the execution behavior of a very simple Java application, and focus on the new features rather than the nuances of the example application. We begin with our application under test, by first recording its method invocations of interest, followed by generating a test suite from the recording.


2.1 Recording Application Behavior

Below is the sample class ("SimpleStringTest"), implementing the method ("testThisMethod") we plan to record invocations of. In this Technology Preview, the API Recorder supports only simple parameter and return value types (a known area for future improvement that is being addressed).


package api.tutorial;

public class SimpleStringTest {
	public String testThisMethod(String inputA, String inputB){
		return "" + inputA + inputB;
	}
	
	public static void main(String[] args){
		SimpleStringTest sst = new SimpleStringTest();
			System.out.println(
			sst.testThisMethod("My application ", 
					"successfully completed execution."));
	}
}

To begin with the profiling launch configuration wizard, select Profile... from the Run menu.



Figure 2.1.1 - Profiling Launch Configuration - Monitor tab

From the list of available launch configuration types on the left, select Java Application. Create a new launch configuration of this type by clicking the New icon.
Using the Browse and Search features of the launch configuration wizard, select the project and main class for the application to be recorded.
Under the Monitor tab, click to enable the selection of the API Recording data collector and the associated Java Applications analysis type. With Java Applications highlighted, click the Edit Options.
Within the target selection table, add rows of filters describing the recording targets. See the appendix in this tutorial for more information regarding target specification.

In this tutorial, we target the invocation of the testThisMethod method by applying filters matching any method invocation within the associated Java package api.tutorial.
Figure 2.1.2 - Recording target filters

Click Finish. Click Profile to launch the Java application with the API Recorder agent attached. In this case, no user interaction with the Java application is necessary to induce the requested code path. All invocations of the testThisMethod method, in this case only a single one, prior to the termination of the user application are recorded.

Upon completion of the recorded application, switch to the TPTP Profiling and Logging perspective, if the workbench has not done so automatically.

Figure 2.2 provides the standard TPTP profiling views summarizing the application trace data.


Figure 2.2 - Profiling Summary



2.2 Generating and Running a JUnit Test Suite based on a Recording

To generate a TPTP JUnit test suite from the Profiling and Logging perspective, right-click on the recording and select Generate TPTP JUnit Test Suite from the context menu. Here we select the test generator and the target file for the generated test suite. For this tutorial, we select TPTP JUnit Test Suite Generator as the test suite generator and provide a target test suite file name.



Figure 2.2.1 - Test Generation Configuration

The generated JUnit tests can be found in the specified project under the tests.junit Java package. By right-clicking on the generated TestSuite.testsuite file, and selecting Run As... >> Test, the test suite will be executed. The results of these tests will be presented in the standard TPTP JUnit test results.


Figure 2.2.2 - Test Execution Results



Appendix: Recording Target Specification Filters
A target specification indicates the classes and methods to which the recording should be applied.

The target specification is optional. When no target is specified, the recording will be applied to all classes that are processed by the instrumentation engine.

The target specification lets you create filter rules to include or exclude methods from instrumentation based upon patterns that are matched against the package, class, and method name, plus the method's signature. The patterns can contain wildcards, where "*" matches zero or more characters.

If a target's wildcard patterns match a method's package, class, name and signature, the type property determines whether the method is instrumented (include) or not (exclude). If the patterns do not match a method's package, class, name and signature, by default, the method is instrumented.
The target includes the following properties:

Property Description
type Required. The type property determines whether or not a recording is applied to a target method. Specify include to apply the recording to classes and methods that match the wildcard patterns, exclude to exclude them.
package Optional. Specify a wildcard pattern to match against the package portion of class names. For example: java.util* matches every class in the java.util package and its subpackages. If not specified, the default value is *.
className Optional. Specify a wildcard pattern to match against class names. If not specified, the default value is *.
method Optional. Specify a wildcard pattern to match against method names. If not specified, the default value is *.
signature Optional. Specify a wildcard pattern to match against a method's signature. (The signature is the string representing the method's arguments and return type.) Use the Java internal format for method signatures. For example: (Ljava/lang/Object;)D is the signature of a method that takes an Object as a parameter and returns a double. This wildcard pattern can be used to distinguish among overloaded methods. If not specified, the default value is *.