The Eclipse Test & Performance Tools Platform (TPTP) is a powerful tool for evaluating the overall performance and quality of Java source code. In addition to well-known features for measuring code performance, TPTP now includes a framework that offers the user a uniform focus for all forms of static analysis. Introduced in TPTP 4.1, the static analysis framework seeks to create an environment where third-party analysis producers can integrate existing tooling and/or create new high performance analysis engines to eliminate the need for the user to learn different workflows for each form of analysis.
This first article in the series describes the user-focused view of the TPTP static analysis framework, and will focus solely on the user interface and basic analysis concepts. Other articles will delve deeper into the API’s available for writing Java code review rules and other forms of analysis.
Measuring the overall quality of code can be a difficult task. Developers naturally assume they write perfect code but in practice we all know this is simply not true. As a result, there exist many products that perform static code analysis. Most of these tools evaluate code and produce enlightening reports that describe the real quality of the code, but they vary in many, sometimes subtle, ways that confuse users.
The obvious problem with these tools is that they all have user interfaces of varying complexity, function and operation. With today’s products it is simply impossible to quickly switch from one analysis tool to another without climbing another learning curve. Further, each tool may perform a slightly different kind of analysis (e.g. Code review, type/state analysis, architectural discovery, etc.), and though beneath the surface they are quite different, these forms of analysis should all appear and act the same way from the end-user perspective since to a user it is all just analysis.
TPTP 4.1 introduces a new framework that allows both third party analysis tool vendors and those writing new tooling to quickly integrate different forms of analysis while maintaining the consistency of operation required by users. This article aims to provide a quick overview of the end-user capabilities of the TPTP static analysis framework. The programmatic workings of the analysis framework will be the subject of future articles in this series.
The get started with the static analysis feature, you must first install the latest release of TPTP. In order to obtain the complete static analysis framework with the Java code review provider and rules, you need to download a TPTP 4.2 build. This tool can be obtained from http://www.eclipse.org/tptp. Note that TPTP 4.2 has some additional requirements so read the installation instructions carefully.
Once installed, the analysis capabilities should automatically appear in the Java and Debug perspectives but these features can be added to any Eclipse perspective as well. To accomplish this, choose the “Window > Customize Perspective…” option from the Eclipse menu bar.
When the dialog appears, select the “Commands” tab and click the “Analysis” checkbox. Then accept the change by clicking the “OK” button.
You should immediately notice the new Analysis additions to the Eclipse toolbar and menu. These options allow you to create, modify or execute analysis configurations. Select the “Run > Analysis…” menu option to display the main analysis configuration dialog. You will see a dialog very much like the one used to run or debug code from the Eclipse workbench, and in fact for simplicity, it has been designed to function similarly to the dialogs you already use.
Analysis configurations can be added or removed using the buttons in the bottom left part of the dialog. As the name implies, a configuration is used to determine which forms of analysis and which rules are executed as well as the scope of analysis (e.g. a project, a working set or the whole workspace). To get started, select the “Analysis” element in the “Configurations” list on the left side of the configuration dialog, then click the “New” button. You will notice that the right side of the dialog changes to show the basic configuration interface.
The first step in creating an analysis configuration is to determine the default range of resources on which the analysis will be performed. This is accomplished by selecting the desired range on the “Scope” tab. The available options currently perform analysis of the entire workspace, a working set or a set of projects. For the purposes of this article, create a new configuration and on the “Scope” tab, select the “Analyze entire workspace” option.
The “Rules” tab determines the forms of analysis that will be performed. You will notice that the “Rules” tab displays a tree allowing selection/deselection of analysis elements and includes some additional buttons. The top-most nodes of the domain tree are analysis providers, which represent the type of analysis tools that are known to the analysis framework. Providers contain categories, which are loose organizations of rules and/or other categories. Rules perform all the heavy lifting in the analysis process by defining the conditions that generate results during the analysis process.
Note that each node in the tree is prefixed with a checkbox that controls the enabling state of the element. When an element is selected or deselected, all of its children nodes are set to the same state, which allows for quick selection of entire categories or even the entire tree. For the purposes of this article, select the entire “Code Review for Java” branch.
When a rule is selected in the tree, the “Details…” button will become enabled. Clicking this button invokes a secondary dialog to show any configuration and additional information that exists for a rule. Within this dialog you can change the severity rating of the selected rule along with any other parameters it defines. Depending on the rule the details dialog may also show other details on additional tabs.
To start the analysis process, click the “Analyze” button. When you do this you will see the “Analysis Results” view appear in the Eclipse workbench. Results will flow into this view as they are created by the running rules.
Once the results are generated, they can be filtered or viewed, and if the rule supplies a quick fix feature, they can be repaired.
It is important to note that the viewer used to render a result is a function of the type of data it contains. When viewing results you might see a source file opened in the editor with highlighted text, or a UML diagram, or a table of statistical data. There is really no common way to view a result – this is determined by the rule author. For the Java code review analysis provider supplied with TPTP 4.2, all results are viewed as editable Java source files.
If you right-click on a result, you can perform some special tasks. Correcting the cause of some problems is trivial, so the rule author may provide a “quickfix” routine. If the quickfix menu option is enabled, selecting it will walk users through a process to correct the problem.
The context menu also includes an option for filtering. Normally the analysis view filters rules that did not produce results. All results can be displayed or filtered using the “Show All/Show Results Only” option.
In addition to the rules supplied by TPTP and any rules contributed by third party developers, it is also possible for end users to create custom categories and custom rules from templates. To create new custom rules and categories go to the Eclipse preference pages by selecting the “Window->Preferences option. Then in the preference tree select the “Analysis->Custom Rules and Categories” page.
Click the “Add Category…” button to add a new custom category. This takes you through a simple wizard where you can choose the parent category and a name for the new one. The tree control for custom categories will show the complete path of categories for any custom category. Note that only previously defined custom categories can be deleted.
Click the “Add Rule…” button to start the rule creation wizard. The first wizard screen allows the user to select where in the analysis category tree the rule will be located. On the first wizard page, select a category and click the “Next>” button.
On the second wizard page you will see a list of all know rule templates. You can select the rule template you wish to use as the basis for your new rule. Note that not all forms of analysis support custom rules. However TPTP Java Code Review supplies several that are at your disposal.
On the final wizard for rule creation, you will see entries for each parameter defined in the rule template. In this example the selected rule template defined only one parameter, so we can enter only a qualified class name in the field provided.
When the OK button is selected the template-based rule will be created and placed into the rule tree. The user is now free to select this rule as part of any analysis configuration.
In part 1 of this tutorial, we looked at the common static analysis framework provided by TPTP. It offers a consistent user interface for selecting the type of analysis to perform and presents results from all forms in a common viewer.
In part 2 of this series, we will take a closer look at the Java code review capabilities in TPTP 4.2. This will include an in-depth study of basic rule authoring using the supplied API and more advanced features you can use to create rule templates or rules with variable data.