Unleashing the Power of Refactoring

Introduction

The Java Development Tooling (JDT) as part of the Eclipse top-level project provides a rich set of automated refactorings. It includes basic refactorings such as safe rename and move refactorings, advanced refactorings like "Extract Method" or "Extract Superclass", and complex refactorings to be performed across large workspaces such as "Use Supertype" or "Infer Type Arguments". However, for specific tasks that have to be repeated over and over again, writing your own refactoring may be a viable solution to automate tedious code rewriting processes in your development chain.

Recent trends in the Java refactoring tooling have shown an increased awareness of API code, which in general must not be changed by manual rewriting nor by automated refactorings. The "Change Method Signature" refactoring offers the option to generate a delegate method to preserve API compatibility, as do the safe move or rename refactorings. A recently introduced refactoring called "Introduce Indirection" serves a similar purpose, except that this refactoring may be used to alter the way a binary API is used in client code. It is applicable to method invocations and rewrites them to use a new indirection method forwarding to the original API. Such indirection methods can be used to add additional checks before calling external APIs, log information to the console or help adopting new API by implementing any necessary glue code.

The "Introduce Indirection" refactoring performed on the method A#foo() in Figure 1 first searches for all method invocations to foo. The only occurrence can be found in the constructor of class B. The refactoring inserts the new indirection method indirection(A) which delegates to the original foo method. The method invocation to foo in constructor B() is redirected to call the new indirection method.

In this article, we will combine the richness of the JDT tooling API with the power of the extensible services provided by the LTK refactoring framework to implement an "Introduce Indirection" refactoring from scratch. Readers will become familiar with the architecture of refactorings, refactoring history integration, refactoring scripting support and Java-specific facilities such as searching the Java workspace or rewriting existing Java code.

Runtime Requirements

Here is a list of requirements for running the example refactoring in this article:

• J2SE 5 Java Runtime Environment
• Eclipse SDK 3.2 or higher

The example code discussed in this article is available here.

Writing your own Java Refactoring

In general, implementing a refactoring is not an easy task. The specific situation will present many problems that need to be overcome in addition to the refactoring design itself. For example, it is common to find that there are workspaces that do not compile without errors, problems adhering to the programming language rules and errors in user input. Covering all these issues in this article is not feasible; however, we will provide a high-level overview of how to implement a refactoring. The example refactoring developed in this article is fully functional, but lacks thorough error handling, some advanced precondition checking and semantic shift analysis.

In the first part of the article, we will identify requirements our example refactoring must adhere to. In the second part, we will describe the refactoring architecture as implemented by the LTK Refactoring plug-ins. The most commonly used ingredients for a refactoring are discussed and explained in detail. The third part presents the implementation of the example refactoring "Introduce Indirection". We will demonstrate the basic services offered by the JDT tooling and the LTK Refactoring toolkit using code snippets of the source code from the example refactoring.

Performing some Requirements Analysis

Before diving into coding a full-blown Java refactoring, it may be advantageous to think about what properties and capabilities our example refactoring should have. We first identify a series of functional requirements:

• Refactoring Implementation: The "Introduce Indirection" refactoring should implement a refactoring to replace all method invocations to a certain method by a call to a new static method declared in an arbitrary type. This static method is called the indirection method in the following discussions and takes a single argument of the common super type of the receivers of all method invocations. Its implementation simply forwards to the original method as default.
• Refactoring User Interface: The example refactoring should offer a basic refactoring wizard which provides facilities to enter the necessary input used to initialize the refactoring object.
• Precondition Checking: The example refactoring should perform some basic precondition checking such as asserting an error-free workspace and valid user input. This includes checking for existing Java methods and types, as well as a correct new name for the new indirection method.
• Preservation of Semantics: The refactoring must preserve the semantics of the refactored code according to the Java programming language. In particular, this includes correct changing of import declarations and resolving all visibility-related issues.
• Refactoring History Integration: The example refactoring should be seamlessly integrated into the refactoring history of the Eclipse workspace. Executing the "Introduce Indirection" refactoring must result in a refactoring descriptor which is persisted in the global refactoring history by the refactoring framework.
• Refactoring Scripting: The refactoring should be executable using refactoring scripts. Its implementation must be designed to allow object creation and initialization to occur at different points of time. We have to contribute a refactoring contribution object which allows the refactoring framework to dynamically instantiate our example refactoring.

The necessary framework and services needed to implement the identified requirements will be discussed in the next part of this article. Besides the functional requirements, we also list some non-functional requirements which lead the implementation design of the refactoring:

• Java Compilation: The refactoring must hold at most one AST at a time in memory. This ensures scalability of the refactoring implementation when executing the refactoring on large workspaces.
• Refactoring Performance: The refactoring implementation should be designed with performance and low resource needs in mind. This implies careful usage of the Java model, the Java search and opening of Java files during change generation.
• Storing of State: The example refactoring should not store any intermediate state except for change information. Intermediate state may raise life-cycle problems when refactorings are executed using a refactoring wizard user interface.

The above requirements will form the basis of the discussions that follow in this article. We start by giving a quick overview of the design and architecture of refactorings.

Behind the Scenes of Refactorings: Architecture and Design

Automated refactorings implemented for the Eclipse Platform benefit from a powerful refactoring framework supplied by the plug-in org.eclipse.ltk.core.refactoring and its user-interface counterpart org.eclipse.ltk.ui.refactoring. This refactoring framework provides the necessary infrastructure to contribute your refactoring to the refactoring history, the refactoring scripting facility and the Eclipse workbench itself. It comes with services to reliably execute a refactoring on a local workspace, taking care of the details related to precondition checking, change creation and change validation. The refactoring framework user-interface provides abstract implementations of refactoring wizards, refactoring input pages and will show precondition checking errors and change previews.

Here are the most common components that need to be implemented for a new refactoring:

Refactoring Class: The refactoring class is the principal component of a refactoring and implements most of the refactoring-specific functionality. It is required to extend the abstract class org.eclipse.ltk.core.refactoring.Refactoring. For a quick overview of refactoring participants please see The Language Toolkit: An API for Automated Refactorings in Eclipse-based IDEs

Most of the implementation of a refactoring is distributed among the following three template methods of class Refactoring:

checkInitialConditions(IProgressMonitor): This method is called when launching the refactoring and used to implement basic activation checking. Typically, checkInitialConditions confirms that the workspace to be refactored appears to be in a consistent state. In our case, the checkInitialConditions method of the example "Introduce Indirection" refactoring checks for the existence of the compilation unit containing the method to introduce an indirection and confirms that its Java model structure can be determined. It returns a status object of type org.eclipse.ltk.core.refactoring.RefactoringStatus. A refactoring status is used to communicate the result of the precondition checking process to the refactoring execution framework. A status of severity RefactoringStatus#FATAL terminates the refactoring because basic preconditions have not been satisfied.

Implementations of checkInitialConditions should be short-running, since the result of the initial condition checking may determine the behavior of the refactoring user-interface on startup.

Method checkInitialConditions may be called more than once, depending on the refactoring activation process in the user-interface!

checkFinalConditions(IProgressMonitor): This method is called after checkInitialConditions, once the user has provided all necessary inputs to the refactoring. Implementations of checkFinalConditions are usually long-running and perform all remaining precondition checks before change generation. In most cases, in addition to further precondition checking, checkFinalConditions also collects the necessary data to, later on, facilitate change generation. The reason for this is that final precondition checking almost always performs the same computations as change generation later does as well. In our case, the checkFinalConditions method of the example "Introduce Indirection" refactoring performs the remaining precondition checks, searches for references to the input method and rewrites all compilation units where a reference to the input method has been found. The result of the rewriting process is stored as a set of change descriptions to be retrieved later during change generation.

Returning a status of severity RefactoringStatus#FATAL terminates the precondition checking and presents the unsatisfied conditions to the user. In this case, of course the refactoring preconditions need to be met in order to go on to the next steps.

Method checkFinalConditions may be called more than once depending on the user-interface of the refactoring and its interaction with the refactoring class, but always after all calls to checkInitialConditions and before the invocation of createChange!

createChange(IProgressMonitor): This method is called after all preconditions have been checked and the refactoring framework has not detected any unsatisfied condition resulting in a fatal error status. In most instances, createChange returns a change object of type org.eclipse.ltk.core.refactoring.Change, based on the pre-computed information from checkFinalConditions. The change object is used by the refactoring user-interface to generate a preview of the change, and by the core refactoring framework to apply the recorded change to the workspace. After createChange has been called and the resulting change has been applied to the workspace, the refactoring is considered terminated.

Refactoring Descriptor (optional): Refactoring descriptors are mementos that capture the properties of a specific refactoring instance in order to uniquely describe that particular instance. A refactoring descriptor extends the abstract class org.eclipse.ltk.core.refactoring.RefactoringDescriptor and stores vital information such as a unique refactoring id, a timestamp, a human-readable description and further data which is specific to a certain refactoring. The id of a refactoring descriptor allows the refactoring framework to distinguish various types of refactorings. Implementing the method createRefactoring(RefactoringStatus) will return a fully configured refactoring instance indicating that all necessary input has been provided to the refactoring and that the returned instance is ready to be executed by the framework. In our case, the refactoring descriptor for the "Introduce Indirection" refactoring stores information about the input method, the type declaring the new indirection method, the name of the indirection method and a flag indicating whether or not to update references (Figure 2).

Refactoring descriptors are obtained by the refactoring framework by calling Change#getDescriptor() as soon as a change has been applied to the workspace. Optionally, this method returns an org.eclipse.ltk.core.refactoring.RefactoringChangeDescriptor object encapsulating the actual refactoring descriptor.

Refactoring Contribution (optional): Refactoring contributions are a means to register refactorings with the core refactoring framework dynamically instantiating a refactoring object. This mechanism is used by the refactoring history service and the refactoring scripting service to recreate a particular, fully configured refactoring instance from a memento such as a refactoring script. Refactoring contributions are registered via the extension point org.eclipse.ltk.core.refactoring.refactoringContributions and must extend the abstract class org.eclipse.ltk.core.refactoring.RefactoringContribution. Refactoring contributions are optional, meaning that a refactoring can be executed without being registered with the refactoring framework using the described mechanism.

A successful implementation of a refactoring contribution must implement the following two template methods:

createDescriptor(String, String, String, String, Map, int): This method is used by the refactoring framework to create a refactoring descriptor based on a memento such as a refactoring script. The format of the fifth method argument, the argument map, is refactoring-specific. In our case, the refactoring contribution for the example refactoring uses a simple key-value pair scheme to store the state of a refactoring instance.

retrieveArgumentsMap(RefactoringDescriptor): This method is used by the refactoring framework to persist refactoring-specific state in a memento such as a refactoring script. Implementations of this method must return an argument map in the same format as it is passed to the method createDescriptor. This pair of methods provides a generic and extensible mechanism to persist and instantiate refactoring descriptors.

Refactorings that are intended to participate in refactoring scripting must therefore provide an appropriate refactoring contribution and register it with the refactoring framework using the extension point org.eclipse.ltk.core.refactoring.refactoringContributions.

Refactoring Wizard: Refactoring wizards are used to present refactorings in the user-interface. A refactoring wizard implementation must extend the abstract class org.eclipse.ltk.ui.refactoring.RefactoringWizard. Refactoring wizards provide all the logic to orchestrate the display of error pages and change preview pages depending on the status of the precondition checking and change generation. A wizard class is required to implement the abstract method addUserInputPages to add refactoring-specific input pages to the refactoring wizard. A new input page is added to the "Introduce Indirection" wizard consisting of a text field to enter the indirection method name, a combo box to specify the type which declares the new indirection method and a checkbox to control whether or not references are updated.

Refactoring Action: Refactoring actions are used to launch the refactoring from the user-interface. Usually, the task of an action consists of listening to selection changes from the workbench selection service and updating its enablement state accordingly. Checking whether or not the action should be rendered in enabled state should happen quickly, which is the reason why we only check for a selected method in our example refactoring action. For a full treatment of the subject please consult Contributing Actions to the Eclipse Workbench.

Deep Dive: Implementing the "Introduce Indirection" Refactoring

The first part of this article provides an overview of all the components that we will implement in our example "Introduce Indirection" refactoring. We will present our example components in the same order as they appeared in the architecture overview. All the necessary source code for the following discussion is provided with this article. See the resources section at the end of this article for further information.

In the Land of Refactorings

The implementation of the refactoring class is located in IntroduceIndirectionRefactoring.java. From line 97 to 105 we declare all fields describing the state of the refactoring. First, we declare a map of type Map<ICompilationUnit, TextFileChange> which later on is used to store already computed change objects. Further, we store the refactoring inputs identified in Figure 2 in corresponding instance variables of the refactoring class. Obviously, we also declare the necessary accessor methods to be used by the refactoring wizard to set up the refactoring according to the user input.

The method checkInitialConditions(IProgressMonitor) is fairly simple in our case. The only input that has to be set at this point in time is the method handle. The other input elements are not known yet. On line 222 we test whether the method handle has been correctly set, and test for its existence on line 224. Finally, we also check whether the method handle represents a binary method and whether the declaring compilation unit is in a reasonably well-formed state (line 227). In case one of the above conditions is not satisfied, we return a refactoring status with fatal severity to terminate the refactoring immediately. Otherwise we return a new refactoring status with default severity RefactoringStatus#OK to signal the refactoring framework to proceed with the execution of the refactoring.

The second method which implements precondition checking, checkFinalConditions(IProgressMonitor), is somewhat more complex. This is discussed in more detail earlier in this article: the computations for precondition checking and change generation intersect to a fair degree. From line 134 to 180 we search for all references to the input method and group the resulting search matches by project and compilation unit.

The ASTParser creates a new AST with resolved bindings for every compilation unit passed to the API described above. The refactoring obtains the ASTs via an ASTRequestor, one AST at a time.

The AST requestor then delegates the precondition checking and change generation to the method rewriteCompilationUnit(ASTRequestor, ICompilationUnit, Collection, CompilationUnit, RefactoringStatus) which implements further precondition checking and rewrites the obtained AST.

Method rewriteCompilationUnit coordinates the rewriting process by deciding what to rewrite in which compilation unit. If the compilation unit happens to be the one which declares the declaring type of the new indirection method (line 511), we call rewriteDeclaringType(ASTRequestor, ASTRewrite, ImportRewrite, ICompilationUnit, CompilationUnit) to insert the new indirection method into the existing type declaration. Next, we check whether references to the input method have to be updated (line 513). If yes, we try to locate the search matches in the AST and call rewriteMethodInvocation(ASTRequestor, ASTRewrite, ImportRewrite, MethodInvocation). If successful, we invoke rewriteAST(ICompilationUnit, ASTRewrite, ImportRewrite) to rewrite the AST and store a description of the change to be executed on the compilation unit. If not, we call rewriteAST as well, but immediately return without rewriting any method invocations.

Let's have a quick look at method rewriteDeclaringType. We need to acquire a binding of the input method. The API ASTRequestor#createBindings(String[]) can be used for this purpose.

The remaining code of this method is straight-forward. First, we assemble the new method declaration for the indirection method. On line 552 we add the necessary modifiers to declare the method as "public static". Then we check whether the input method is declared as "static". If this is the case, we have to insert one additional method argument which represents the target of the (not yet redirected) method call. Further, if the declaring type of the input method is generic, any type arguments of enclosing types have to be added as well.

On lines 574 to 611 we copy the method arguments, type parameters and exceptions, create the body of the method declaration with one method invocation statement that implements the actual indirection and construct a method comment according to the project preferences.

Finally, on lines 613 to 616 we insert the newly created method declaration into the type declaration.

The method rewriteMethodInvocation is somewhat simpler. On lines 623 to 627 we use the same pattern as described in Figure 6 to obtain a binding for the declaring type of the new indirection method. Contrary to rewriteDeclaringType, this method not only performs AST rewriting but also some precondition checking. We check whether the original method invocation has some type arguments. In this case we have to skip the method invocation. Instead of rewriting it, we return a refactoring status object with a warning severity explaining the reason why this occurrence has not been rewritten.

Method invocations with "this" expression receivers need some further attention. The code on lines 648 to 670 handles such cases and inserts the necessary qualifications for enclosing types. On line 673, we move the old method arguments to the argument list of the newly created method invocation.

Finally, we replace the old method invocation with the new one and return the refactoring status which has been computed during precondition checking (line 675).

The method rewriteAST implements the actual AST rewriting process. It is used exactly once per compilation unit and creates a text edit tree based on the recorded changes on the AST. First, we call the API ASTRewrite#rewriteAST to obtain the text edit tree corresponding to the modifications recorded on the AST. Next, we call the API ImportRewrite#rewriteImports(IProgressMonitor) to obtain the text edit tree capturing the import declaration changes. Since these text edit trees are disjoint by design, we can simply merge them and create an org.eclipse.ltk.core.refactoring.TextFileChange object from the resulting multi text edit. On line 493 we also set the text type of the change to "java".

The resulting change object is stored to be later picked up by the change generation implemented in method createChange(IProgressMonitor). Resource-wise, these change objects are quite cheap to keep in memory, since they only encapsulate a text edit tree which describes the modifications to be performed on the underlying compilation unit buffer.

There is not much left to do for the method createChange. All change information has already been gathered and is ready to be encapsulated by a composite changed object (line 326). The role of the getDescriptor() method that is overridden for the returned change object is described in the next section on refactoring scripting.

Meet Refactoring History and Scripting Services

In the previous section we have provided an overview of the basic functionality of the example refactoring. In this section we will show how to integrate the refactoring with the refactoring core framework, the refactoring history and refactoring scripting services!

In order to let the refactoring framework persist the refactoring instance in the refactoring history, we simply override the getDescriptor() method of the change object being returned by createChange(). The method getDescriptor() returns an org.eclipse.ltk.core.refactoring.RefactoringChangeDescriptor object encapsulating a custom refactoring descriptor. Its implementation can be found in IntroduceIndirectionDescriptor.java.

The constructor of the refactoring descriptor takes a project name, a human-readable description, a comment and an argument map as arguments. In case the refactoring could not be uniquely associated with a single project, we may pass null as project name. The argument map is a simple dictionary with String-typed key-value pairs capturing the four refactoring input elements listed in Figure 9. The constructor of the base class org.eclipse.ltk.core.refactoring.RefactoringDescriptor takes an additional refactoring id ("net.eclipsemag.introduce.indirection") and refactoring descriptor flags (RefactoringDescriptor.STRUCTURAL_CHANGE | RefactoringDescriptor.MULTI_CHANGE) which indicate that the refactoring may causes structural changes to the Java workspace which can span multiple files. This information is used by the refactoring history service to provide categorization of refactorings and optimize query times for context-specific history queries. The factory method createRefactoring(RefactoringStatus) instantiates a new refactoring object, calls initialize(Map) to set the input of the refactoring based on the data from the refactoring descriptor and returns the fully configured refactoring instance which is ready to be executed.

The second component necessary to integrate the refactoring into the refactoring framework is a refactoring contribution. Its implementation resides in file IntroduceIndirectionRefactoringContribution.java. The factory method createDescriptor returns a new IntroduceIndirectionDescriptor object initialized with the specified arguments. Nothing more is needed to dynamically instantiate a refactoring instance!

The second method to be re-implemented is retrieveArgumentMap(RefactoringDescriptor). We test whether the passed refactoring descriptor is indeed an IntroduceIndirectionDescriptor (line 17). If it is, we only have to return its argument map. Otherwise, we pass the ball to the super implementation. With these two small additions, our example refactorings is able to be automatically recorded and eventually replayed on any arbitrary workspace.

Presenting the Refactoring to the User

So far, we have discussed the refactoring implementation which is necessary to execute the refactoring head-less, without any user interaction. The primary focus of this article is the design and implementation of Java refactorings. We do not describe in detail how refactoring user-interface are constructed. More information can be found by consulting any of the Eclipse Corner SWT articles listed in the resources section of this article.

The user input needed to execute the refactoring has been identified in Figure 2. We just contribute a UserInputWizardPage with the necessary SWT widgets to let the user provide this input. The code for the refactoring wizard and its input page can be found in IntroduceIndirectionWizard.java and IntroduceIndirectionInputPage.java, respectively.

The only step which is missing now is the action to launch the refactoring from the workbench. For our example refactoring we use an IWorkbenchWindowActionDelegate for the sake of simplicity. The code of this action delegate can be found in IntroduceIndirectionAction.java. Such a delegate can be easily registered with the extension point "org.eclipse.ui.actionSets". The following snippet shows how to do this:

The snippet displayed in Figure 11 is the last step towards a fully functional "Introduce Indirection" refactoring. Having arrived here, we have now implemented an advanced Java refactoring with a clean user-interface, high rewriting performance and full refactoring history and refactoring scripting support!

Running the Introduce Indirection Example Refactoring

Test driving the example refactoring implemented in this article takes just a few simple steps. Unzip the code download for this article to your Eclipse workspace. Import the unzipped code as a Java project called "net.eclipsemag.refactoring". Follow the next four steps and explore the result!

• Create a new Eclipse Application and launch it
• Import some Java code into the workspace of the Eclipse Application
• Select a Java method in the editor outline or the Package Explorer
• Invoke Introduce Indirection... from the Eclipse Articles main menu

Summary

The article described an example refactoring implementation for an "Introduce Indirection" refactoring. It walks the reader through the architecture and design of refactorings, discusses implementation details and provides guidance on implementing a Java refactoring using the APIs offered by the JDT project and the LTK Refactoring framework. In particular, it focuses on common design principles and refactoring architecture. We show how a refactoring is designed from scratch using a proven methodology, we explain how a self-written refactoring can participate in the workbench refactoring history and the refactoring scripting service and we present a user-interface to interact with the refactoring. Finally, we give a simplified but working implementation of an "Introduce Indirection" refactoring to prove the viability of the refactoring design described in this article.

Acknowledgements

Thanks go to Dirk Bäumer and Bernd Kolb for valuable comments and suggestions on an initial draft of this article.

Resources

• Eclipse Project Homepage
• Eclipse JDT Project Homepage
• The Language Toolkit: An API for Automated Refactorings in Eclipse-based IDEs
• Contributing Actions to the Eclipse Workbench
• Eclipse Corner SWT Articles
• Code Download for this article