Unleashing the Power of Refactoring
Refactorings as a tool to automate behavior-preserving transformations to source code are not only very popular in agile development environments, but have been widely established as a cornerstone of the daily software development process, regardless of the methodology being used. Most major development environments such as Eclipse offer a set of powerful refactorings to substantially increase development productivity. While built-in refactorings perform most of the tedious and error-prone tasks such as renaming or moving software elements, sometimes the need arises to develop a custom solution to streamline repetitive tasks. It is time to write your own refactoring!
In this article, Tobias Widmer sheds light on the services offered by the Eclipse Java Development Tools (JDT) and the Refactoring Language Toolkit (LTK) to support automated Java™ refactorings, explains how these services are used by refactorings to perform searches on the Java workspace, rewrite existing code and provide a rich user-interface to present the results of the refactoring. To demonstrate this combination of Java-specific and language-neutral frameworks, this article presents a simple but working refactoring implementation for an 'Introduce Indirection' refactoring designed to introduce an indirection method for existing method invocations.
The Java Development Tooling (JDT) as part of the Eclipse top-level project provides a rich set of automated refactorings. It includes basic refactorings such as safe rename and move refactorings, advanced refactorings like "Extract Method" or "Extract Superclass", and complex refactorings to be performed across large workspaces such as "Use Supertype" or "Infer Type Arguments". However, for specific tasks that have to be repeated over and over again, writing your own refactoring may be a viable solution to automate tedious code rewriting processes in your development chain.
Recent trends in the Java refactoring tooling have shown an increased awareness of API code, which in general must not be changed by manual rewriting nor by automated refactorings. The "Change Method Signature" refactoring offers the option to generate a delegate method to preserve API compatibility, as do the safe move or rename refactorings. A recently introduced refactoring called "Introduce Indirection" serves a similar purpose, except that this refactoring may be used to alter the way a binary API is used in client code. It is applicable to method invocations and rewrites them to use a new indirection method forwarding to the original API. Such indirection methods can be used to add additional checks before calling external APIs, log information to the console or help adopting new API by implementing any necessary glue code.
The "Introduce Indirection" refactoring performed on
A#foo() in Figure 1 first
searches for all method invocations to
foo. The only
occurrence can be found in the constructor of class
refactoring inserts the new indirection method
which delegates to the original
foo method. The method
foo in constructor
redirected to call the new indirection method.
In this article, we will combine the richness of the JDT tooling API with the power of the extensible services provided by the LTK refactoring framework to implement an "Introduce Indirection" refactoring from scratch. Readers will become familiar with the architecture of refactorings, refactoring history integration, refactoring scripting support and Java-specific facilities such as searching the Java workspace or rewriting existing Java code.
Here is a list of requirements for running the example refactoring in this article:
- J2SE 5 Java Runtime Environment
- Eclipse SDK 3.2 or higher
The example code discussed in this article is available here.
Writing your own Java Refactoring
In general, implementing a refactoring is not an easy task. The specific situation will present many problems that need to be overcome in addition to the refactoring design itself. For example, it is common to find that there are workspaces that do not compile without errors, problems adhering to the programming language rules and errors in user input. Covering all these issues in this article is not feasible; however, we will provide a high-level overview of how to implement a refactoring. The example refactoring developed in this article is fully functional, but lacks thorough error handling, some advanced precondition checking and semantic shift analysis.
In the first part of the article, we will identify requirements our example refactoring must adhere to. In the second part, we will describe the refactoring architecture as implemented by the LTK Refactoring plug-ins. The most commonly used ingredients for a refactoring are discussed and explained in detail. The third part presents the implementation of the example refactoring "Introduce Indirection". We will demonstrate the basic services offered by the JDT tooling and the LTK Refactoring toolkit using code snippets of the source code from the example refactoring.
Performing some Requirements Analysis
Before diving into coding a full-blown Java refactoring, it may be advantageous to think about what properties and capabilities our example refactoring should have. We first identify a series of functional requirements:
- Refactoring Implementation: The "Introduce Indirection" refactoring should implement a refactoring to replace all method invocations to a certain method by a call to a new static method declared in an arbitrary type. This static method is called the indirection method in the following discussions and takes a single argument of the common super type of the receivers of all method invocations. Its implementation simply forwards to the original method as default.
- Refactoring User Interface: The example refactoring should offer a basic refactoring wizard which provides facilities to enter the necessary input used to initialize the refactoring object.
- Precondition Checking: The example refactoring should perform some basic precondition checking such as asserting an error-free workspace and valid user input. This includes checking for existing Java methods and types, as well as a correct new name for the new indirection method.
- Preservation of Semantics: The refactoring must preserve the semantics of the refactored code according to the Java programming language. In particular, this includes correct changing of import declarations and resolving all visibility-related issues.
- Refactoring History Integration: The example refactoring should be seamlessly integrated into the refactoring history of the Eclipse workspace. Executing the "Introduce Indirection" refactoring must result in a refactoring descriptor which is persisted in the global refactoring history by the refactoring framework.
- Refactoring Scripting: The refactoring should be executable using refactoring scripts. Its implementation must be designed to allow object creation and initialization to occur at different points of time. We have to contribute a refactoring contribution object which allows the refactoring framework to dynamically instantiate our example refactoring.
The necessary framework and services needed to implement the identified requirements will be discussed in the next part of this article. Besides the functional requirements, we also list some non-functional requirements which lead the implementation design of the refactoring:
- Java Compilation: The refactoring must hold at most one AST at a time in memory. This ensures scalability of the refactoring implementation when executing the refactoring on large workspaces.
- Refactoring Performance: The refactoring implementation should be designed with performance and low resource needs in mind. This implies careful usage of the Java model, the Java search and opening of Java files during change generation.
- Storing of State: The example refactoring should not store any intermediate state except for change information. Intermediate state may raise life-cycle problems when refactorings are executed using a refactoring wizard user interface.
The above requirements will form the basis of the discussions that follow in this article. We start by giving a quick overview of the design and architecture of refactorings.
Behind the Scenes of Refactorings: Architecture and Design
Automated refactorings implemented for the Eclipse Platform
benefit from a powerful refactoring framework supplied by the plug-in
and its user-interface counterpart
This refactoring framework provides the necessary infrastructure to
contribute your refactoring to the refactoring history, the refactoring
scripting facility and the Eclipse workbench itself. It comes with
services to reliably execute a refactoring on a local workspace, taking
care of the details related to precondition checking, change creation
and change validation. The refactoring framework user-interface provides
abstract implementations of refactoring wizards, refactoring input pages
and will show precondition checking errors and change previews.
Here are the most common components that need to be implemented for a new refactoring:
Refactoring Class: The refactoring class is the principal
component of a refactoring and implements most of the
refactoring-specific functionality. It is required to extend the
For a quick overview of refactoring participants please see The
Language Toolkit: An API for Automated Refactorings in Eclipse-based
Most of the implementation of a refactoring is distributed among
the following three template methods of class
checkInitialConditions(IProgressMonitor): This method is called when launching the refactoring and used to implement basic activation checking. Typically,
checkInitialConditionsconfirms that the workspace to be refactored appears to be in a consistent state. In our case, the
checkInitialConditionsmethod of the example "Introduce Indirection" refactoring checks for the existence of the compilation unit containing the method to introduce an indirection and confirms that its Java model structure can be determined. It returns a status object of type
org.eclipse.ltk.core.refactoring.RefactoringStatus. A refactoring status is used to communicate the result of the precondition checking process to the refactoring execution framework. A status of severity
RefactoringStatus#FATALterminates the refactoring because basic preconditions have not been satisfied.
checkInitialConditionsshould be short-running, since the result of the initial condition checking may determine the behavior of the refactoring user-interface on startup.
checkFinalConditions(IProgressMonitor): This method is called after
checkInitialConditions, once the user has provided all necessary inputs to the refactoring. Implementations of
checkFinalConditionsare usually long-running and perform all remaining precondition checks before change generation. In most cases, in addition to further precondition checking,
checkFinalConditionsalso collects the necessary data to, later on, facilitate change generation. The reason for this is that final precondition checking almost always performs the same computations as change generation later does as well. In our case, the
checkFinalConditionsmethod of the example "Introduce Indirection" refactoring performs the remaining precondition checks, searches for references to the input method and rewrites all compilation units where a reference to the input method has been found. The result of the rewriting process is stored as a set of change descriptions to be retrieved later during change generation.
Returning a status of severity
RefactoringStatus#FATALterminates the precondition checking and presents the unsatisfied conditions to the user. In this case, of course the refactoring preconditions need to be met in order to go on to the next steps.
createChange(IProgressMonitor): This method is called after all preconditions have been checked and the refactoring framework has not detected any unsatisfied condition resulting in a fatal error status. In most instances,
createChangereturns a change object of type
org.eclipse.ltk.core.refactoring.Change, based on the pre-computed information from
checkFinalConditions. The change object is used by the refactoring user-interface to generate a preview of the change, and by the core refactoring framework to apply the recorded change to the workspace. After
createChangehas been called and the resulting change has been applied to the workspace, the refactoring is considered terminated.
Refactoring Descriptor (optional): Refactoring descriptors
are mementos that capture the properties of a specific refactoring
instance in order to uniquely describe that particular instance. A
refactoring descriptor extends the abstract class
and stores vital information such as a unique refactoring id, a
timestamp, a human-readable description and further data which is
specific to a certain refactoring. The id of a refactoring descriptor
allows the refactoring framework to distinguish various types of
refactorings. Implementing the method
will return a fully configured refactoring instance indicating that all
necessary input has been provided to the refactoring and that the
returned instance is ready to be executed by the framework. In our case,
the refactoring descriptor for the "Introduce Indirection"
refactoring stores information about the input method, the type
declaring the new indirection method, the name of the indirection method
and a flag indicating whether or not to update references (Figure 2).
Refactoring descriptors are obtained by the refactoring framework
Change#getDescriptor() as soon as a change has
been applied to the workspace. Optionally, this method returns an
object encapsulating the actual refactoring descriptor.
Refactoring Contribution (optional): Refactoring
contributions are a means to register refactorings with the core
refactoring framework dynamically instantiating a refactoring object.
This mechanism is used by the refactoring history service and the
refactoring scripting service to recreate a particular, fully configured
refactoring instance from a memento such as a refactoring script.
Refactoring contributions are registered via the extension point
and must extend the abstract class
Refactoring contributions are optional, meaning that a refactoring can
be executed without being registered with the refactoring framework
using the described mechanism.
A successful implementation of a refactoring contribution must implement the following two template methods:
createDescriptor(String, String, String, String, Map, int): This method is used by the refactoring framework to create a refactoring descriptor based on a memento such as a refactoring script. The format of the fifth method argument, the argument map, is refactoring-specific. In our case, the refactoring contribution for the example refactoring uses a simple key-value pair scheme to store the state of a refactoring instance.
retrieveArgumentsMap(RefactoringDescriptor): This method is used by the refactoring framework to persist refactoring-specific state in a memento such as a refactoring script. Implementations of this method must return an argument map in the same format as it is passed to the method
createDescriptor. This pair of methods provides a generic and extensible mechanism to persist and instantiate refactoring descriptors.
Refactoring Wizard: Refactoring wizards are used to
present refactorings in the user-interface. A refactoring wizard
implementation must extend the abstract class
Refactoring wizards provide all the logic to orchestrate the display of
error pages and change preview pages depending on the status of the
precondition checking and change generation. A wizard class is required
to implement the abstract method
addUserInputPages to add
refactoring-specific input pages to the refactoring wizard. A new input
page is added to the "Introduce Indirection" wizard consisting
of a text field to enter the indirection method name, a combo box to
specify the type which declares the new indirection method and a
checkbox to control whether or not references are updated.
Refactoring Action: Refactoring actions are used to launch the refactoring from the user-interface. Usually, the task of an action consists of listening to selection changes from the workbench selection service and updating its enablement state accordingly. Checking whether or not the action should be rendered in enabled state should happen quickly, which is the reason why we only check for a selected method in our example refactoring action. For a full treatment of the subject please consult Contributing Actions to the Eclipse Workbench.
Deep Dive: Implementing the "Introduce Indirection" Refactoring
The first part of this article provides an overview of all the components that we will implement in our example "Introduce Indirection" refactoring. We will present our example components in the same order as they appeared in the architecture overview. All the necessary source code for the following discussion is provided with this article. See the resources section at the end of this article for further information.
In the Land of Refactorings
The implementation of the refactoring class is located in
From line 97 to 105 we declare all fields describing the state of the
refactoring. First, we declare a map of type
TextFileChange> which later on is used to store already computed
change objects. Further, we store the refactoring inputs identified in Figure 2 in corresponding instance variables of the
refactoring class. Obviously, we also declare the necessary accessor
methods to be used by the refactoring wizard to set up the refactoring
according to the user input.
|Refactoring Inputs||Java Type|
|Indirection Method Name||
|Declaring Type Handle||
|Update References Flag||
is fairly simple in our case. The only input that has to be set at this
point in time is the method handle. The other input elements are not
known yet. On line 222 we test whether the method handle has been
correctly set, and test for its existence on line 224. Finally, we also
check whether the method handle represents a binary method and whether
the declaring compilation unit is in a reasonably well-formed state
(line 227). In case one of the above conditions is not satisfied, we
return a refactoring status with fatal severity to terminate the
refactoring immediately. Otherwise we return a new refactoring status
with default severity
RefactoringStatus#OK to signal the refactoring
framework to proceed with the execution of the refactoring.
The second method which implements precondition checking,
checkFinalConditions(IProgressMonitor), is somewhat more complex. This
is discussed in more detail earlier in this article: the computations
for precondition checking and change generation intersect to a fair
degree. From line 134 to 180 we search for all references to the input
method and group the resulting search matches by project and compilation
ASTParser creates a new
AST with resolved bindings for every
compilation unit passed to the API described above. The refactoring
obtains the ASTs via an
ASTRequestor, one AST at a time.
The AST requestor then delegates the precondition checking and
change generation to the method
ICompilationUnit, Collection, CompilationUnit, RefactoringStatus) which
implements further precondition checking and rewrites the obtained AST.
rewriteCompilationUnit coordinates the rewriting process
by deciding what to rewrite in which compilation unit. If the
compilation unit happens to be the one which declares the declaring type
of the new indirection method (line 511), we call
rewriteDeclaringType(ASTRequestor, ASTRewrite, ImportRewrite,
ICompilationUnit, CompilationUnit) to insert the new indirection method
into the existing type declaration. Next, we check whether references to
the input method have to be updated (line 513). If yes, we try to locate
the search matches in the AST and call
rewriteMethodInvocation(ASTRequestor, ASTRewrite, ImportRewrite,
MethodInvocation). If successful, we invoke
ASTRewrite, ImportRewrite) to rewrite the AST and store a description of
the change to be executed on the compilation unit. If not, we call
rewriteAST as well, but immediately return without rewriting any method
Let's have a quick look at method
rewriteDeclaringType. We need
to acquire a binding of the input method. The API
ASTRequestor#createBindings(String) can be used for this purpose.
The remaining code of this method is straight-forward. First, we assemble the new method declaration for the indirection method. On line 552 we add the necessary modifiers to declare the method as "public static". Then we check whether the input method is declared as "static". If this is the case, we have to insert one additional method argument which represents the target of the (not yet redirected) method call. Further, if the declaring type of the input method is generic, any type arguments of enclosing types have to be added as well.
On lines 574 to 611 we copy the method arguments, type parameters and exceptions, create the body of the method declaration with one method invocation statement that implements the actual indirection and construct a method comment according to the project preferences.
Finally, on lines 613 to 616 we insert the newly created method declaration into the type declaration.
rewriteMethodInvocation is somewhat simpler. On lines
623 to 627 we use the same pattern as described in Figure 6 to obtain a
binding for the declaring type of the new indirection method. Contrary
rewriteDeclaringType, this method not only performs AST rewriting but
also some precondition checking. We check whether the original method
invocation has some type arguments. In this case we have to skip the
method invocation. Instead of rewriting it, we return a refactoring
status object with a warning severity explaining the reason why this
occurrence has not been rewritten.
Method invocations with "this" expression receivers need some further attention. The code on lines 648 to 670 handles such cases and inserts the necessary qualifications for enclosing types. On line 673, we move the old method arguments to the argument list of the newly created method invocation.
Finally, we replace the old method invocation with the new one and return the refactoring status which has been computed during precondition checking (line 675).
rewriteAST implements the actual AST rewriting
process. It is used exactly once per compilation unit and creates a text
edit tree based on the recorded changes on the AST. First, we call the
ASTRewrite#rewriteAST to obtain the text edit tree corresponding to
the modifications recorded on the AST. Next, we call the API
ImportRewrite#rewriteImports(IProgressMonitor) to obtain the text edit
tree capturing the import declaration changes. Since these text edit
trees are disjoint by design, we can simply merge them and create an
org.eclipse.ltk.core.refactoring.TextFileChange object from the
resulting multi text edit. On line 493 we also set the text type of the
change to "java".
The resulting change object is stored to be later picked up by
the change generation implemented in method
createChange(IProgressMonitor). Resource-wise, these change objects are
quite cheap to keep in memory, since they only encapsulate a text edit
tree which describes the modifications to be performed on the underlying
compilation unit buffer.
There is not much left to do for the method
change information has already been gathered and is ready to be
encapsulated by a composite changed object (line 326). The role of the
getDescriptor() method that is overridden for the returned change object
is described in the next section on refactoring scripting.
Meet Refactoring History and Scripting Services
In the previous section we have provided an overview of the basic functionality of the example refactoring. In this section we will show how to integrate the refactoring with the refactoring core framework, the refactoring history and refactoring scripting services!
In order to let the refactoring framework persist the refactoring
instance in the refactoring history, we simply override the
getDescriptor() method of the change object being returned by
createChange(). The method
object encapsulating a custom refactoring descriptor. Its implementation
can be found in
|Refactoring Descriptor Input||Java Type||Values|
|Project Name||String||IProject#getName or null|
|Comment||String||non-empty or null|
|Argument Map||Map<String, String>||non-null|
The constructor of the refactoring descriptor takes a project
name, a human-readable description, a comment and an argument map as
arguments. In case the refactoring could not be uniquely associated with
a single project, we may pass null as project name. The argument map is
a simple dictionary with
String-typed key-value pairs capturing the four
refactoring input elements listed in Figure 9.
The constructor of the base class
org.eclipse.ltk.core.refactoring.RefactoringDescriptor takes an
additional refactoring id
"net.eclipsemag.introduce.indirection") and refactoring
descriptor flags (
RefactoringDescriptor.MULTI_CHANGE) which indicate that the
refactoring may causes structural changes to the Java workspace which
can span multiple files. This information is used by the refactoring
history service to provide categorization of refactorings and optimize
query times for context-specific history queries. The factory method
createRefactoring(RefactoringStatus) instantiates a new refactoring
initialize(Map) to set the input of the refactoring based
on the data from the refactoring descriptor and returns the fully
configured refactoring instance which is ready to be executed.
The second component necessary to integrate the refactoring into
the refactoring framework is a refactoring contribution. Its
implementation resides in file
IntroduceIndirectionRefactoringContribution.java. The factory method
createDescriptor returns a new
initialized with the specified arguments. Nothing more is needed to
dynamically instantiate a refactoring instance!
The second method to be re-implemented is
retrieveArgumentMap(RefactoringDescriptor). We test whether the passed
refactoring descriptor is indeed an
17). If it is, we only have to return its argument map. Otherwise, we
pass the ball to the super implementation. With these two small
additions, our example refactorings is able to be automatically recorded
and eventually replayed on any arbitrary workspace.
Presenting the Refactoring to the User
So far, we have discussed the refactoring implementation which is necessary to execute the refactoring head-less, without any user interaction. The primary focus of this article is the design and implementation of Java refactorings. We do not describe in detail how refactoring user-interface are constructed. More information can be found by consulting any of the Eclipse Corner SWT articles listed in the resources section of this article.
The user input needed to execute the refactoring has been
identified in Figure 2. We just contribute a
UserInputWizardPage with the necessary SWT widgets to let the user
provide this input. The code for the refactoring wizard and its input
page can be found in
The only step which is missing now is the action to launch the
refactoring from the workbench. For our example refactoring we use an
for the sake of simplicity. The code of this action delegate can be
IntroduceIndirectionAction.java. Such a delegate
can be easily registered with the extension point
"org.eclipse.ui.actionSets". The following snippet shows how
to do this:
The snippet displayed in Figure 11 is the last step towards a fully functional "Introduce Indirection" refactoring. Having arrived here, we have now implemented an advanced Java refactoring with a clean user-interface, high rewriting performance and full refactoring history and refactoring scripting support!
Running the Introduce Indirection Example Refactoring
Test driving the example refactoring implemented in this article takes just a few simple steps. Unzip the code download for this article to your Eclipse workspace. Import the unzipped code as a Java project called "net.eclipsemag.refactoring". Follow the next four steps and explore the result!
- Create a new Eclipse Application and launch it
- Import some Java code into the workspace of the Eclipse Application
- Select a Java method in the editor outline or the Package Explorer
- Invoke from the Eclipse Articles main menu
The article described an example refactoring implementation for an "Introduce Indirection" refactoring. It walks the reader through the architecture and design of refactorings, discusses implementation details and provides guidance on implementing a Java refactoring using the APIs offered by the JDT project and the LTK Refactoring framework. In particular, it focuses on common design principles and refactoring architecture. We show how a refactoring is designed from scratch using a proven methodology, we explain how a self-written refactoring can participate in the workbench refactoring history and the refactoring scripting service and we present a user-interface to interact with the refactoring. Finally, we give a simplified but working implementation of an "Introduce Indirection" refactoring to prove the viability of the refactoring design described in this article.
Thanks go to Dirk Bäumer and Bernd Kolb for valuable comments and suggestions on an initial draft of this article.
- Eclipse Project Homepage
- Eclipse JDT Project Homepage
- The Language Toolkit: An API for Automated Refactorings in Eclipse-based IDEs
- Contributing Actions to the Eclipse Workbench
- Eclipse Corner SWT Articles
- Code Download for this article
IBM is a registered trademark of International Business Machines Corporation in the United States, other countries, or both.
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
Other company, product, and service names may be trademarks or service marks of others.