Xpand2

Xpand2

The Xpand language is used in templates to control the output generation. This documentation describes the general syntax and semantics of the Xpand language.

Typing the guillemets (« and ») used in the templates is supported by the Eclipse editor: which provides keyboard shortcuts with Ctrl + < and Ctrl + > .

Templates are stored in files with the extension .xpt . Template files must reside on the Java classpath of the generator process.

Almost all characters used in the standard syntax are part of ASCII and should therefore be available in any encoding. The only limitation are the tag brackets ( guillemets ), for which the characters "«" (Unicode 00AB) and "»" (Unicode 00BB) are used. So for reading templates, an encoding should be used that supports these characters (e.g. ISO-8859-1 or UTF-8).

Names of properties, templates, namespaces etc. must only contain letters, numbers and underscores.

Here is a first example of a template:

«IMPORT meta::model»
«EXTENSION my::ExtensionFile»

«DEFINE javaClass FOR Entity»
   «FILE fileName()»
      package «javaPackage()»;

      public class «name» {
         // implementation
      }
   «ENDFILE»
«ENDDEFINE»

A template file consists of any number of IMPORT statements, followed by any number of EXTENSION statements, followed by one or more DEFINE blocks (called definitions).

The central concept of Xpand is the DEFINE block, also called a template. This is the smallest identifiable unit in a template file. The tag consists of a name, an optional comma-separated parameter list, as well as the name of the metamodel class for which the template is defined.

«DEFINE templateName(formalParameterList) FOR MetaClass»
   a sequence of statements
«ENDDEFINE»

To some extent, templates can be seen as special methods of the metaclass – there is always an implicit this parameter which can be used to address the "underlying" model element; in our example above, this model element is "MetaClass".

As in Java, a formal parameter list entry consists of the type followed by the name of that parameter.

The body of a template can contain a sequence of other statements including any text.

A full parametric polymorphism is available for templates. If there are two templates with the same name that are defined for two metaclasses which inherit from the same superclass, Xpand will use the corresponding subclass template, in case the template is called for the superclass. Vice versa, the template of the superclass would be used in case a subclass template is not available. Note that this not only works for the target type, but for all parameters. Technically, the target type is handled as the first parameter.

So, let us assume you have the following metamodel:


Assume further, you would have a model which contains a collection of A, B and C instances in the property listOfAs. Then, you can write the following template:

«DEFINE someOtherDefine FOR SomeMetaClass»
   «EXPAND implClass FOREACH listOfAs»
«ENDDEFINE»

«DEFINE implClass FOR A»
   // this is the code generated for the superclass A
«ENDDEFINE»

«DEFINE implClass FOR B»
   // this is the code generated for the subclass B
«ENDDEFINE»

«DEFINE implClass FOR C»
   // this is the code generated for the subclass C
«ENDDEFINE»

So for each B in the list, the template defined for B is executed, for each C in the collection the template defined for C is invoked, and for all others (which are then instances of A) the default template is executed.

Using the workflow engine it is now possible to package ( e.g. zip) a written generator and deliver it as a kind of black box. If you want to use such a generator but need to change some small generation stuff, you can make use of the AROUND aspects.

«AROUND qualifiedDefinitionName(parameterList)? FOR type»
   a sequence of statements
«ENDAROUND» 

AROUND lets you add templates in an non-invasive way (you do not need to touch the generator templates). Because aspects are invasive, a template file containing AROUND aspects must be wrapped by configuration (see next section).

AOP is basically about weaving code into different points inside the call graph of a software module. Such points are called Join Points . In Xpand , there is only one join point so far: a call to a definition.

You specify on which join points the contributed code should be executed by specifying something like a 'query' on all available join points. Such a query is called a point cut .

«AROUND [pointcut]»
   do stuff
«ENDAROUND»

A pointcut consists of a fully qualified name, parameter types and the target type.

This section describes the workflow component that is provided to perform the code generation, i.e. run the templates. You should have a basic idea of how the workflow engine works. A simple generator component configuration could look as follows:

<component class="org.eclipse.xpand2.Generator">
   <fileEncoding value="ISO-8859-1"/>
   <metaModel class="org.eclipse.xtend.typesystem.emf.EmfMetaModel">
       <metaModelPackage value="org.eclipse.emf.ecore.EcorePackage"/>
   </metaModel>
   <expand value="example::Java::all FOR myModel"/>

   <!-- aop configuration -->
   <advices value='example::Advices1, example::Advices2'/>

   <!--  output configuration -->
   <outlet path='main/src-gen'/>
   <outlet name='TO_SRC' path='main/src' overwrite='false'/>
   <beautifier class="org.eclipse.xpand2.output.JavaBeautifier"/>
   <beautifier class="org.eclipse.xpand2.output.XmlBeautifier"/>

   <!-- protected regions configuration -->
   <prSrcPaths value="main/src"/>
   <prDefaultExcludes value="false"/>
   <prExcludes value="*.xml"/>
</component>

Now, let us go through the different properties one by one.

Beautifying the generated code is a good idea. It is very important that generated code looks good, because developers should be able to understand it. On the other hand template files should look good, too. It is thus best practice to write nice looking template files and not to care how the generated code looks – and then you run a beautifier over the generated code to fix that problem. Of course, if a beautifier is not available, or if white space has syntactical meaning (as in Python), you would have to write your templates with that in mind (using the minus character before closing brackets as described in a preceding section).

The Xpand workflow component can be configured with multiple beautifiers:

<beautifier
   class="org.eclipse.xpand2.output.JavaBeautifier"/>
<beautifier
   class="org.eclipse.xpand2.output.XMLBeautifier"/>

These are the two beautifiers delivered with Xpand . If you want to use your own beautifier, you would just need to implement the PostProcessor Java interface:

package org.eclipse.xpand2.output;

public interface PostProcessor {
   public void beforeWriteAndClose(FileHandle handle);
   public void afterClose(FileHandle handle);
}

The beforeWriteAndClose method is called for each ENDFILE statement.

The Xpand engine will generate code for each processed FILE statement. This implies that files are written that might not have changed to the previous generator run. Normally it does not matter that files are rewritten. There are at least two good reasons when it is better to avoid rewriting of files:

  1. The generated source code will be checked in. In general it is not the recommended way to go to check in generated code, but sometimes you will have to. Especially with CVS there is the problem that rewritten files are recognized as modified, even if they haven't changed. So the problem arises that identical files get checked in again and again (or you revert it manually). When working in teams the problem even becomes worse, since team members will have conflicts when checking in.

  2. When it can be predicted that the generator won't produce different content before a file is even about to be created by a FILE statement then this can boost performance. Of course it is not trivial to predict that a specific file won't result in different content before it is even created. This requires information from a prior generator run and evaluation against the current model to process. Usually a diff model would be used as input for the decision.

Case 1) will prevent file writing after a FILE statement has been evaluated, case 2) will prevent creating a file at all.

To achieve this it is possible to add Veto Strategies to the generator, which are implementations of interface org.eclipse.xpand2.output.VetoStrategy or org.eclipse.xpand2.output.VetoStrategy2. Use VetoStrategy2 if you implement your own.

VetoStrategy2 declares two methods:

  • boolean hasVetoBeforeOpen (FileHandle)

    This method will be called before a file is being opened and generated. Return true to suppress the file creation.

  • boolean hasVeto (FileHandle)

    This method will be called after a file has been produced and after all configured PostProcessors have been invoked. Return true to suppress writing the file.

Veto Strategies are configured per Outlet. It is possible to add multiple stratgy instances to each Outlet.

  <component id="generator" class="org.eclipse.xpand2.Generator" skipOnErrors="true">
    <metaModel class="org.eclipse.xtend.typesystem.uml2.UML2MetaModel"/>
    <expand value="templates::Root::Root FOR model"/>
    <fileEncoding value="ISO-8859-1"/>
      <outlet path="src-gen">
         <postprocessor class="org.eclipse.xpand2.output.JavaBeautifier"/>
         
<vetoStrategy class="org.eclipse.xpand2.output.NoChangesVetoStrategy"/>

      </outlet>
  </component>

One VetoStrategy is already provided. The org.eclipse.xpand2.output.NoChangesVetoStrategy is a simple implementation that will compare the produced output, after it has been postprocessed, with the target file. If the content is identical the strategy vetoes the file writing. This strategy is effective, but has two severe drawbacks:

  1. The file has been created at least in memory before. This consumes time and memory. If applying code formatting this usually implies that the file is temporarily written.

  2. The existing file must be read into memory. This also costs time and memory.

Much better would be to even prevent the creation of files by having a valid implementation for the hasVetoBeforeOpen() method. Providing an implementation that predicts that files do not have to be created requires domain knowledge, thus a standard implementation is not available.

The number of skipped files will be reported by the Generator component like this:

2192 INFO  - Generator(generator): generating <...>
3792 INFO  - 
Skipped writing of 2 files to outlet
 [default](src-gen)

This example shows how to use aspect-oriented programming techniques in Xpand templates. It is applicable to EMF based and Classic systems. However, we explain the idea based on the emfExample – hence you should read that before.

There are many circumstances when template-AOP is useful. Here are two examples:

Scenario 1: Assume you have a nice generator that generates certain artifacts. The generator (or cartridge) might be a third party product, delivered in a single JAR file. Still you might want to adapt certain aspects of the generation process – without modifying the original generator .

Scenario 2: You are building a family of generators that can generate variations of the generate code, e.g. Implementations for different embedded platforms. In such a scenario, you need to be able to express those differences (variabilities) sensibly without creating a non-understandable chaos of if statements in the templates.

To illustrate the idea of extending a generator without "touching" it, let us create a new project called org.eclipse.demo.emf.datamodel.generator-aop. The idea is that it will "extend" the original org.eclipse.demo.emf.datamodel.generator project introduced in the emfExample . So this new projects needs to have a project dependency to the former one.

An AOP system always needs to define a join point model; this is, you have to define, at which locations of a (template) program you can add additional (template) code. In Xpand , the join points are simply templates (i.e. DEFINE .. ENDDEFINE ) blocks. An "aspect template" can be declared AROUND previously existing templates. If you take a look at the org.eclipse.demo.emf.datamodel.generator source folder of the project, you can find the Root.xpt template file. Inside, you can find a template called Impl that generates the implementation of the JavaBean.

«DEFINE Entity FOR data::Entity»
   «FILE baseClassFileName() »
      // generated at «timestamp()»
      public abstract class «baseClassName()» {
         «EXPAND Impl»
      }
   «ENDFILE»
«ENDDEFINE»

«DEFINE Impl FOR data::Entity»
   «EXPAND GettersAndSetters»
«ENDDEFINE»

«DEFINE Impl FOR data::PersistentEntity»
   «EXPAND GettersAndSetters»
    public void save() {

   }
«ENDDEFINE»

What we now want to do is as follows: Whenever the Impl template is executed, we want to run an additional template that generates additional code (for example, some kind of meta information for frameworks – the specific code is not important for the example here).

So, in our new project, we define the following template file:

«AROUND Impl FOR data::Entity»
   «FOREACH attribute AS a»
      public static final AttrInfo «a.name»Info = new AttrInfo(
         "«a.name»", «a.type».class );
   «ENDFOREACH»
   «targetDef.proceed()»
«ENDAROUND»

So, this new template wraps around the existing template called Impl It first generates additional code and then forwards the execution to the original template using targetDef.proceed(). So, in effect, this is a BEFORE advice. Moving the proceed statement to the beginning makes it an AFTER advice, omitting it, makes it an override.

In general, the syntax for the AROUND construct is as follows:

<<AROUND fullyQualifiedDefinitionNameWithWildcards
      (Paramlist (*)?) FOR TypeName>>
   do Stuff
<<ENDAROUND>>

Here are some examples:

<<AROUND *(*) FOR Object>>

matches all templates

<<AROUND *define(*) FOR Object>>

matches all templates with define at the end of its name and any number of parameters

<<AROUND org::eclipse::xpand2::* FOR Entity>>

matches all templates with namespace org::eclipse::xpand2:: that do not have any parameters and whose type is Entity or a subclass

<<AROUND *(String s) FOR Object>>

matches all templates that have exactly one String parameter

<<AROUND *(String s,*) FOR Object>>

matches all templates that have at least one String parameter

<<AROUND my::Template::definition(String s) FOR Entity>>

matches exactly this single definition

Inside an AROUND, there is the variable targetDef, which has the type xpand2::Definition. On this variable, you can call proceed, and also query a number of other things:

<<AROUND my::Template::definition(String s) FOR String>>
   log('invoking '+<<targetDef.name>>+' with '+this)
   <<targetDef.proceed()>>
<<ENDAROUND>>