Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » How to correctly modify the order of containment references in an XTextEditor?
How to correctly modify the order of containment references in an XTextEditor? [message #1776135] Mon, 13 November 2017 04:30 Go to next message
Nicolas Rouquette is currently offline Nicolas RouquetteFriend
Messages: 38
Registered: December 2014
I needed to implement a "normalization" command to re-order elements in my DSL (if you're interested in details, see:

There are two parts to this:

1) A UI-level menu/command/handler to invoke the normalization service from an XTextEditor

2) The actual normalization service

Initially, I wrote (1) like this:

public class NormalizeOMLContentsOrder extends AbstractHandler {
	override def Object execute(ExecutionEvent event) throws ExecutionException {
		val XtextEditor editor = EditorUtils.getActiveXtextEditor(event)
		val doc = editor?.document
		if (null !== doc) {
	protected static val IUnitOfWork.Void<XtextResource> normalizeOMLResource = 
	new IUnitOfWork.Void<XtextResource>() {
		override def void process(XtextResource state) throws Exception {

Then I defined "OMLExtensions.normalize()" functions that reorder AST elements in various containment references in the metamodel (i.e., "EList"s).

For a simple example like this:

open terminology <> {

	// 1
	concept A
	// 2
	aspect B
	// 3

	A extendsAspect B

	// 4

I would get the following:

open terminology <> {

	// 1
	aspect Bconcept A
	// 2
	A extendsAspect B

	// 4

The problem is in this line:

	aspect Bconcept A

which should have had some hidden whitespace, e.g.:

	aspect B

        concept A

or perhaps:

	aspect B
        concept A

In the debugger, I noticed that the problem happens after the AST elements have been re-ordered in org.eclipse.xtext.ui.editor.model.edit.ReconcilingUnitOfWork:

	public T exec(XtextResource state) throws Exception {
		String original = document.get();
		DocumentChangeListener documentChangeListener = new DocumentChangeListener();
		T result;
		try {
			// lazy linking URIs might change, so resolve everything before applying any changes
			EcoreUtil2.resolveAll(state, CancelIndicator.NullImpl);
			result = work.exec(state); // here, work = NormalizeOMLContentsOrder.normalizeOMLResource
			final TextEdit edit = composer.endRecording(); // essential whitespace is lost here.
			if (edit != null) {
					throw new IllegalStateException("Cannot modify document textually and semantically within the same unit of work");
				RewriteSessionEditProcessor processor = new RewriteSessionEditProcessor(document, edit, TextEdit.UPDATE_REGIONS | TextEdit.CREATE_UNDO);
		} catch (RuntimeException e) {
			throw e;
		} catch (Exception e) {
			throw new RuntimeException(e);
		} finally {
		return result;

During the call to "composer.endRecording()", the document is formatted; which results in calls to the DSL's formatter dispatch methods.
I noticed that at that stage, there are several hidden regions that have no terminal tokens. For example, based on the above, the formatter's "textRegionExtensions" shows:

Columns: 1:offset 2:length 3:kind 4: text 5:grammarElement
Kind: H=IHiddenRegion S=ISemanticRegion B/E=IEObjectRegion

  0  0 H
       B Extent               Extent
        B TerminologyGraph     Extent:modules+=Module path:Extent/modules[0]
  0  4   S "open"               TerminologyGraph:kind=TerminologyKind
  4  1   H " "                  Whitespace:TerminalRule'WS'
  5 11   S "terminology"        TerminologyGraph:'terminology'
 16  1   H " "                  Whitespace:TerminalRule'WS'
 17 32   S "<" TerminologyGraph:iri=IRI
 49  1   H " "                  Whitespace:TerminalRule'WS'
 50  1   S "{"                  TerminologyGraph:'{'
 51      H "\n\n\t"             Whitespace:TerminalRule'WS'
           "// 1\n"             Comment:TerminalRule'SL_COMMENT'
    11     "\t\n\t"             Whitespace:TerminalRule'WS'
         B Aspect'B'            TerminologyGraph:boxStatements+=TerminologyBoxStatement path:TerminologyGraph/boxStatements[0]=Extent/modules[0]
 62  6    S "aspect"             Aspect:'aspect'
 68  1    H " "                  Whitespace:TerminalRule'WS'
 69  1    S "B"                  Aspect:name=ID
         E Aspect'B'            TerminologyGraph:boxStatements+=TerminologyBoxStatement path:TerminologyGraph/boxStatements[0]=Extent/modules[0]
 70  0   H
         B Concept'A'           TerminologyGraph:boxStatements+=TerminologyBoxStatement path:TerminologyGraph/boxStatements[1]=Extent/modules[0]
 70  7    S "concept"            Concept:'concept'
 77  1    H " "                  Whitespace:TerminalRule'WS'
 78  1    S "A"                  Concept:name=ID
         E Concept'A'           TerminologyGraph:boxStatements+=TerminologyBoxStatement path:TerminologyGraph/boxStatements[1]=Extent/modules[0]
 79      H "\n\t\n\t"           Whitespace:TerminalRule'WS'
           "// 2\n"             Comment:TerminalRule'SL_COMMENT'
    12     "\t\n\t"             Whitespace:TerminalRule'WS'
         B AspectSpecializationAxiom TerminologyGraph:boxStatements+=TerminologyBoxStatement path:TerminologyGraph/boxStatements[2]=Extent/modules[0]
 91  1    S "A"                  AspectSpecializationAxiom:subEntity=[Entity|Reference]
 92  1    H " "                  Whitespace:TerminalRule'WS'
 93 13    S "extendsAspect"      AspectSpecializationAxiom:'extendsAspect'
106  1    H " "                  Whitespace:TerminalRule'WS'
107  1    S "B"                  AspectSpecializationAxiom:superAspect=[Aspect|Reference]
         E AspectSpecializationAxiom TerminologyGraph:boxStatements+=TerminologyBoxStatement path:TerminologyGraph/boxStatements[2]=Extent/modules[0]
108      H "\n\n\t"             Whitespace:TerminalRule'WS'
     8     "// 4\n"             Comment:TerminalRule'SL_COMMENT'
116  1   S "}"                  TerminologyGraph:'}'
        E TerminologyGraph     Extent:modules+=Module path:Extent/modules[0]
       E Extent               Extent
117  1 H "\n"                 Whitespace:TerminalRule'WS'

The problem seems to come from this line:

 70  0   H

This ought to have some kind of terminal space, e.g.:

 70  0   H "\n\n\t"

After a while, I noticed that XText attaches org.eclipse.xtext.nodemodel.INode as an annotation on an EObject (the AST element).

So, I tried to delete all such annotations as part of my normalization logic:

	static def void removeAllINodes(List<EObject> queue) {
		if (!queue.empty) {
			val e = queue.remove(0)
			val List<INode> nodes = e.eAdapters().filter(INode).toList
	// Delete previous concrete syntax INodes before changing the order of elements.
	// This is important as subsequent serialization will trigger formatting the contents.
	// During that process, XText would use cached INodes, if available and doing so could produce incorrectly formatted text 
	// that can be ill-formed according to the grammar.
	static def dispatch void normalize(Extent ext) {
		val queue = new ArrayList<EObject>()
		// ... DSL-specific logic

My question now is whether this is a kosher way of performing a modification on the order of various elements in a DSL resource associated with an XTextEditor.

In limited testing, my solution seems to work; however, I suspect there must be corner cases where removing all INode annotations could cause problems. So, the question is how to do this safely in case something bad happens during the actual reordering logic?

- Nicolas.
Re: How to correctly modify the order of containment references in an XTextEditor? [message #1776136 is a reply to message #1776135] Mon, 13 November 2017 05:03 Go to previous message
Christian Dietrich is currently online Christian DietrichFriend
Messages: 11576
Registered: July 2009
Senior Member
I am sure this is one of the known bugs of the serializer
But I did not find the one I have a dejavu right now
So file a ticket at

Need professional support for Xtext, Xpand, EMF?
Go to:
Twitter : @chrdietrich
Blog :
Previous Topic:What is wrong with my grammar
Next Topic:Could quickfix be position sensitive?
Goto Forum:

Current Time: Tue Jan 16 07:42:08 GMT 2018

Powered by FUDForum. Page generated in 0.01445 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software