I needed to implement a "normalization" command to re-order elements in my DSL (if you're interested in details, see: https://github.com/JPL-IMCE/gov.nasa.jpl.imce.oml/issues/155)
There are two parts to this:
1) A UI-level menu/command/handler to invoke the normalization service from an XTextEditor
2) The actual normalization service
Initially, I wrote (1) like this:
public class NormalizeOMLContentsOrder extends AbstractHandler {
override def Object execute(ExecutionEvent event) throws ExecutionException {
val XtextEditor editor = EditorUtils.getActiveXtextEditor(event)
val doc = editor?.document
if (null !== doc) {
doc.modify(normalizeOMLResource)
}
null
}
protected static val IUnitOfWork.Void<XtextResource> normalizeOMLResource =
new IUnitOfWork.Void<XtextResource>() {
override def void process(XtextResource state) throws Exception {
state
.contents
.filter(Extent)
.forEach[ext|
OMLExtensions.normalize(ext)
]
}
}
}
Then I defined "OMLExtensions.normalize()" functions that reorder AST elements in various containment references in the metamodel (i.e., "EList"s).
For a simple example like this:
open terminology <http://imce.jpl.nasa.gov/test1> {
// 1
concept A
// 2
aspect B
// 3
A extendsAspect B
// 4
}
I would get the following:
open terminology <http://imce.jpl.nasa.gov/test1> {
// 1
aspect Bconcept A
// 2
A extendsAspect B
// 4
}
The problem is in this line:
which should have had some hidden whitespace, e.g.:
or perhaps:
In the debugger, I noticed that the problem happens after the AST elements have been re-ordered in org.eclipse.xtext.ui.editor.model.edit.ReconcilingUnitOfWork:
@Override
public T exec(XtextResource state) throws Exception {
String original = document.get();
DocumentChangeListener documentChangeListener = new DocumentChangeListener();
T result;
try {
document.addDocumentListener(documentChangeListener);
// lazy linking URIs might change, so resolve everything before applying any changes
EcoreUtil2.resolveAll(state, CancelIndicator.NullImpl);
composer.beginRecording(state);
result = work.exec(state); // here, work = NormalizeOMLContentsOrder.normalizeOMLResource
final TextEdit edit = composer.endRecording(); // essential whitespace is lost here.
if (edit != null) {
if(documentChangeListener.hasDocumentChanged())
throw new IllegalStateException("Cannot modify document textually and semantically within the same unit of work");
RewriteSessionEditProcessor processor = new RewriteSessionEditProcessor(document, edit, TextEdit.UPDATE_REGIONS | TextEdit.CREATE_UNDO);
processor.performEdits();
}
} catch (RuntimeException e) {
document.set(original);
throw e;
} catch (Exception e) {
document.set(original);
throw new RuntimeException(e);
} finally {
document.removeDocumentListener(documentChangeListener);
}
return result;
}
During the call to "composer.endRecording()", the document is formatted; which results in calls to the DSL's formatter dispatch methods.
I noticed that at that stage, there are several hidden regions that have no terminal tokens. For example, based on the above, the formatter's "textRegionExtensions" shows:
Columns: 1:offset 2:length 3:kind 4: text 5:grammarElement
Kind: H=IHiddenRegion S=ISemanticRegion B/E=IEObjectRegion
0 0 H
B Extent Extent
B TerminologyGraph Extent:modules+=Module path:Extent/modules[0]
0 4 S "open" TerminologyGraph:kind=TerminologyKind
4 1 H " " Whitespace:TerminalRule'WS'
5 11 S "terminology" TerminologyGraph:'terminology'
16 1 H " " Whitespace:TerminalRule'WS'
17 32 S "<http://imce.jp..." TerminologyGraph:iri=IRI
49 1 H " " Whitespace:TerminalRule'WS'
50 1 S "{" TerminologyGraph:'{'
51 H "\n\n\t" Whitespace:TerminalRule'WS'
"// 1\n" Comment:TerminalRule'SL_COMMENT'
11 "\t\n\t" Whitespace:TerminalRule'WS'
B Aspect'B' TerminologyGraph:boxStatements+=TerminologyBoxStatement path:TerminologyGraph/boxStatements[0]=Extent/modules[0]
62 6 S "aspect" Aspect:'aspect'
68 1 H " " Whitespace:TerminalRule'WS'
69 1 S "B" Aspect:name=ID
E Aspect'B' TerminologyGraph:boxStatements+=TerminologyBoxStatement path:TerminologyGraph/boxStatements[0]=Extent/modules[0]
70 0 H
B Concept'A' TerminologyGraph:boxStatements+=TerminologyBoxStatement path:TerminologyGraph/boxStatements[1]=Extent/modules[0]
70 7 S "concept" Concept:'concept'
77 1 H " " Whitespace:TerminalRule'WS'
78 1 S "A" Concept:name=ID
E Concept'A' TerminologyGraph:boxStatements+=TerminologyBoxStatement path:TerminologyGraph/boxStatements[1]=Extent/modules[0]
79 H "\n\t\n\t" Whitespace:TerminalRule'WS'
"// 2\n" Comment:TerminalRule'SL_COMMENT'
12 "\t\n\t" Whitespace:TerminalRule'WS'
B AspectSpecializationAxiom TerminologyGraph:boxStatements+=TerminologyBoxStatement path:TerminologyGraph/boxStatements[2]=Extent/modules[0]
91 1 S "A" AspectSpecializationAxiom:subEntity=[Entity|Reference]
92 1 H " " Whitespace:TerminalRule'WS'
93 13 S "extendsAspect" AspectSpecializationAxiom:'extendsAspect'
106 1 H " " Whitespace:TerminalRule'WS'
107 1 S "B" AspectSpecializationAxiom:superAspect=[Aspect|Reference]
E AspectSpecializationAxiom TerminologyGraph:boxStatements+=TerminologyBoxStatement path:TerminologyGraph/boxStatements[2]=Extent/modules[0]
108 H "\n\n\t" Whitespace:TerminalRule'WS'
8 "// 4\n" Comment:TerminalRule'SL_COMMENT'
116 1 S "}" TerminologyGraph:'}'
E TerminologyGraph Extent:modules+=Module path:Extent/modules[0]
E Extent Extent
117 1 H "\n" Whitespace:TerminalRule'WS'
The problem seems to come from this line:
This ought to have some kind of terminal space, e.g.:
After a while, I noticed that XText attaches org.eclipse.xtext.nodemodel.INode as an annotation on an EObject (the AST element).
So, I tried to delete all such annotations as part of my normalization logic:
static def void removeAllINodes(List<EObject> queue) {
if (!queue.empty) {
val e = queue.remove(0)
val List<INode> nodes = e.eAdapters().filter(INode).toList
e.eAdapters.removeAll(nodes)
queue.addAll(e.eContents)
removeAllINodes(queue)
}
}
// Delete previous concrete syntax INodes before changing the order of elements.
// This is important as subsequent serialization will trigger formatting the contents.
// During that process, XText would use cached INodes, if available and doing so could produce incorrectly formatted text
// that can be ill-formed according to the grammar.
static def dispatch void normalize(Extent ext) {
val queue = new ArrayList<EObject>()
queue.add(ext)
removeAllINodes(queue)
// ... DSL-specific logic
}
My question now is whether this is a kosher way of performing a modification on the order of various elements in a DSL resource associated with an XTextEditor.
In limited testing, my solution seems to work; however, I suspect there must be corner cases where removing all INode annotations could cause problems. So, the question is how to do this safely in case something bad happens during the actual reordering logic?
- Nicolas.