Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13
generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781887] Wed, 14 February 2018 10:38 Go to next message
Clément Hurlin is currently offline Clément HurlinFriend
Messages: 9
Registered: February 2014
Junior Member
We've been facing an issue after updating from xtext 2.9.2 to xtext 2.13 and from Eclipse Luna to Eclipse Oxygen. The generated *ParsetreeConstructor files from our various grammars do not compile, because part of the .xtext input grammar ends up in it! Here's an excerpt of a generated ParsetreeConstructor:

/************ begin Rule EmbeddedSection ****************
 *
 * /* opening and closing stratumId should match */ EmbeddedSection:
 * 	"*O" WS? stratumID=NON_ASTERISK_STRING CR
 * 	embeddedSmaps+=SMAP*
 * 	"*C" WS? closingStratumID=NON_ASTERISK_STRING CR;
 *
 **/


Note how "EmbeddedSection" isn't in a comment. It comes from the corresponding part of the .xtext grammar:

/* opening and closing stratumId should match */
EmbeddedSection:
	"*O" WS? stratumID=NON_ASTERISK_STRING CR
	embeddedSmaps+=SMAP*
	"*C" WS? closingStratumID=NON_ASTERISK_STRING CR;


It is related to the handling of comments, the disturbing part being "/* opening and closing stratumId should match */" in our grammar. If I remove it, generation works. I have several examples of the same kind (in the same grammar, and in other grammars too).

We are still using the old Generator class in our mwe2 files (org.eclipse.xtext.generator). I know it's not being maintained, but you know time flies and our grammars are 5 years old now :-)
Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781888 is a reply to message #1781887] Wed, 14 February 2018 10:43 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12043
Registered: July 2009
Senior Member
so you dont use the old generator only but the old serializer too?
you could subclass and patch the fragment/xpt file

did you investigate where this is called. most places use
grammarFragmentToStr("// ", lineDelimiter)




Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781890 is a reply to message #1781888] Wed, 14 February 2018 10:47 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12043
Registered: July 2009
Senior Member
p.S. could you provide a hello world project. createing a old style project by hand is pita

Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781892 is a reply to message #1781890] Wed, 14 February 2018 11:08 Go to previous messageGo to next message
Karsten Thoms is currently offline Karsten ThomsFriend
Messages: 602
Registered: July 2009
Location: Dortmund, Germany
Senior Member

Maybe you could change your comment style in the grammar?
Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781895 is a reply to message #1781892] Wed, 14 February 2018 11:18 Go to previous messageGo to next message
Clément Hurlin is currently offline Clément HurlinFriend
Messages: 9
Registered: February 2014
Junior Member
@ChristianDietrich> Yes we use the old serializer too (org.eclipse.xtext.generator.parseTreeConstructor.ParseTreeConstructorFragment). What do you mean by "investigate where this is called" ? I'm sorry I'm not the xtext guy in the company and don't really know which code is called by mwe2. I'll look into providing a simple example, it seems feasible.

@KarstenThoms> changing the comments to // indeed avoids the problem
Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781897 is a reply to message #1781895] Wed, 14 February 2018 11:40 Go to previous messageGo to next message
Clément Hurlin is currently offline Clément HurlinFriend
Messages: 9
Registered: February 2014
Junior Member
Here's an example project to reproduce the problem: http://dl.free.fr/jWNrj1Pjx

Reproduction procedure:

* Unpack the archive
* Import the com.provenrun.smap project into Eclipse Oxygen 4.7.0 with Xtext 2.13 installed, it'll have errors; it's normal
* Right-click on src/com/provenrun/smap/SMAP.xtext > Run as MWE2 Workflow
* The generator will ask you to download http://download.itemis.com/antlr-generator-3.2.0-patch.jar, do it and then the generator should finish successfully
* Then witness that /com.provenrun.smap/src-gen/com/provenrun/smap/parseTreeConstruction/SMAPParsetreeConstructor.java has errors
Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781901 is a reply to message #1781897] Wed, 14 February 2018 12:39 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12043
Registered: July 2009
Senior Member
you can try to patch like



Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781905 is a reply to message #1781901] Wed, 14 February 2018 14:49 Go to previous messageGo to next message
Clément Hurlin is currently offline Clément HurlinFriend
Messages: 9
Registered: February 2014
Junior Member
Thanks Christian, but how exactly should I apply this patch (we're working against released versions of xtext, not the source) ?
Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781906 is a reply to message #1781905] Wed, 14 February 2018 14:52 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12043
Registered: July 2009
Senior Member
i dont get your question:

import patch as project
add dependency from your project to patch project
exchange the current fragment with the patched one


Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781910 is a reply to message #1781906] Wed, 14 February 2018 15:11 Go to previous messageGo to next message
Clément Hurlin is currently offline Clément HurlinFriend
Messages: 9
Registered: February 2014
Junior Member
I wasn't aware it was so easy to replace the fragment being used! I confirm it fixes the bug on all 3 grammars I tested, thank you very much; we can continue our update now.

And I've confirmed with the local xtext master that we'll update to the new generator soon :-) so hopefully we won't bother you again ☺

On a final note, I've tried to turn all multiline comments into single line comments (//) in our grammars, but there was one position where */ was used in a rule, which still triggered the bug:


ML_StandaloneSMDoc:
	{ML_StandaloneSMDoc}
	// first line
	body+=ML_StandaloneSMDocFirstLineBody
	// Intermediate lines
	(body+=NewLine //
	//
	body+=ML_SMDocNextLineBody)* //
	// closing
	body+=NewLine?
	'*/';  // <- Here's the one


so it isn't a viable workaround after all.

[Updated on: Wed, 14 February 2018 15:13]

Report message to a moderator

Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781911 is a reply to message #1781910] Wed, 14 February 2018 15:17 Go to previous message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12043
Registered: July 2009
Senior Member
yes the problem is your keyword.

Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Previous Topic:Clickable link in LabelProvider
Next Topic:How to define a greedy version of Xtext until (->)?
Goto Forum:
  


Current Time: Sun Jun 24 22:52:53 GMT 2018

Powered by FUDForum. Page generated in 0.02127 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top