Eclipse Community Forums: TMF (Xtext) » generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13

Home » Modeling » TMF (Xtext) » generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13

generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781887]

Wed, 14 February 2018 05:38

Eclipse User

We've been facing an issue after updating from xtext 2.9.2 to xtext 2.13 and from Eclipse Luna to Eclipse Oxygen. The generated *ParsetreeConstructor files from our various grammars do not compile, because part of the .xtext input grammar ends up in it! Here's an excerpt of a generated ParsetreeConstructor:

/************ begin Rule EmbeddedSection ****************
 *
 * /* opening and closing stratumId should match */ EmbeddedSection:
 * 	"*O" WS? stratumID=NON_ASTERISK_STRING CR
 * 	embeddedSmaps+=SMAP*
 * 	"*C" WS? closingStratumID=NON_ASTERISK_STRING CR;
 *
 **/

Note how "EmbeddedSection" isn't in a comment. It comes from the corresponding part of the .xtext grammar:

/* opening and closing stratumId should match */
EmbeddedSection:
	"*O" WS? stratumID=NON_ASTERISK_STRING CR
	embeddedSmaps+=SMAP*
	"*C" WS? closingStratumID=NON_ASTERISK_STRING CR;

It is related to the handling of comments, the disturbing part being "/* opening and closing stratumId should match */" in our grammar. If I remove it, generation works. I have several examples of the same kind (in the same grammar, and in other grammars too).

We are still using the old Generator class in our mwe2 files (org.eclipse.xtext.generator). I know it's not being maintained, but you know time flies and our grammars are 5 years old now :-)

Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781888 is a reply to message #1781887]

Wed, 14 February 2018 05:43

Eclipse User

so you dont use the old generator only but the old serializer too?
you could subclass and patch the fragment/xpt file

did you investigate where this is called. most places use
grammarFragmentToStr("// ", lineDelimiter)

Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781890 is a reply to message #1781888]

Wed, 14 February 2018 05:47

Eclipse User

p.S. could you provide a hello world project. createing a old style project by hand is pita

Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781892 is a reply to message #1781890]

Wed, 14 February 2018 06:08

Eclipse User

Maybe you could change your comment style in the grammar?

Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781895 is a reply to message #1781892]

Wed, 14 February 2018 06:18

Eclipse User

@ChristianDietrich> Yes we use the old serializer too (org.eclipse.xtext.generator.parseTreeConstructor.ParseTreeConstructorFragment). What do you mean by "investigate where this is called" ? I'm sorry I'm not the xtext guy in the company and don't really know which code is called by mwe2. I'll look into providing a simple example, it seems feasible.

@KarstenThoms> changing the comments to // indeed avoids the problem

Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781897 is a reply to message #1781895]

Wed, 14 February 2018 06:40

Eclipse User

Here's an example project to reproduce the problem: http://dl.free.fr/jWNrj1Pjx

Reproduction procedure:

* Unpack the archive
* Import the com.provenrun.smap project into Eclipse Oxygen 4.7.0 with Xtext 2.13 installed, it'll have errors; it's normal
* Right-click on src/com/provenrun/smap/SMAP.xtext > Run as MWE2 Workflow
* The generator will ask you to download http://download.itemis.com/antlr-generator-3.2.0-patch.jar, do it and then the generator should finish successfully
* Then witness that /com.provenrun.smap/src-gen/com/provenrun/smap/parseTreeConstruction/SMAPParsetreeConstructor.java has errors

Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781901 is a reply to message #1781897]

Wed, 14 February 2018 07:39

Eclipse User

you can try to patch like

Attachment: generator-patch.zip
(Size: 11.75KB, Downloaded 111 times)

Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781905 is a reply to message #1781901]

Wed, 14 February 2018 09:49

Eclipse User

Thanks Christian, but how exactly should I apply this patch (we're working against released versions of xtext, not the source) ?

Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781906 is a reply to message #1781905]

Wed, 14 February 2018 09:52

Eclipse User

i dont get your question:

import patch as project
add dependency from your project to patch project
exchange the current fragment with the patched one

Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781910 is a reply to message #1781906]

Wed, 14 February 2018 10:11

Eclipse User

I wasn't aware it was so easy to replace the fragment being used! I confirm it fixes the bug on all 3 grammars I tested, thank you very much; we can continue our update now.

And I've confirmed with the local xtext master that we'll update to the new generator soon :-) so hopefully we won't bother you again ☺

On a final note, I've tried to turn all multiline comments into single line comments (//) in our grammars, but there was one position where */ was used in a rule, which still triggered the bug:


ML_StandaloneSMDoc:
	{ML_StandaloneSMDoc}
	// first line
	body+=ML_StandaloneSMDocFirstLineBody
	// Intermediate lines
	(body+=NewLine //
	//
	body+=ML_SMDocNextLineBody)* //
	// closing
	body+=NewLine?
	'*/';  // <- Here's the one

so it isn't a viable workaround after all.

[Updated on: Wed, 14 February 2018 10:13] by Moderator

Re: generated ParsetreeConstructor file doesn't compile after upgrade from xtext 2.9.2 to xtext 2.13 [message #1781911 is a reply to message #1781910]

Wed, 14 February 2018 10:17

Eclipse User

yes the problem is your keyword.

Previous Topic:	Trying to write a DSL with Xtext to specify an XML-like language
Next Topic:	How to define a greedy version of Xtext until (->)?

Goto Forum:

-=] Back to Top [=-

Current Time: Fri Jul 25 05:59:33 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter