Eclipse Community Forums: TMF (Xtext) » Cannot set backtrack to false

Help

Home

Home » Modeling » TMF (Xtext) » Cannot set backtrack to false(mwe2)

Show: Today's Messages :: Show Polls :: Message Navigator

Cannot set backtrack to false [message #1823357]

Wed, 25 March 2020 13:20

Christian Sodomka

Messages: 8
Registered: May 2019

Junior Member

We have a problem with java.lang.IllegalStateException when auto formating a distinct dsl file.
I have read in some forum that this might be because of the use of backtracking, respectively left recursion grammar.
As we actually don't have such, I wanted to disable it with the statement

			parserGenerator = {
				options = {
					ignoreCase = true 	// Case Insensitive Keywords
					backtrack = false
				}
			}

just to be sure in the mwe2 file.
It seems that it doe not work properly because we still have the entry

backtrack=true

in the InternalTabsParser.g file.
How comes? And how to check whether the backtrack option is switched off?

Thanks for help
Christian

Report message to a moderator

Re: Cannot set backtrack to false [message #1823358 is a reply to message #1823357]

Wed, 25 March 2020 13:26

Christian Dietrich

Messages: 13724
Registered: July 2009

Senior Member

Hi, you may debug AbstractAntlrGrammarGenerator
can you please provide a complete example.

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Wed, 25 March 2020 13:38]

Report message to a moderator

Re: Cannot set backtrack to false [message #1823360 is a reply to message #1823358]

Wed, 25 March 2020 14:23

Christian Sodomka

Messages: 8
Registered: May 2019

Junior Member

If you mean the grammar, I don't think it makes sense because it has about 870 lines of code.
I already debugged the application and found the spot where it crashes. Now I want to set the left recursion to off to be sure there is none ( I suppose the parser genarator should throw an error then), but the entry in the mwe2 file did not lead to anything new and after searching for backtrack I still found it set to true in the InternalTabsParser.g file (whatever this file is)
So my actual problem is maybe to set the left recursion to off.

Report message to a moderator

Re: Cannot set backtrack to false [message #1823361 is a reply to message #1823360]

Wed, 25 March 2020 14:38

Christian Dietrich

Messages: 13724
Registered: July 2009

Senior Member

but the code backtrack=true must be done somewhere in the generator.
and the only place in the code of the (new) generator that creates backtrack=true is
AbstractAntlrGrammarGenerator

=> if you debug the workflow it should stop there.
or do you use the old deprecated workflow/generator

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Cannot set backtrack to false [message #1823362 is a reply to message #1823361]

Wed, 25 March 2020 14:50

Christian Sodomka

Messages: 8
Registered: May 2019

Junior Member

Well the entry in the mwe2 file should do the trick, right?
I use the new workflow/generator (it's mwe"2") at least as far as I know.
Well I will try to debug this thing...

Report message to a moderator

Re: Cannot set backtrack to false [message #1823367 is a reply to message #1823362]

Wed, 25 March 2020 15:26

Christian Dietrich

Messages: 13724
Registered: July 2009

Senior Member

the workflow is mwe2 for both old and new

parserGenerator = {

indicates you use the new one
(overlooked that at the first glance)

=> please debug where the backtrack=true comes from in AbstractAntlrGrammarGenerator

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Cannot set backtrack to false [message #1823370 is a reply to message #1823362]

Wed, 25 March 2020 15:30

Christian Sodomka

Messages: 8
Registered: May 2019

Junior Member

Uhh. This looks like an xtend bug to me! I experienced this before but not with templates.
Bad I cannot include snapshots here, but here is the code sniplet from AbstractAntlrGrammarGenerator (sorry for the bad characters):

	protected def compileParserOptions(Grammar it, AntlrOptions options) '''

		options {
			�IF !isCombinedGrammar�
				tokenVocab=�grammarNaming.getLexerGrammar(it).simpleName�;
			�ENDIF�
			�IF grammarNaming.getInternalParserSuperClass(it) !== null�
				superClass=�grammarNaming.getInternalParserSuperClass(it).simpleName�;
			�ENDIF�
			�IF isParserBackTracking(options) || options.memoize || options.k >= 0�
				�IF isParserBackTracking(options)�
					backtrack=true;
				�ENDIF�
				�IF options.memoize�
					memoize=true;
				�ENDIF�
				�IF options.k >= 0�
					memoize=�options.k�;
				�ENDIF�
>>>>		�ENDIF�
		}
	'''
	
	protected def isParserBackTracking(Grammar it, AntlrOptions options) {
		options.backtrack
	}

When you come to the point marked with ">>>>" the output is (copied from variable _builder.segments[]:

[
, options {, 
, 	, tokenVocab=, InternalTabsLexer, ;, 
, 	, superClass=, AbstractInternalContentAssistParser, ;, 
, 	, backtrack=true;,  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
]

Which is an error according to debugged variable options.backtrack which is false
so isParserBackTracking(options) should never return true!

Had this xtend error before in normal xtend code with nested ifs. If you insert "{}" after the condition everything is fine, if not sometimes it goes into the nested if, even if the condition is false.
But you cannot do this here with template syntax!

So I am really stuck :-(

PS: uploaded the screenshots

Attachment: 2020-03-25 16_27_49-eclipse-workspace - org.eclipse.xtext.xtext.generator.parser.antlr.AbstractAntlr.png
(Size: 17.46KB, Downloaded 14 times)
Attachment: 2020-03-25 16_28_43-eclipse-workspace - org.eclipse.xtext.xtext.generator.parser.antlr.AbstractAntlr.png
(Size: 25.60KB, Downloaded 15 times)

Report message to a moderator

Re: Cannot set backtrack to false [message #1823375 is a reply to message #1823370]

Wed, 25 March 2020 16:42

Christian Dietrich

Messages: 13724
Registered: July 2009

Senior Member

if you debug you can click on the stackframe with the xtend file and select show source -> java from the context menu
then you should be able to see who adds the backtrace=true to the _builder/StringConcatenation
you can also debug the getter and setter in org.eclipse.xtext.xtext.generator.parser.antlr.AntlrOptions
(rightklick, open generated file to see the java code)
to see who calls setBacktrack with true

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Cannot set backtrack to false [message #1823376 is a reply to message #1823375]

Wed, 25 March 2020 16:44

Christian Dietrich

Messages: 13724
Registered: July 2009

Senior Member

then i debug with a hello world dsl i see

options {
superClass=AbstractInternalAntlrParser;

and

options {
superClass=AbstractInternalContentAssistParser;

no tokenVocab

in the _builder

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Cannot set backtrack to false [message #1823377 is a reply to message #1823376]

Wed, 25 March 2020 16:47

Christian Dietrich

Messages: 13724
Registered: July 2009

Senior Member

please also have a look at the subclass that customizeses

org.eclipse.xtext.xtext.generator.parser.antlr.AntlrContentAssistGrammarGenerator.isParserBackTracking(Grammar, AntlrOptions)

override protected isParserBackTracking(Grammar it, AntlrOptions options) {
super.isParserBackTracking(it, options) || !allPredicatedElements.isEmpty
}

=> i assume you have allPredicatedElements

and the content assist grammar uses that as criteria
to enable backtracking.

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Wed, 25 March 2020 16:48]

Report message to a moderator

Re: Cannot set backtrack to false [message #1823407 is a reply to message #1823377]

Thu, 26 March 2020 08:37

Christian Sodomka

Messages: 8
Registered: May 2019

Junior Member

Well yes you are right. In the subclass the value is set according to !allPredicatedElements.isEmpty. Does that mean the program does not care what I set in the options and overwrites it accordingly to the current situation?!
What for is the option then?

Report message to a moderator

Re: Cannot set backtrack to false [message #1823409 is a reply to message #1823407]

Thu, 26 March 2020 08:48

Christian Dietrich

Messages: 13724
Registered: July 2009

Senior Member

there is none. if you use -> or => the ca parser needs backtracking (please note: this is the content assist parser, not the build parser)

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Cannot set backtrack to false [message #1823414 is a reply to message #1823409]

Thu, 26 March 2020 09:07

Christian Sodomka

Messages: 8
Registered: May 2019

Junior Member

Well actually I just wanted to check our rather complicated grammar for left recursive. I have learned that this is not possible by just switching the parser generator via options to this state (backtrack=false), because this might be overwritten.
But if it is overwritten, then the grammar is left recursive.

[Updated on: Thu, 26 March 2020 09:30]

Report message to a moderator

Previous Topic:	Maven dependencies
Next Topic:	how to create an external cross-reference in the AST

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Tue Jan 26 00:50:18 GMT 2021

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter