Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Xtext parser rules overlap problem
Xtext parser rules overlap problem [message #1772474] Mon, 11 September 2017 08:49 Go to next message
Svilen Yanovski is currently offline Svilen YanovskiFriend
Messages: 3
Registered: September 2017
Junior Member
Hello,
I'm trying to create parser with Xtext/Xtend but i'm struggling with setting/resolving the grammar and my AST fail to map the input correctly.
In general Xtext seems to map the 1st match of possible match only, and didn't proceed checking if there are other viable options. This happens for example when i have the following 2 rules:

TypeArguments:
'<' args+=TypeArgument (',' args+=TypeArgument)* '>'

//catching declarations like "List<String, String>"
and

ComparisonOperationExpression:
ArithmeticalOperationExpression ({ComparisonOperationExpression.leftExpression=current} comments+=Comments =>operand=COMPARABLE_OPERANDS rightExpression=ArithmeticalOperationExpression)*

terminal COMPARABLE_OPERANDS:
('=='|'!='|'<='|'>='|'<'|'>')

In this case i'm just receiving warnings (hidden) that there are multiple matches, but always gets the first declaration for creating the AST.
There are other such clashes in the grammar and i'm not quite sure what is the best way to resolve them.

The Xtext grammar file is attached here. In the parser options i've set "backtracking" = true:

parserGenerator = {
options = {
backtrack = true
}
}


I've attached a screenshot of an example code where the parser cannot recognize the "<" operator. When change to "==" the problem disappears.

Please, can you help with a possible solution if you encountered problem like this before.

Regards,
Svilen :)
Re: Xtext parser rules overlap problem [message #1772569 is a reply to message #1772474] Tue, 12 September 2017 14:30 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12322
Registered: July 2009
Senior Member
did you consider to do the same a xbase does (using '<''=' instead of '<='

OpCompare:
'>=' | '<' '=' | '>' | '<' ;

(to tame the lexer)

and predicates instead of backtracing for the parser e.g.

XConstructorCall returns XExpression:
{XConstructorCall}
'new' constructor=[types::JvmConstructor|QualifiedName]
(=>'<' typeArguments+=JvmArgumentTypeReference (',' typeArguments+=JvmArgumentTypeReference)* '>')?
(=>explicitConstructorCall?='('


Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Xtext parser rules overlap problem [message #1772737 is a reply to message #1772569] Thu, 14 September 2017 14:00 Go to previous messageGo to next message
Svilen Yanovski is currently offline Svilen YanovskiFriend
Messages: 3
Registered: September 2017
Junior Member
Hi Christian,

Thank you for the advise regarding my problem. I've tried today to rewrite the lexer but unfortunately this is not solving the problem. Cases like "List<String> " are still causing clashes in the parser rules.
I'm parsing existing Java classes (not some custom language) with this program in order to build AST and i cannot change the parse rules really.

Regarding the backtracking - i'm using it in order to avoid left recursion in the lexer.

Regards,
Svilen

[Updated on: Thu, 14 September 2017 14:37]

Report message to a moderator

Re: Xtext parser rules overlap problem [message #1772749 is a reply to message #1772737] Thu, 14 September 2017 17:23 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12322
Registered: July 2009
Senior Member
you should still try to get rid of backtracking.

minimize your grammar to identify problems

FormalParameterList:
	params+=FormalParameter (',' params+=FormalParameter)* (',' formalParam=LastFormalParameter)?
    |   LastFormalParameter
;


LastFormalParameter:
	modifiers+=VariableModifier* type=Type '...' decl=VariableDeclaratorId
;

FormalParameter:
	modifiers+=VariableModifier* type=Type decl=VariableDeclaratorId
;

VariableModifier:
	{VariableModifier}
	'final'
;

VariableDeclaratorId:
	idName=ID ('[' ']')*
;

ClassOrInterfaceType:
	ids+=ID args+=TypeArguments? ('.' ids+=ID args+=TypeArguments? )*
//	QualifiedName
;

TypeArguments:
	'<' args+=TypeArgument (',' args+=TypeArgument)* '>'
;

 
Type:
	ClassOrInterfaceType ('[' ']')*
//    |   primitive=PRIMITIVE_TYPE ('[' ']')*
;


TypeArgument:
	type=Type
    | {TypeArgument} '?' (('extends' | 'super') boundedType=Type)?
;


adapt the wf to obtain a debug grammar to be used in antlr works

language = StandardLanguage {
			name = "org.xtext.example.mydsl1.MyDsl"
			fileExtensions = "mydsl1"

			serializer = {
				generateStub = false
			}
			validator = {
				// composedCheck = "org.eclipse.xtext.validation.NamesAreUniqueValidator"
			}
			parserGenerator = {
				debugGrammar = true
			}
		}


....


Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Xtext parser rules overlap problem [message #1773090 is a reply to message #1772749] Thu, 21 September 2017 11:10 Go to previous message
Svilen Yanovski is currently offline Svilen YanovskiFriend
Messages: 3
Registered: September 2017
Junior Member
Hi Christian,

I've finally managed to fix the issue by simply pre-processing the java classes and replacing the <> symbols for Type declaration with another unique symbol. Ofc the grammar is changed too to parse the new symbols as type decl. Looks like my experience was too limited to utilize a proper solution, but anyway - workaround is also ok for me and the performance is not an issue.
Thank you for trying to help - it's much appreciated :)

Regards,
Svilen
Previous Topic:Problem with multiple ecore refering
Next Topic:Help with modifying source and variables views during debugging (redirected)
Goto Forum:
  


Current Time: Wed Oct 17 20:00:15 GMT 2018

Powered by FUDForum. Page generated in 0.01979 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top