Eclipse Community Forums: TMF (Xtext) » Xtext parser rules overlap problem

Help

Home

Home » Modeling » TMF (Xtext) » Xtext parser rules overlap problem

Show: Today's Messages :: Show Polls :: Message Navigator

Xtext parser rules overlap problem [message #1772474]

Mon, 11 September 2017 08:49

Svilen Yanovski

Messages: 3
Registered: September 2017

Junior Member

Hello,
I'm trying to create parser with Xtext/Xtend but i'm struggling with setting/resolving the grammar and my AST fail to map the input correctly.
In general Xtext seems to map the 1st match of possible match only, and didn't proceed checking if there are other viable options. This happens for example when i have the following 2 rules:

TypeArguments:
'<' args+=TypeArgument (',' args+=TypeArgument)* '>'

//catching declarations like "List<String, String>"
and

ComparisonOperationExpression:
ArithmeticalOperationExpression ({ComparisonOperationExpression.leftExpression=current} comments+=Comments =>operand=COMPARABLE_OPERANDS rightExpression=ArithmeticalOperationExpression)*

terminal COMPARABLE_OPERANDS:
('=='|'!='|'<='|'>='|'<'|'>')

In this case i'm just receiving warnings (hidden) that there are multiple matches, but always gets the first declaration for creating the AST.
There are other such clashes in the grammar and i'm not quite sure what is the best way to resolve them.

The Xtext grammar file is attached here. In the parser options i've set "backtracking" = true:

parserGenerator = {
options = {
backtrack = true
}
}

I've attached a screenshot of an example code where the parser cannot recognize the "<" operator. When change to "==" the problem disappears.

Please, can you help with a possible solution if you encountered problem like this before.

Regards,
Svilen :)

Attachment: test.jpg
(Size: 22.87KB, Downloaded 132 times)
Attachment: XtextGrammar.xtext
(Size: 14.57KB, Downloaded 59 times)

Report message to a moderator

Re: Xtext parser rules overlap problem [message #1772569 is a reply to message #1772474]

Tue, 12 September 2017 14:30

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

did you consider to do the same a xbase does (using '<''=' instead of '<='

OpCompare:
'>=' | '<' '=' | '>' | '<' ;

(to tame the lexer)

and predicates instead of backtracing for the parser e.g.

XConstructorCall returns XExpression:
{XConstructorCall}
'new' constructor=[types::JvmConstructor|QualifiedName]
(=>'<' typeArguments+=JvmArgumentTypeReference (',' typeArguments+=JvmArgumentTypeReference)* '>')?
(=>explicitConstructorCall?='('

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Xtext parser rules overlap problem [message #1772737 is a reply to message #1772569]

Thu, 14 September 2017 14:00

Svilen Yanovski

Messages: 3
Registered: September 2017

Junior Member

Hi Christian,

Thank you for the advise regarding my problem. I've tried today to rewrite the lexer but unfortunately this is not solving the problem. Cases like "List<String> " are still causing clashes in the parser rules.
I'm parsing existing Java classes (not some custom language) with this program in order to build AST and i cannot change the parse rules really.

Regarding the backtracking - i'm using it in order to avoid left recursion in the lexer.

Regards,
Svilen

[Updated on: Thu, 14 September 2017 14:37]

Report message to a moderator

Re: Xtext parser rules overlap problem [message #1772749 is a reply to message #1772737]

Thu, 14 September 2017 17:23

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

you should still try to get rid of backtracking.

minimize your grammar to identify problems

FormalParameterList:
	params+=FormalParameter (',' params+=FormalParameter)* (',' formalParam=LastFormalParameter)?
    |   LastFormalParameter
;


LastFormalParameter:
	modifiers+=VariableModifier* type=Type '...' decl=VariableDeclaratorId
;

FormalParameter:
	modifiers+=VariableModifier* type=Type decl=VariableDeclaratorId
;

VariableModifier:
	{VariableModifier}
	'final'
;

VariableDeclaratorId:
	idName=ID ('[' ']')*
;

ClassOrInterfaceType:
	ids+=ID args+=TypeArguments? ('.' ids+=ID args+=TypeArguments? )*
//	QualifiedName
;

TypeArguments:
	'<' args+=TypeArgument (',' args+=TypeArgument)* '>'
;

 
Type:
	ClassOrInterfaceType ('[' ']')*
//    |   primitive=PRIMITIVE_TYPE ('[' ']')*
;


TypeArgument:
	type=Type
    | {TypeArgument} '?' (('extends' | 'super') boundedType=Type)?
;

adapt the wf to obtain a debug grammar to be used in antlr works

language = StandardLanguage {
			name = "org.xtext.example.mydsl1.MyDsl"
			fileExtensions = "mydsl1"

			serializer = {
				generateStub = false
			}
			validator = {
				// composedCheck = "org.eclipse.xtext.validation.NamesAreUniqueValidator"
			}
			parserGenerator = {
				debugGrammar = true
			}
		}

....

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Xtext parser rules overlap problem [message #1773090 is a reply to message #1772749]

Thu, 21 September 2017 11:10

Svilen Yanovski

Messages: 3
Registered: September 2017

Junior Member

Hi Christian,

I've finally managed to fix the issue by simply pre-processing the java classes and replacing the <> symbols for Type declaration with another unique symbol. Ofc the grammar is changed too to parse the new symbols as type decl. Looks like my experience was too limited to utilize a proper solution, but anyway - workaround is also ok for me and the performance is not an issue.
Thank you for trying to help - it's much appreciated :)

Regards,
Svilen

Report message to a moderator

Previous Topic:	Problem with multiple ecore refering
Next Topic:	Help with modifying source and variables views during debugging (redirected)

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Fri Apr 26 15:19:31 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter