Eclipse Community Forums: TMF (Xtext) » Xtext parser rules overlap problem

Home » Modeling » TMF (Xtext) » Xtext parser rules overlap problem

Xtext parser rules overlap problem [message #1772474]

Mon, 11 September 2017 04:49

Eclipse User

Hello,
I'm trying to create parser with Xtext/Xtend but i'm struggling with setting/resolving the grammar and my AST fail to map the input correctly.
In general Xtext seems to map the 1st match of possible match only, and didn't proceed checking if there are other viable options. This happens for example when i have the following 2 rules:

TypeArguments:
'<' args+=TypeArgument (',' args+=TypeArgument)* '>'

//catching declarations like "List<String, String>"
and

ComparisonOperationExpression:
ArithmeticalOperationExpression ({ComparisonOperationExpression.leftExpression=current} comments+=Comments =>operand=COMPARABLE_OPERANDS rightExpression=ArithmeticalOperationExpression)*

terminal COMPARABLE_OPERANDS:
('=='|'!='|'<='|'>='|'<'|'>')

In this case i'm just receiving warnings (hidden) that there are multiple matches, but always gets the first declaration for creating the AST.
There are other such clashes in the grammar and i'm not quite sure what is the best way to resolve them.

The Xtext grammar file is attached here. In the parser options i've set "backtracking" = true:

parserGenerator = {
options = {
backtrack = true
}
}

I've attached a screenshot of an example code where the parser cannot recognize the "<" operator. When change to "==" the problem disappears.

Please, can you help with a possible solution if you encountered problem like this before.

Regards,
Svilen :)

Attachment: test.jpg
(Size: 22.87KB, Downloaded 148 times)
Attachment: XtextGrammar.xtext
(Size: 14.57KB, Downloaded 102 times)

Re: Xtext parser rules overlap problem [message #1772569 is a reply to message #1772474]

Tue, 12 September 2017 10:30

Eclipse User

did you consider to do the same a xbase does (using '<''=' instead of '<='

OpCompare:
'>=' | '<' '=' | '>' | '<' ;

(to tame the lexer)

and predicates instead of backtracing for the parser e.g.

XConstructorCall returns XExpression:
{XConstructorCall}
'new' constructor=[types::JvmConstructor|QualifiedName]
(=>'<' typeArguments+=JvmArgumentTypeReference (',' typeArguments+=JvmArgumentTypeReference)* '>')?
(=>explicitConstructorCall?='('

Re: Xtext parser rules overlap problem [message #1772737 is a reply to message #1772569]

Thu, 14 September 2017 10:00

Eclipse User

Hi Christian,

Thank you for the advise regarding my problem. I've tried today to rewrite the lexer but unfortunately this is not solving the problem. Cases like "List<String> " are still causing clashes in the parser rules.
I'm parsing existing Java classes (not some custom language) with this program in order to build AST and i cannot change the parse rules really.

Regarding the backtracking - i'm using it in order to avoid left recursion in the lexer.

Regards,
Svilen

[Updated on: Thu, 14 September 2017 10:37] by Moderator

Re: Xtext parser rules overlap problem [message #1772749 is a reply to message #1772737]

Thu, 14 September 2017 13:23

Eclipse User

you should still try to get rid of backtracking.

minimize your grammar to identify problems

FormalParameterList:
	params+=FormalParameter (',' params+=FormalParameter)* (',' formalParam=LastFormalParameter)?
    |   LastFormalParameter
;


LastFormalParameter:
	modifiers+=VariableModifier* type=Type '...' decl=VariableDeclaratorId
;

FormalParameter:
	modifiers+=VariableModifier* type=Type decl=VariableDeclaratorId
;

VariableModifier:
	{VariableModifier}
	'final'
;

VariableDeclaratorId:
	idName=ID ('[' ']')*
;

ClassOrInterfaceType:
	ids+=ID args+=TypeArguments? ('.' ids+=ID args+=TypeArguments? )*
//	QualifiedName
;

TypeArguments:
	'<' args+=TypeArgument (',' args+=TypeArgument)* '>'
;

 
Type:
	ClassOrInterfaceType ('[' ']')*
//    |   primitive=PRIMITIVE_TYPE ('[' ']')*
;


TypeArgument:
	type=Type
    | {TypeArgument} '?' (('extends' | 'super') boundedType=Type)?
;

adapt the wf to obtain a debug grammar to be used in antlr works

language = StandardLanguage {
			name = "org.xtext.example.mydsl1.MyDsl"
			fileExtensions = "mydsl1"

			serializer = {
				generateStub = false
			}
			validator = {
				// composedCheck = "org.eclipse.xtext.validation.NamesAreUniqueValidator"
			}
			parserGenerator = {
				debugGrammar = true
			}
		}

....

Re: Xtext parser rules overlap problem [message #1773090 is a reply to message #1772749]

Thu, 21 September 2017 07:10

Eclipse User

Hi Christian,

I've finally managed to fix the issue by simply pre-processing the java classes and replacing the <> symbols for Type declaration with another unique symbol. Ofc the grammar is changed too to parse the new symbols as type decl. Looks like my experience was too limited to utilize a proper solution, but anyway - workaround is also ok for me and the performance is not an issue.
Thank you for trying to help - it's much appreciated :)

Regards,
Svilen

Previous Topic:	Problem with multiple ecore refering
Next Topic:	Help with modifying source and variables views during debugging (redirected)

Goto Forum:

-=] Back to Top [=-

Current Time: Wed Jul 23 08:41:40 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter