Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Terminal Rule vs. Keyword Prioritization
Terminal Rule vs. Keyword Prioritization [message #1831331] Mon, 17 August 2020 15:08 Go to next message
L R is currently offline L RFriend
Messages: 11
Registered: July 2019
Junior Member
I am having difficulty determining how Xtext prioritizes between terminal rules and keyword prioritization (at least I think that is part of my problem.). The issue that I am seeing is related to three different uses of the ' symbol and corresponding white space.

With the grammar as is, the statement: "d1 = somename'anothername;" fails to parse. If I change to the terminal rule version of AttributeName (currently commented out), then the two lines, "d2 = somename ' anothername;" and "e1 = somename'(anothername);", fail to parse.

It seems to me that character literal terminal rule is somehow hiding the ' symbol in the attribute name parser rule, but I don't understand why.

Simple Grammar:
grammar org.xtext.example.mydsl.MyDsl hidden(WS, ML_COMMENT, SL_COMMENT)

generate myDsl "http://www.xtext.org/example/mydsl/MyDsl"
import "http://www.eclipse.org/emf/2002/Ecore" as ecore

File:
	statements+=Statement*;

Statement:
	name=ID "=" exp=Expression ';';

Expression:
	{CharacterLiteral} value=CHARACTER_LITERAL |
	{NameExpression} Name |
	{QualifiedExpression} type=ID "'" '(' exp=Expression ')';

Name:
	SimpleName | AttributeName;

SimpleName:
	ID;

AttributeName:
	ID "'" ID;

//AttributeName:
//	ATTRIBUTE_NAME
//;
//
//terminal ATTRIBUTE_NAME:
//	ID "'" ID
//;

terminal ID:
	LETTER ('_'? LETTER_OR_DIGIT)*;

terminal CHARACTER_LITERAL:
	"'" LETTER_OR_DIGIT "'";

terminal fragment LETTER_OR_DIGIT:
	LETTER | DIGIT;

terminal fragment LETTER:
	UPPER_CASE_LETTER | LOWER_CASE_LETTER;

terminal fragment UPPER_CASE_LETTER:
	'A'..'Z';

terminal fragment LOWER_CASE_LETTER:
	'a'..'z';

terminal fragment DIGIT:
	'0'..'9';

terminal ML_COMMENT:
	'/*'->'*/';

terminal SL_COMMENT:
	'--' !('\n' | '\r')* ('\r'? '\n')?;

terminal WS:
	(' ' | '\t' | '\r' | '\n')+;

terminal ANY_OTHER:
	.;




Simple Unit Test:
@Test
def void testAll() {
	val result = parseHelper.parse('''
		b = 'a';
		c = somename;
		d1 = somename'anothername;
		d2 = somename ' anothername;
		e1 = somename'(anothername);
		e2 = somename ' (anothername);
	''')
	Assertions.assertNotNull(result)
	val errors = result.eResource.errors
	Assertions.assertTrue(errors.isEmpty, '''Unexpected errors: «errors.join(", ")»''')
}

Re: Terminal Rule vs. Keyword Prioritization [message #1831333 is a reply to message #1831331] Mon, 17 August 2020 15:20 Go to previous message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
in this case the problem is that the lexer is greedy
you could let it bracktrack

parserGenerator = {
options = {
backtrackLexer = true
}
}


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Previous Topic: XText Integration with Angular application
Next Topic:Cross-referencing challenges
Goto Forum:
  


Current Time: Wed Oct 21 07:29:17 GMT 2020

Powered by FUDForum. Page generated in 0.02382 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top