Eclipse Community Forums: TMF (Xtext) » Custom Lexer and Semantic Predicates issue

Home » Modeling » TMF (Xtext) » Custom Lexer and Semantic Predicates issue(Custom Lexer and Semantic Predicates issue)

Show: Today's Messages :: Show Polls :: Message Navigator

Custom Lexer and Semantic Predicates issue [message #1579057]

Thu, 22 January 2015 19:37

Luis De Bello

Messages: 95
Registered: January 2015

Member

Hi guys,

I have one grammar where you can define namespace and son objects something similar like this

%namespace prefix http://www.xtext.org
%output application/xml
{
message: HelloWorld
}

I have two different terminals

terminal UNQUOTED_STRING:
('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '-' |'0'..'9')*;

terminal CUSTOM_STRING:
('a'..'z' | 'A'..'Z') ('a'..'z' | 'A'..'Z' | ':' | '/' | '.' | '0'..'9')*;

As you can see the CUSTOM_STRING includes UNQUOTED_STRING but I want my lexer to emit different tokens depending on context if the line start with %namespace if you emit CUSTOM_STRING in other case UNQUOTED_STRING. I am doing this in order to define the KeyValuePair as UNQUOTED_STRING ":" UNQUOTED_STRING.

To get this work I had to customise my lexer grammar.

RULE_UNQUOTED_STRING : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'-'|'0'..'9')* {if (isNamespacePrefix == true) {isNamespacePrefix = false; isNamespaceUri = true;}};

RULE_CUSTOM_STRING : {isNamespaceUri}?=> ('a'..'z'|'A'..'Z') ('a'..'z'|'A'..'Z'|':'|'/'|'.'|'0'..'9')* {isNamespaceUri=false;};

This is working great the problem that I have is from eclipse editor because if you change the text "application/xml" to application/xm my lexer will try to handle this as CUSTOM_STRING but the semantic predicate will throw exception so I am getting a infinite loop.

To be honest I do not have idea how to solve my issue, maybe someone had a similar issue or can provide me some ideas.

Thanks in advance.

Luis

Report message to a moderator

Re: Custom Lexer and Semantic Predicates issue [message #1782564 is a reply to message #1579057]

Mon, 26 February 2018 14:01

Tamas Miklossy

Messages: 157
Registered: February 2016

Senior Member

Hi Luis,

The exception causing the infinite loop was an org.antlr.runtime.FailedPredicateException, right?

As far as I can see, this problem has already been addressed on 22 Oct 2015 by this commit:
https://github.com/eclipse/xtext-core/commit/34d61d4d80c6992045ed013a0ac4eb337a1f4e87
and should be fixed in Xtext 2.9.

Since I am facing a similar problem with Xtext 2.7.3, I am very interested in to know whether the problem in your case still exist or not.
Furthermore, could you please tell me which lexer (runtime/contentAssist/highlighting) did you customize?

Report message to a moderator

Previous Topic:	Unable to generate parser for a big grammar
Next Topic:	configure multiple generators with XText 2.9

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Thu Apr 25 00:50:09 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter