Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Custom Lexer and Semantic Predicates issue(Custom Lexer and Semantic Predicates issue)
Custom Lexer and Semantic Predicates issue [message #1579057] Thu, 22 January 2015 19:37 Go to next message
Luis De Bello is currently offline Luis De BelloFriend
Messages: 95
Registered: January 2015
Hi guys,

I have one grammar where you can define namespace and son objects something similar like this

%namespace prefix
%output application/xml
message: HelloWorld

I have two different terminals

('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '-' |'0'..'9')*;

('a'..'z' | 'A'..'Z') ('a'..'z' | 'A'..'Z' | ':' | '/' | '.' | '0'..'9')*;

As you can see the CUSTOM_STRING includes UNQUOTED_STRING but I want my lexer to emit different tokens depending on context if the line start with %namespace if you emit CUSTOM_STRING in other case UNQUOTED_STRING. I am doing this in order to define the KeyValuePair as UNQUOTED_STRING ":" UNQUOTED_STRING.

To get this work I had to customise my lexer grammar.

RULE_UNQUOTED_STRING : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'-'|'0'..'9')* {if (isNamespacePrefix == true) {isNamespacePrefix = false; isNamespaceUri = true;}};

RULE_CUSTOM_STRING : {isNamespaceUri}?=> ('a'..'z'|'A'..'Z') ('a'..'z'|'A'..'Z'|':'|'/'|'.'|'0'..'9')* {isNamespaceUri=false;};

This is working great the problem that I have is from eclipse editor because if you change the text "application/xml" to application/xm my lexer will try to handle this as CUSTOM_STRING but the semantic predicate will throw exception so I am getting a infinite loop.

To be honest I do not have idea how to solve my issue, maybe someone had a similar issue or can provide me some ideas.

Thanks in advance.

Re: Custom Lexer and Semantic Predicates issue [message #1782564 is a reply to message #1579057] Mon, 26 February 2018 14:01 Go to previous message
Tamas Miklossy is currently offline Tamas MiklossyFriend
Messages: 6
Registered: February 2016
Junior Member
Hi Luis,

The exception causing the infinite loop was an org.antlr.runtime.FailedPredicateException, right?

As far as I can see, this problem has already been addressed on 22 Oct 2015 by this commit:
and should be fixed in Xtext 2.9.

Since I am facing a similar problem with Xtext 2.7.3, I am very interested in to know whether the problem in your case still exist or not.
Furthermore, could you please tell me which lexer (runtime/contentAssist/highlighting) did you customize?
Previous Topic:Unable to generate parser for a big grammar
Next Topic:Code coverage for Xtend (outside of maven build)
Goto Forum:

Current Time: Thu Mar 22 21:58:58 GMT 2018

Powered by FUDForum. Page generated in 0.01685 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software