Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Custom Lexer and Semantic Predicates issue(Custom Lexer and Semantic Predicates issue)
Custom Lexer and Semantic Predicates issue [message #1579057] Thu, 22 January 2015 19:37 Go to next message
Luis De Bello is currently offline Luis De BelloFriend
Messages: 95
Registered: January 2015
Member
Hi guys,

I have one grammar where you can define namespace and son objects something similar like this

%namespace prefix http://www.xtext.org
%output application/xml
{
message: HelloWorld
}

I have two different terminals

terminal UNQUOTED_STRING:
('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '-' |'0'..'9')*;

terminal CUSTOM_STRING:
('a'..'z' | 'A'..'Z') ('a'..'z' | 'A'..'Z' | ':' | '/' | '.' | '0'..'9')*;

As you can see the CUSTOM_STRING includes UNQUOTED_STRING but I want my lexer to emit different tokens depending on context if the line start with %namespace if you emit CUSTOM_STRING in other case UNQUOTED_STRING. I am doing this in order to define the KeyValuePair as UNQUOTED_STRING ":" UNQUOTED_STRING.

To get this work I had to customise my lexer grammar.

RULE_UNQUOTED_STRING : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'-'|'0'..'9')* {if (isNamespacePrefix == true) {isNamespacePrefix = false; isNamespaceUri = true;}};

RULE_CUSTOM_STRING : {isNamespaceUri}?=> ('a'..'z'|'A'..'Z') ('a'..'z'|'A'..'Z'|':'|'/'|'.'|'0'..'9')* {isNamespaceUri=false;};

This is working great the problem that I have is from eclipse editor because if you change the text "application/xml" to application/xm my lexer will try to handle this as CUSTOM_STRING but the semantic predicate will throw exception so I am getting a infinite loop.

To be honest I do not have idea how to solve my issue, maybe someone had a similar issue or can provide me some ideas.

Thanks in advance.

Luis
Re: Custom Lexer and Semantic Predicates issue [message #1782564 is a reply to message #1579057] Mon, 26 February 2018 14:01 Go to previous message
Tamas Miklossy is currently offline Tamas MiklossyFriend
Messages: 11
Registered: February 2016
Junior Member
Hi Luis,

The exception causing the infinite loop was an org.antlr.runtime.FailedPredicateException, right?

As far as I can see, this problem has already been addressed on 22 Oct 2015 by this commit:
https://github.com/eclipse/xtext-core/commit/34d61d4d80c6992045ed013a0ac4eb337a1f4e87
and should be fixed in Xtext 2.9.

Since I am facing a similar problem with Xtext 2.7.3, I am very interested in to know whether the problem in your case still exist or not.
Furthermore, could you please tell me which lexer (runtime/contentAssist/highlighting) did you customize?
Previous Topic:Unable to generate parser for a big grammar
Next Topic:Code coverage for Xtend (outside of maven build)
Goto Forum:
  


Current Time: Thu Jun 21 06:57:53 GMT 2018

Powered by FUDForum. Page generated in 0.02265 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top