Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Terminal Rule Error
icon5.gif  Terminal Rule Error [message #899938] Fri, 03 August 2012 01:53 Go to next message
Kevin Sun is currently offline Kevin Sun
Messages: 32
Registered: August 2010
Location: China
Member
I meet a problem about the terminal rule. The grammar I used is following
grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals hidden(CONDITIONAL_COMPILE_BEGIN)

generate myDsl "http://www.xtext.org/example/mydsl/MyDsl"

Model:
	greetings+=Greeting*;
	
Greeting:
	'#' name=ID '!'	;

terminal INCLUDE:
	'#' ('include'|'INCLUDE') WS ID;


And the following case should be accepted by the grammer
#input!


But I got the error message as following
Multiple markers at this line
- missing EOF at 'ut'
- mismatched character 'p' expecting 'c'

I tried to find the reason. And, I found that the lexer using an DFA to predict which terminal rule should be used. When it find the letter '#' followed by the 'i', it decide to use the terminal rule "INCLUDE". But it fail to recognize the token, because the input is "#input" not "#include".

Beside using backtrack of Lexer, what can I do to solve this problem?
By the way, I need the terminal rule "INCLUDE" in that form. Because I want to treat the string "#include someName" as a token.

[Updated on: Fri, 03 August 2012 02:04]

Report message to a moderator

Re: Terminal Rule Error [message #899939 is a reply to message #899938] Fri, 03 August 2012 02:03 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian Dietrich
Messages: 4399
Registered: July 2009
Senior Member
Hi,

your grammar is ambigous. you may want to use a datatype rule instead


INCLUDE hidden():
'#' ('include'|'INCLUDE') WS ID;

Report message to a moderator

Re: Terminal Rule Error [message #899940 is a reply to message #899939] Fri, 03 August 2012 02:07 Go to previous messageGo to next message
Kevin Sun is currently offline Kevin Sun
Messages: 32
Registered: August 2010
Location: China
Member
thank you for your reply.

I forget to mention that I do need the terminal rule "INCLUDE" in that form. Because I want to treat the string "#include someName" as a token.

Report message to a moderator

Re: Terminal Rule Error [message #899941 is a reply to message #899940] Fri, 03 August 2012 02:08 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian Dietrich
Messages: 4399
Registered: July 2009
Senior Member
What do you mean by:

Quote:
I want to treat the string "#include someName" as a token.

Report message to a moderator

Re: Terminal Rule Error [message #899942 is a reply to message #899941] Fri, 03 August 2012 02:12 Go to previous messageGo to next message
Kevin Sun is currently offline Kevin Sun
Messages: 32
Registered: August 2010
Location: China
Member
I need the Lexer recognize "#include someName" as a token.

If there is no easy way to solve this problem, I'll change it to a datatype rule. But I hope I can remain it as a terminal rule.

Report message to a moderator

Re: Terminal Rule Error [message #899943 is a reply to message #899942] Fri, 03 August 2012 02:14 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian Dietrich
Messages: 4399
Registered: July 2009
Senior Member
Still the question: why?

Report message to a moderator

Re: Terminal Rule Error [message #899944 is a reply to message #899943] Fri, 03 August 2012 02:23 Go to previous messageGo to next message
Kevin Sun is currently offline Kevin Sun
Messages: 32
Registered: August 2010
Location: China
Member
I have done something like preprocess in the Lexer. I know it is a little unreasonable. The "INCLUDE" is a flag that I should filter all the things behind it. In the Lexer, I change the type of the token that has been read to be "Other". All the tokens behind the "INCLUDE" will not be seen by the Parser. And to get the configure information to do this job I have ask another question as below.
http://www.eclipse.org/forums/index.php/m/893934/#msg_893934

Report message to a moderator

Re: Terminal Rule Error [message #899971 is a reply to message #899938] Fri, 03 August 2012 05:42 Go to previous messageGo to next message
Jens Kuenzer is currently offline Jens Kuenzer
Messages: 12
Registered: October 2009
Junior Member
How about:


Greeting:
	NUMBER_SIGN name=ID '!'	;

terminal INCLUDE:
	'#' ('include'|'INCLUDE') WS ID;

terminal NUMBER_SIGN:
	'#' ;


The difference is that the NUMBER_SIGN is listed after the INCLUDE. So If INCLUDE does not match it still can match NUMBER_SIGN. I have not tested it, and still often get confused with TERMINAL rules. I just have recognized that such things can make huge difference.
It has some ugly side effects as you can not use a greeting like this:
#include !

So if possible better use datatype rules.

Report message to a moderator

Re: Terminal Rule Error [message #899973 is a reply to message #899938] Fri, 03 August 2012 05:44 Go to previous message
Sebastian Zarnekow is currently offline Sebastian Zarnekow
Messages: 2788
Registered: July 2009
Senior Member
Did you try to use something like

terminal IdWithHash: '#' ID;

and a value converter that strips the # from that token?

Regards,
Sebastian
--
Looking for professional support for Xtext, Xtend or Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 03.08.12 07:54, schrieb Kevin Sun:
> I meet a problem about the terminal rule. The grammar I used is following
>
> grammar org.xtext.example.mydsl.MyDsl with
> org.eclipse.xtext.common.Terminals hidden(CONDITIONAL_COMPILE_BEGIN)
>
> generate myDsl "http://www.xtext.org/example/mydsl/MyDsl"
>
> Model:
> greetings+=Greeting*;
>
> Greeting:
> '#' name=ID '!' ;
>
> terminal INCLUDE:
> '#' ('include'|'INCLUDE') WS ID;
>
>
> And the following case should be accepted by the grammer
> #input!
>
> But I got the error message as following
> Multiple markers at this line
> - missing EOF at 'ut'
> - mismatched character 'p' expecting 'c'
>
> I tried to find the reason. And, I found that the lexer using an DFA to
> predict which terminal rule should be used. When it find the letter '#'
> followed by the 'i', it decide to use the terminal rule "INCLUDE". But
> it fail to recognize the token, because the input is "#input" not
> "#include".
>
> Beside using backtrack of Lexer, what can I do to solve this problem?

Report message to a moderator

Previous Topic:Dependency in grammars
Next Topic:Couldn't resolve reference to JvmIdentifiableElement 'generateStub'.
Goto Forum:
  

[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Mon May 20 15:29:01 EDT 2013

Powered by FUDForum. Page generated in 0.01723 seconds