Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Shift-reduce conflict between grammar rule and terminal in Xtext
Shift-reduce conflict between grammar rule and terminal in Xtext [message #902264] Thu, 16 August 2012 16:14 Go to next message
Tu Do is currently offline Tu Do
Messages: 21
Registered: August 2012
Junior Member
This is the grammar:

    grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals

    generate myDsl "MyDsl"

    START: elem += DOG ;

    DOG: 'DOG' INT ';' ;

    terminal CAT : ('A'..'Z')('A'..'Z')('A'..'Z')' '('0'..'9')+;


When Xtext sees the line like DOG 1234, it can't resolve and will give the error "mismatch input DOG 1234 expecting DOG" something like that. I thought that when Xtext encounters a keyword, it should have higher precedence than the terminal. But it seems like it's collided. How can I fix this?

Note that DOG and CAT is used in different placed (i.e. the terminal CAT is used in other rule, but its definition conflicts with DOG rule)
Re: Shift-reduce conflict between grammar rule and terminal in Xtext [message #902288 is a reply to message #902264] Thu, 16 August 2012 19:29 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik Lindberg
Messages: 2500
Registered: July 2009
Senior Member
How to make a cat sound like a dog: drench it in petrol and light a match...

Translated to Xtext:

In your example, the input "DOG 1" is a CAT, and not keyword 'DOG'
followed by INT.

You can instead do:

DOG : CAT ';'

And have a value converter that makes sure that the value of the lexed
CAT is a DOG :)

Meooow
- henrik


On 2012-16-08 18:14, Tu Do wrote:
> This is the grammar:
>
>
> grammar org.xtext.example.mydsl.MyDsl with
> org.eclipse.xtext.common.Terminals
>
> generate myDsl "MyDsl"
>
> START: elem += DOG ;
>
> DOG: 'DOG' INT ';' ;
>
> terminal CAT : ('A'..'Z')('A'..'Z')('A'..'Z')' '('0'..'9')+;
>
>
> When Xtext sees the line like DOG 1234, it can't resolve and will give
> the error "mismatch input DOG 1234 expecting DOG" something like that. I
> thought that when Xtext encounters a keyword, it should have higher
> precedence than the terminal. But it seems like it's collided. How can I
> fix this?
>
> Note that DOG and CAT is used in different placed (i.e. the terminal CAT
> is used in other rule, but its definition conflicts with DOG rule)
Re: Shift-reduce conflict between grammar rule and terminal in Xtext [message #902316 is a reply to message #902288] Fri, 17 August 2012 02:51 Go to previous messageGo to next message
Tu Do is currently offline Tu Do
Messages: 21
Registered: August 2012
Junior Member
@Henrik Lindberg Certainly, making a DOG sounds like a CAT is the easiest way. However, in my case, a DOG must be a DOG and cannot be mixed with a CAT Sad. I found a simpler solution:

    grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals

    generate myDsl "MyDsl"

    START: elem += DOG ;

    DOG: 'DOG' INT ';' ;

    CAT: 'CAT' ALPHA+' 'INT ';' ;

    terminal ALPHA : ('A'..'Z');


Now, it is distinguishable. But I think your solution is proper for this case in general. I will look into value converter to convert from a given general input to a specific input. Thanks for this suggestion.

[Updated on: Fri, 17 August 2012 06:29]

Report message to a moderator

Re: Shift-reduce conflict between grammar rule and terminal in Xtext [message #902328 is a reply to message #902316] Fri, 17 August 2012 06:10 Go to previous messageGo to next message
Alexander Nittka is currently offline Alexander Nittka
Messages: 1156
Registered: July 2009
Senior Member
Hi,

keywords have a higher precedence than other terminals indeed, but only if they lead to a longer token. Your original CAT terminal definition included a white space, so "DOG 1234" was picked up by that terminal.

With your current ALPHA terminal rule you will have other problems. It conflicts with the default ID terminal which you import via "with ...Terminals".

"CAT ABC 12;" will not satisfy the CAT datatype rule.

Alex


Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext@itemis.de
Re: Shift-reduce conflict between grammar rule and terminal in Xtext [message #902365 is a reply to message #902328] Fri, 17 August 2012 10:51 Go to previous messageGo to next message
Tu Do is currently offline Tu Do
Messages: 21
Registered: August 2012
Junior Member
Alexander Nittka wrote on Fri, 17 August 2012 02:10
Hi,

keywords have a higher precedence than other terminals indeed, but only if they lead to a longer token. Your original CAT terminal definition included a white space, so "DOG 1234" was picked up by that terminal.

With your current ALPHA terminal rule you will have other problems. It conflicts with the default ID terminal which you import via "with ...Terminals".

"CAT ABC 12;" will not satisfy the CAT datatype rule.

Alex

Hi Alex,

It's true as you said. I got a conflict with terminal ID. However, with my solution, I can make DOG and CAT to be keywords, which are required. Further more, I need some rules which consist of one keyword and one alphabet character.

The approach suggested by Hendrick worked, but the keyword can't be highlighted.
Re: Shift-reduce conflict between grammar rule and terminal in Xtext [message #902381 is a reply to message #902365] Fri, 17 August 2012 11:53 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik Lindberg
Messages: 2500
Registered: July 2009
Senior Member
On 2012-17-08 12:51, Tu Do wrote:

> The approach suggested by Hendrick worked, but the keyword can't be
> highlighted.

You can highlight them as keywords using semantic highlighting.

- henrik
Re: Shift-reduce conflict between grammar rule and terminal in Xtext [message #902401 is a reply to message #902381] Fri, 17 August 2012 13:24 Go to previous messageGo to next message
Tu Do is currently offline Tu Do
Messages: 21
Registered: August 2012
Junior Member
Henrik Lindberg wrote on Fri, 17 August 2012 07:53
On 2012-17-08 12:51, Tu Do wrote:

> The approach suggested by Hendrick worked, but the keyword can't be
> highlighted.

You can highlight them as keywords using semantic highlighting.

- henrik

So, other than the grammar rule, can we specify tokens which are not in the grammar to be keywords?

Also earlier you said that using value converter to check the lexed cat is a dog. Shouldn't this be the job of validtor? I think value converter is just for converting from object to object only.

Re: Shift-reduce conflict between grammar rule and terminal in Xtext [message #902423 is a reply to message #902401] Fri, 17 August 2012 14:22 Go to previous message
Henrik Lindberg is currently offline Henrik Lindberg
Messages: 2500
Registered: July 2009
Senior Member
On 2012-17-08 15:24, Tu Do wrote:
> Henrik Lindberg wrote on Fri, 17 August 2012 07:53
>> On 2012-17-08 12:51, Tu Do wrote:
>>
>> > The approach suggested by Hendrick worked, but the keyword can't be
>> > highlighted.
>>
>> You can highlight them as keywords using semantic highlighting.
>>
>> - henrik
>
> So, other than the grammar rule, can we specify tokens which are not in
> the grammar to be keywords?
>
Out of the box you get a mapping from lexer tokens to highlighting
specs. And also out of the box all keywords are highlighted in the
Eclipse style.

You can customize this several ways.

The simplest is semantic highlighting which kicks in asynchronously. You
can then set/change the highlighting of anything (it is based on offsets
and lengths in the source text). (See PPSemanticHighlighter in
cloudsmith/geppetto @github for an example of semantic and textual
highlighting).

A more advanced way is to customize the lexer used for highlighting - it
does not have to have the same tokens etc. as the real grammar - but it
must naturally be matched with the specification in the highlighter.

The pros and cons are naturally that you get the token based lexer for
free and it provides very fast highlighting - the semantic one, since it
is asynchronous has a sometimes noticeable delay (first painted plain,
then as keyword, etc). With more work (customized highlighting lexer)
you may have difficulties expressing the rules if you need semantic
information + it is more work to write it.

> Also earlier you said that using value converter to check the lexed cat
> is a dog. Shouldn't this be the job of validtor? I think value converter
> is just for converting from object to object only.
>
That works too - depends on how you use the result. Say you assign a
boolean based on if you observed a DOG or not, and you have a rule that
accepts DOG, CAT, (and a bunch of other animals) as string, then you
have no model element to attach the check to. You would need a model
element representing DOG, that assigns the string to be able to validate
it with a check.

Hope that helps.

Regards
- henrik
Previous Topic:Reload model after changing it in the background
Next Topic:How do you extract DSL out of Java objects?
Goto Forum:
  


Current Time: Fri Sep 19 11:57:00 GMT 2014

Powered by FUDForum. Page generated in 0.03135 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software