Eclipse Community Forums: TMF (Xtext) » Problem with grammar Terminal element

Help

Home

Home » Modeling » TMF (Xtext) » Problem with grammar Terminal element

Show: Today's Messages :: Show Polls :: Message Navigator

Problem with grammar Terminal element [message #716058]

Tue, 16 August 2011 11:10

Elhamlaoui Mahmoud

Messages: 268
Registered: March 2010

Senior Member

Hi,

Plz,i have a problem with the following grammar, what i tried to do is that any thing typed betwwen '>' and '</' should be considerded as Text but something is missing

Racine returns Racine:
	{Racine}
	'<racine '
		('id' '=' id=String0)?
	'>'
		(tronc=Tronc)?
	</racine>

Tronc returns Tronc :
	{Tronc}
	'<tronc '
		('id' '=' id=String0)?
		'>'
		(leafs+=Leafs)*
   '</tronc>'
    ;
	
Leafs returns Leafs:
 Text ;	
 
Text returns Text:
{Text}

value = String3
    ;
    String3 returns type::String:
	TEXTSTRUCT  /* TODO: implement this rule and an appropriate IValueConverter */
	;
terminal TEXTSTRUCT:  '>' -> '</';

Any ideas

Report message to a moderator

Re: Problem with grammar Terminal element [message #716060 is a reply to message #716058]

Tue, 16 August 2011 11:26

Alexander Nittka

Messages: 1193
Registered: July 2009

Senior Member

Hi,

note that you have other keywords that contain ">" and "</" (so even if the grammar "would work", it would not, as ">" would have to be read twice, once for closing a tag and once as the start of the TEXTSTRUCT terminal). So lexing will fail already. Note that the lexer works without context, trying to make tokens as long as possible. As a consequence a TESTSTRUCT-token will already start at the end of the opening racine-tag. There are several possible approaches, in particular the following two:
-write your own lexer
-make Textstruct a datatype rule (or rather modify String3)

If you gave the full grammar rather than a simplified version, the second alternative was feasible:
String3 hidden(): ID|INT|STRING|WS|ANY_OTHER|"="|list_of_other_allowed_keywords;

Alex

P.S.: Don't use white spaces in a keyword ('<racine' instead of '<racine '; better yet '<''racine').

Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext@itemis.de

Report message to a moderator

Re: Problem with grammar Terminal element [message #716065 is a reply to message #716058]

Tue, 16 August 2011 11:40

Henrik Lindberg

Messages: 2509
Registered: July 2009

Senior Member

For starters, if you have a terminal that eats '>' -> '</' you will
probably not get '</tronc>' as a keyword as it starts with the terminal
'</'. You have keywords with space in them (that is always a source of
problems).

Things like this are much easier to handle at the lexical level -
provide an external lexer instead and have it gobble everything between
'>' and '</' but have it deliver the start/end as tokens - then your
grammar is very straight forward - i.e. something like

Racine :
'<' 'racine' ('id' '=' id=STRING)? '>'
tronc=Tronc?
'</' 'racine' '>'
;

etc..

Done this way, you can handle whitespace and comments as hidden, and you
don't have to worry about input like:
"< racine
id = "hello"
>"

Irrespective of if you solved in an external lexer, or done with data
rules and terminals in the grammar you need to deal with escapes and if
the '>' and/or '</' appear in strings (unless the use of > < are
required in the source).

- henrik

On 8/16/11 1:10 PM, elhamlaoui wrote:
> Hi,
>
> Plz,i have a problem with the following grammar, what i tried to do is
> that any thing typed betwwen '>' and '</' should be considerded as Text
> but something is missing
>
>
> Racine returns Racine:
> {Racine}
> '<racine '
> ('id' '=' id=String0)?
> '>'
> (tronc=Tronc)?
> </racine>
>
> Tronc returns Tronc :
> {Tronc}
> '<tronc '
> ('id' '=' id=String0)?
> '>'
> (leafs+=Leafs)*
> '</tronc>'
> ;
>
> Leafs returns Leafs:
> Text ;
>
> Text returns Text:
> {Text}
>
> value = String3
> ;
> String3 returns type::String:
> TEXTSTRUCT /* TODO: implement this rule and an appropriate
> IValueConverter */
> ;
> terminal TEXTSTRUCT: '>' -> '</';
>
> Any ideas

Report message to a moderator

Re: Problem with grammar Terminal element [message #716100 is a reply to message #716065]

Tue, 16 August 2011 13:02

Elhamlaoui Mahmoud

Messages: 268
Registered: March 2010

Senior Member

Hi thanks both of you,

the reason i have done '<racine ' with a space is that if i dont do so 'racineid' would be a correct syntax and its not the case ther emust have at least one space betwenn the racine and id. is there other way to do so?

thanks a lot

Report message to a moderator

Re: Problem with grammar Terminal element [message #716106 is a reply to message #716100]

Tue, 16 August 2011 13:20

Alexander Nittka

Messages: 1193
Registered: July 2009

Senior Member

Hi,

you did not post the full grammar. It would be interesting in particular whether you have a terminal rule for white spaces at all and which terminal rules are hidden. In a "normal" setup "racineid" would not be valid as it could not be tokenised as racine and id separately (the ID terminal would pick it up). It is hard to give a good solution. You could make the WS rule non-hidden (for selected rules and then indicat where a white space is mandatory and where not), or you introduce an ID terminal that would swallow concatenated keywords.

Alex

Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext@itemis.de

Report message to a moderator