Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Terminal confusion
Terminal confusion [message #760144] Thu, 01 December 2011 13:34 Go to next message
Andreas Graf is currently offline Andreas GrafFriend
Messages: 211
Registered: July 2009
Senior Member
Dear Xtext-Gurus,

in which order are terminals matched during the parsing?

I am running into the following problem. My grammar both has possibilities to specify multiplicities
Multiplicity: "[" lower=INT ('..' (upper=INT|unbounded?='*'))? "]";


as well as NUMBER (used at some other place)

    terminal NUMBER returns ecore::EBigDecimal:
	('-')?('0'..'9')+ ('.' ('0'..'9')+);


Before introducing NUMBER, multiplicities worked fine with a syntax like this :
 [2..3] 


However, this now results in:
Multiple markers at this line
- required (...)+ loop did not match anything at character '.'
- extraneous input '.' expecting RULE_INT


and I have to add extra whitespace.

 [2..3] 



I have now idea why it gets confused, because NUMBER should not match anything in that statement.

Any ideas?

Thanks,

Andreas
Re: Terminal confusion [message #760147 is a reply to message #760144] Thu, 01 December 2011 13:39 Go to previous messageGo to next message
Alexander Nittka is currently offline Alexander NittkaFriend
Messages: 1193
Registered: July 2009
Senior Member
Hi,

terminals are *greedy*. As soon as "2." is read, the lexer goes on trying to match NUMBER and fails. This is why you would define number as a datatype rule rather than a terminal rule.

Alex


Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext@itemis.de
Re: Terminal confusion [message #760148 is a reply to message #760147] Thu, 01 December 2011 13:41 Go to previous messageGo to next message
Andreas Graf is currently offline Andreas GrafFriend
Messages: 211
Registered: July 2009
Senior Member
Thanks, that helps. So greedy in that context means not only that the regexp is greedy, but that ANTLR/Xtext does not try any other combinations.
Re: Terminal confusion [message #760163 is a reply to message #760148] Thu, 01 December 2011 13:59 Go to previous messageGo to next message
Alexander Nittka is currently offline Alexander NittkaFriend
Messages: 1193
Registered: July 2009
Senior Member
Indeed. Lexing is done before parsing, independent of the rest of the grammar (i.e. just terminal rules and keywords). As long as the token can be made "longer", the lexer goes on. And as there is a rule that fits the prefix "2.", it is chosen even if it is a dead end.

This is why you try to reduce the number of terminal rules to the absolute minimum and do as much using datatype rules as possible.

Alex


Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext@itemis.de
Re: Terminal confusion [message #760167 is a reply to message #760144] Thu, 01 December 2011 14:08 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3107
Registered: July 2009
Senior Member
Andreas,

please use a data type rule to model NUMBERs.

Regards,
Sebastian
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com


Am 01.12.11 14:34, schrieb Andreas Graf:
> Dear Xtext-Gurus,
>
> in which order are terminals matched during the parsing?
>
> I am running into the following problem. My grammar both has
> possibilities to specify multiplicities
>
> Multiplicity: "[" lower=INT ('..' (upper=INT|unbounded?='*'))? "]";
>
>
> as well as NUMBER (used at some other place)
>
>
> terminal NUMBER returns ecore::EBigDecimal:
> ('-')?('0'..'9')+ ('.' ('0'..'9')+);
>
>
> Before introducing NUMBER, multiplicities worked fine with a syntax like
> this :
>
> [2..3]
>
> However, this now results in:
>
> Multiple markers at this line
> - required (...)+ loop did not match anything at character '.'
> - extraneous input '.' expecting RULE_INT
>
>
> and I have to add extra whitespace.
>
> [2..3]
>
>
> I have now idea why it gets confused, because NUMBER should not match
> anything in that statement.
>
> Any ideas?
>
> Thanks,
>
> Andreas
>
Re: Terminal confusion [message #760213 is a reply to message #760167] Thu, 01 December 2011 15:51 Go to previous messageGo to next message
Andreas Graf is currently offline Andreas GrafFriend
Messages: 211
Registered: July 2009
Senior Member
I used datatype rule for NUMBER, but that still gets into conflicts with the terminal INT. I tried to replace INT in my grammar with DINT, but that then conflicts with the parts where I reference an XExpression.

XExpression internally uses INT, which is not datatype rule (shouldn't it be, then?)

Redefining XIntLiteral to use my DINT results in an ambiguous grammar Sad

Did anybody already manage to integrate NUMBERs with INTs in their grammars (and I need the ".." syntax for multiplicities

Regards,

Andreas

Re: Terminal confusion [message #760216 is a reply to message #760213] Thu, 01 December 2011 15:56 Go to previous messageGo to next message
Alexander Nittka is currently offline Alexander NittkaFriend
Messages: 1193
Registered: July 2009
Senior Member
what does your NUMBER rule lool like?
make sure it does not start with
"terminal NUMBER..." because then it is a terminal rule. It should look something like
NUMBER returns ecore::EBigDecimal: ('-')?('0'..'9')+ ('.' ('0'..'9')+)?;


Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext@itemis.de
Re: Terminal confusion [message #760309 is a reply to message #760216] Thu, 01 December 2011 20:05 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3107
Registered: July 2009
Senior Member
Or probably

Number returns EBigDecimal hidden(): '-'? INT ('.' INT)?;

Just to be picky :-)

Andreas, a terminal can usually not conflict with a data type rule. You
have to make sure to not use Number and INT in an alternative.

Regards,
Sebastian
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com


Am 01.12.11 16:56, schrieb Alexander Nittka:
> what does your NUMBER rule lool like?
> make sure it does not start with
> "terminal NUMBER..." because then it is a terminal rule. It should look
> something like
> NUMBER returns ecore::EBigDecimal: ('-')?('0'..'9')+ ('.' ('0'..'9')+)?;
Re: Terminal confusion [message #760333 is a reply to message #760216] Thu, 01 December 2011 22:35 Go to previous messageGo to next message
Andreas Graf is currently offline Andreas GrafFriend
Messages: 211
Registered: July 2009
Senior Member
Hi Alexander,

is "('0'..'9')" supposed to work in data type rules? For me it seemed to only work for terminals and I had to use "'0'|'1'|'2' etc." in data type rules.

Andreas
Re: Terminal confusion [message #760359 is a reply to message #760333] Fri, 02 December 2011 06:48 Go to previous message
Alexander Nittka is currently offline Alexander NittkaFriend
Messages: 1193
Registered: July 2009
Senior Member
Sorry,
of course you reference other terminal or datatype rules in the Number-rule:
Number: ('-')? INT ('.'INT)?;

Alex


Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext@itemis.de
Previous Topic:Formatting-Indent, documentation out of date
Next Topic:Floats and qualified ids...
Goto Forum:
  


Current Time: Sun Mar 29 13:18:29 GMT 2020

Powered by FUDForum. Page generated in 0.06342 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top