Eclipse Community Forums: TMF (Xtext)

Home » Modeling » TMF (Xtext) » Terminal confusion

Thu, 01 December 2011 08:34

Eclipse User

Dear Xtext-Gurus,

in which order are terminals matched during the parsing?

I am running into the following problem. My grammar both has possibilities to specify multiplicities

Multiplicity: "[" lower=INT ('..' (upper=INT|unbounded?='*'))? "]";

as well as NUMBER (used at some other place)

    terminal NUMBER returns ecore::EBigDecimal:
	('-')?('0'..'9')+ ('.' ('0'..'9')+);

Before introducing NUMBER, multiplicities worked fine with a syntax like this :

 [2..3]

However, this now results in:

Multiple markers at this line
- required (...)+ loop did not match anything at character '.'
- extraneous input '.' expecting RULE_INT

and I have to add extra whitespace.


 [2..3]

I have now idea why it gets confused, because NUMBER should not match anything in that statement.

Any ideas?

Thanks,

Andreas

Re: Terminal confusion [message #760147 is a reply to message #760144]

Thu, 01 December 2011 08:39

Eclipse User

Hi,

terminals are *greedy*. As soon as "2." is read, the lexer goes on trying to match NUMBER and fails. This is why you would define number as a datatype rule rather than a terminal rule.

Alex

Re: Terminal confusion [message #760148 is a reply to message #760147]

Thu, 01 December 2011 08:41

Eclipse User

Thanks, that helps. So greedy in that context means not only that the regexp is greedy, but that ANTLR/Xtext does not try any other combinations.

Re: Terminal confusion [message #760163 is a reply to message #760148]

Thu, 01 December 2011 08:59

Eclipse User

Indeed. Lexing is done before parsing, independent of the rest of the grammar (i.e. just terminal rules and keywords). As long as the token can be made "longer", the lexer goes on. And as there is a rule that fits the prefix "2.", it is chosen even if it is a dead end.

This is why you try to reduce the number of terminal rules to the absolute minimum and do as much using datatype rules as possible.

Alex

Re: Terminal confusion [message #760167 is a reply to message #760144]

Thu, 01 December 2011 09:08

Eclipse User

Andreas,

please use a data type rule to model NUMBERs.

Regards,
Sebastian
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 01.12.11 14:34, schrieb Andreas Graf:
> Dear Xtext-Gurus,
>
> in which order are terminals matched during the parsing?
>
> I am running into the following problem. My grammar both has
> possibilities to specify multiplicities
>
> Multiplicity: "[" lower=INT ('..' (upper=INT|unbounded?='*'))? "]";
>
>
> as well as NUMBER (used at some other place)
>
>
> terminal NUMBER returns ecore::EBigDecimal:
> ('-')?('0'..'9')+ ('.' ('0'..'9')+);
>
>
> Before introducing NUMBER, multiplicities worked fine with a syntax like
> this :
>
> [2..3]
>
> However, this now results in:
>
> Multiple markers at this line
> - required (...)+ loop did not match anything at character '.'
> - extraneous input '.' expecting RULE_INT
>
>
> and I have to add extra whitespace.
>
> [2..3]
>
>
> I have now idea why it gets confused, because NUMBER should not match
> anything in that statement.
>
> Any ideas?
>
> Thanks,
>
> Andreas
>

Re: Terminal confusion [message #760213 is a reply to message #760167]

Thu, 01 December 2011 10:51

Eclipse User

I used datatype rule for NUMBER, but that still gets into conflicts with the terminal INT. I tried to replace INT in my grammar with DINT, but that then conflicts with the parts where I reference an XExpression.

XExpression internally uses INT, which is not datatype rule (shouldn't it be, then?)

Redefining XIntLiteral to use my DINT results in an ambiguous grammar Sad

Did anybody already manage to integrate NUMBERs with INTs in their grammars (and I need the ".." syntax for multiplicities

Regards,

Andreas

Re: Terminal confusion [message #760216 is a reply to message #760213]

Thu, 01 December 2011 10:56

Eclipse User

what does your NUMBER rule lool like?
make sure it does not start with
"terminal NUMBER..." because then it is a terminal rule. It should look something like
NUMBER returns ecore::EBigDecimal: ('-')?('0'..'9')+ ('.' ('0'..'9')+)?;

Re: Terminal confusion [message #760309 is a reply to message #760216]

Thu, 01 December 2011 15:05

Eclipse User

Or probably

Number returns EBigDecimal hidden(): '-'? INT ('.' INT)?;

Just to be picky :-)

Andreas, a terminal can usually not conflict with a data type rule. You
have to make sure to not use Number and INT in an alternative.

Regards,
Sebastian
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 01.12.11 16:56, schrieb Alexander Nittka:
> what does your NUMBER rule lool like?
> make sure it does not start with
> "terminal NUMBER..." because then it is a terminal rule. It should look
> something like
> NUMBER returns ecore::EBigDecimal: ('-')?('0'..'9')+ ('.' ('0'..'9')+)?;

Re: Terminal confusion [message #760333 is a reply to message #760216]

Thu, 01 December 2011 17:35

Eclipse User

Hi Alexander,

is "('0'..'9')" supposed to work in data type rules? For me it seemed to only work for terminals and I had to use "'0'|'1'|'2' etc." in data type rules.

Andreas

Re: Terminal confusion [message #760359 is a reply to message #760333]

Fri, 02 December 2011 01:48

Eclipse User

Sorry,
of course you reference other terminal or datatype rules in the Number-rule:
Number: ('-')? INT ('.'INT)?;

Alex

Previous Topic:	Formatting-Indent, documentation out of date
Next Topic:	Floats and qualified ids...

Goto Forum:

-=] Back to Top [=-

Current Time: Fri Jul 04 15:48:04 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter