Eclipse Community Forums: TMF (Xtext) » Parsing Ambiguity Problem

Help

Home

Home » Modeling » TMF (Xtext) » Parsing Ambiguity Problem

Show: Today's Messages :: Show Polls :: Message Navigator

Parsing Ambiguity Problem [message #902351]

Fri, 17 August 2012 09:05

Johan Eker

Messages: 6
Registered: August 2012

Junior Member

Hi All,

I have parsing issue which I fail to resolve and kindly ask for a little bit of help. I have put together a small sample language that demonstrates problem. The problem I have is to distinguish between a float expression (e.g. "3.14") and a list expression (e.g. "1..4" that represents the integer list [1,2,3,4]). The problem that I fail to get the generated xtext parser to accept the list expression unless additional white space is inserted, i.e. "1 ..4" works while "1..4" failes.
The fact that the float is a terminal and the list is a production might have something to do my headache.

The sample language consist of variable declarations and expressions:

A = 3.14 // Real valued variable with initial value
B = 3+4 // Integer valued variable with initial value
C = 1..4 // List valued variable with initial value consisting of the integer lists [1,2,3,4]
D = X.d + Y.a // We also support field accesses in expression
// (although in the sample language there is now way to actually define fields:-)

The grammar is as follows:

--------------------------8<--------------------------------
grammar org.xtext.example.mydsl.TestLang with org.eclipse.xtext.common.Terminals

import "http://www.eclipse.org/emf/2002/Ecore" as ecore

generate testLang "<URL removed due to forum restrictions on newbies>"

Namespace:
'namespace' name = ID ':'
( variables += Variable) *
'end';
Variable: name = ID '=' value = Expression ';';

Expression:
ExpressionSimple ({Expression.left=current} operator=('+' | '-' | '..') right=ExpressionSimple)?;
ExpressionSimple:
ExpressionInteger | ExpressionFloat | ExpressionOctal | ExpressionVariable ;

ExpressionInteger: value=DECIMAL;

ExpressionOctal: value=OCTAL;

terminal DECIMAL returns ecore::ELong : ('1'..'9') ('0'..'9')*;

terminal OCTAL returns ecore::ELong : '0' ('0'..'7')*;

ExpressionVariable: value=[Variable] ('.' fields+=ID)*;

ExpressionFloat: value=FLOAT;

terminal FLOAT returns ecore::EFloat :
('0'..'9')+ '.' ('0'..'9')* (('e'|'E') ('+' | '-')? ('0'..'9')+)?
| '.' ('0'..'9')+ (('e'|'E') ('+' | '-')? ('0'..'9')+)?
| ('0'..'9')+ ('e'|'E') ('+' | '-')? ('0'..'9')+;

--------------------------8<--------------------------------

So now the problem. Considers the follow program it fails to parse the list expression that initializes variable B4. The reason seems to be a conflict with the float terminal.

namespace NS1:
B1 = 2;
B2 = B1..1;
B3 = 1 + 1;
X = 1;
Y = 1.1;
Z = X.a.b;
B4 = 1..1; // <--gives error
B5 = 1 ..1;
B6 = B1.a..2;
B7 = B1.a.b..B2.a.d;
end

Any insight into solving this problem is greatly appreciated.

Thanks,
Johan

Report message to a moderator

Re: Parsing Ambiguity Problem [message #902354 is a reply to message #902351]

Fri, 17 August 2012 09:33

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

Hid you try to use a data type rule instead of a terminal for the
float thing

--
Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext at itemis dot de

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Parsing Ambiguity Problem [message #902380 is a reply to message #902354]

Fri, 17 August 2012 11:46

Johan Eker

Messages: 6
Registered: August 2012

Junior Member

Hi Christian,

Thanks for your response. Yes I did try that. Something like

FLOAT returns ecore::EFloat :
NUMBER+ '.' NUMBER* (('e'|'E') ('+' | '-')? NUMBER+)?
| '.' NUMBER+ (('e'|'E') ('+' | '-')? NUMBER+)?
| NUMBER+ ('e'|'E') ('+' | '-')? NUMBER+;

terminal NUMBER returns ecore::EFloat :
('0' .. '9');

However, then I run into a bunch of other conflicts between my numeric terminal and it would require rewrite most of them. But you are correct in that it does resolve the ambiguity problem.
Maybe that is what I need to do, unless there is some more obvious way.

/Johan

Report message to a moderator

Re: Parsing Ambiguity Problem [message #902384 is a reply to message #902380]

Fri, 17 August 2012 12:01

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

AS long AS it is a terminal you have no chance since they are parsed
eager and collect as much as the can

--
Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext at itemis dot de

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Parsing Ambiguity Problem [message #902396 is a reply to message #902380]

Fri, 17 August 2012 13:07

Henrik Lindberg

Messages: 2509
Registered: July 2009

Senior Member

I did it like this:

terminal INT : ('0'..'9')+;
REAL hidden(): INT '.' (EXT_INT | INT);
terminal EXT_INT: INT ('e'|'E')('-'|'+') INT;

I could however (in the grammar this was used) not support floats
without an integral part (e.g. ".3e+2") - but the approach is perhaps
worth exploring as it avoids a bunch of problems with 'e' being a
keyword (among other things).

Regards
- henrik

On 2012-17-08 13:46, Johan Eker wrote:
> Hi Christian,
> Thanks for your response. Yes I did try that. Something like
>
> FLOAT returns ecore::EFloat :
> NUMBER+ '.' NUMBER* (('e'|'E') ('+' | '-')? NUMBER+)?
> | '.' NUMBER+ (('e'|'E') ('+' | '-')? NUMBER+)?
> | NUMBER+ ('e'|'E') ('+' | '-')? NUMBER+;
>
> terminal NUMBER returns ecore::EFloat :
> ('0' .. '9');
>
> However, then I run into a bunch of other conflicts between my numeric
> terminal and it would require rewrite most of them. But you are correct
> in that it does resolve the ambiguity problem.
> Maybe that is what I need to do, unless there is some more obvious way.
>
> /Johan
>
>

Report message to a moderator