Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » [Xtext] Serializing an expression DSL with parentheses and different operator priorities
[Xtext] Serializing an expression DSL with parentheses and different operator priorities [message #641997] Mon, 29 November 2010 13:44 Go to next message
Cédric Vidal is currently offline Cédric VidalFriend
Messages: 101
Registered: July 2009
Senior Member
This is a multi-part message in MIME format.
--------------010801080105030909000700
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

Hi guys,

I'm facing a difficulty with Xtext. I'd like the following DSL to be
serializable (I'm running Xtext 1.0.0):

Expression : Addition;

Addition returns Expression :
Multiplication ({Operation.left=current} op=('+'|'-') right=Addition)?
;

Multiplication returns Expression :
Literal ({Operation.left=current} op=('*'|'/') right=Multiplication)?
;

Literal returns Expression:
'(' Expression ')'
| {IntLiteral} value=INT
;

The following model "(5 + 3) * 2" serializes as "5 + 3 * 2" which is not
valid as the parentheses are mandatory to preserve operator priority. It
could serialize either as "(5 + 3) * 2" or "((5 + 3) * 2)" in order to
be valid.

Usually for this kind of problem I use the IValueSerializer API but in
that case, parenthetis are not considered unassigned values.

So I'd like to know where to customize the serialization process in
order to add '(' and ')' keywords where appropriate.

Please find attached a sample project with unit tests.

Regards,

C
Re: [Xtext] Serializing an expression DSL with parentheses and different operator priorities [message #642007 is a reply to message #641997] Mon, 29 November 2010 14:22 Go to previous messageGo to next message
Meinte Boersma is currently offline Meinte BoersmaFriend
Messages: 434
Registered: July 2009
Location: Leiden, Netherlands
Senior Member
If there's no need to go with the usual serializer, you could always use an "old-fashioned" Xpand template to serialize it in a way which makes sense, possibly combining it with some Xtend to "look ahead" to see whether higher-precedence binary operators following the current AST node require parentheses around the latter. (Or you just hack it together in even more old-fashioned Java Wink After all, the grammar isn't very big.)

Re: [Xtext] Serializing an expression DSL with parentheses and different operator priorities [message #642018 is a reply to message #642007] Mon, 29 November 2010 14:52 Go to previous messageGo to next message
Cédric Vidal is currently offline Cédric VidalFriend
Messages: 101
Registered: July 2009
Senior Member
Hi Meinte,

Actually, the sample expression DSL is just a snippet to illustrate my
case. It is part of a much larger DSL and I can't afford going the code
generation template way.

I really need to find a way to customize the serialization process to
handle that case.

Le 29/11/2010 15:22, Meinte Boersma a écrit :
> If there's no need to go with the usual serializer, you could always use
> an "old-fashioned" Xpand template to serialize it in a way which makes
> sense, possibly combining it with some Xtend to "look ahead" to see
> whether higher-precedence binary operators following the current AST
> node require parentheses around the latter. (Or you just hack it
> together in even more old-fashioned Java ;) After all, the grammar isn't
> very big.)
Re: [Xtext] Serializing an expression DSL with parentheses and different operator priorities [message #642023 is a reply to message #642018] Mon, 29 November 2010 15:24 Go to previous messageGo to next message
Cédric Vidal is currently offline Cédric VidalFriend
Messages: 101
Registered: July 2009
Senior Member
Actually, I just realized that in the sample, the parentheses are
actually correctly serialized.

It's in the bigger DSL that they are not. I have to figure out what is
the difference between the two that makes the serialization fail.

Le 29/11/2010 15:54, Cédric Vidal a écrit :
> Hi Meinte,
>
> Actually, the sample expression DSL is just a snippet to illustrate my
> case. It is part of a much larger DSL and I can't afford going the code
> generation template way.
>
> I really need to find a way to customize the serialization process to
> handle that case.
>
> Le 29/11/2010 15:22, Meinte Boersma a écrit :
>> If there's no need to go with the usual serializer, you could always use
>> an "old-fashioned" Xpand template to serialize it in a way which makes
>> sense, possibly combining it with some Xtend to "look ahead" to see
>> whether higher-precedence binary operators following the current AST
>> node require parentheses around the latter. (Or you just hack it
>> together in even more old-fashioned Java ;) After all, the grammar isn't
>> very big.)
>
Re: [Xtext] Serializing an expression DSL with parentheses and different operator priorities [message #642029 is a reply to message #641997] Mon, 29 November 2010 15:35 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3118
Registered: July 2009
Senior Member
Hi Cédric,

your expressions are right associative, so the serializer is completely
right:

Addition returns Expression :
Multiplication ({Operation.left=current} op=('+'|'-') right=Addition)?
;

should actually read

Addition returns Expression :
Multiplication ({Operation.left=current} op=('+'|'-')
right=Multiplication/* <- !!*/)?
;

Regards,
Sebastian
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 29.11.10 14:44, schrieb Cédric Vidal:
> Addition returns Expression :
> Multiplication ({Operation.left=current} op=('+'|'-') right=Addition)?
> ;
Re: [Xtext] Serializing an expression DSL with parentheses and different operator priorities [message #642030 is a reply to message #642029] Mon, 29 November 2010 15:36 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3118
Registered: July 2009
Senior Member
That was to fast. The expression should read:

Addition returns Expression :
Multiplication ({Operation.left=current} op=('+'|'-')
right=Multiplication/* <- !!*/) * /* <- !! */
;

Regards,
Sebastian
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 29.11.10 16:35, schrieb Sebastian Zarnekow:
> Hi Cédric,
>
> your expressions are right associative, so the serializer is completely
> right:
>
> Addition returns Expression :
> Multiplication ({Operation.left=current} op=('+'|'-') right=Addition)?
> ;
>
> should actually read
>
> Addition returns Expression :
> Multiplication ({Operation.left=current} op=('+'|'-')
> right=Multiplication/* <- !!*/)?
> ;
>
> Regards,
> Sebastian
Re: [Xtext] Serializing an expression DSL with parentheses and different operator priorities [message #642081 is a reply to message #642030] Mon, 29 November 2010 18:08 Go to previous messageGo to next message
Cédric Vidal is currently offline Cédric VidalFriend
Messages: 101
Registered: July 2009
Senior Member
Hi Sebastian, thanx for the clarification, indeed, my grammar was
erroneous in respect to associativity. But sadly, the problem is
elsewhere. I've investigated and narrowed the issue.

The following grammar serializes correctly:

Expression : Addition;

Addition returns Expression :
Multiplication ({Operation.left=current} op=OPERATOR_ADD
right=Multiplication) *
;

Multiplication returns Expression :
Literal ({Operation.left=current} op=OPERATOR_MUL right=Literal) *
;

Literal returns Expression:
'(' Expression ')'
| {IntLiteral} value=INT
;

terminal OPERATOR_MUL : '*'|'/';
terminal OPERATOR_ADD : '+'|'-';

When using rules instead of terminals, serialization fails to include
parentheses:

Expression : Addition;

Addition returns Expression :
Multiplication ({Operation.left=current} op=OPERATOR_ADD
right=Multiplication) *
;

Multiplication returns Expression :
Literal ({Operation.left=current} op=OPERATOR_MUL right=Literal) *
;

Literal returns Expression:
'(' Expression ')'
| {IntLiteral} value=INT
;

OPERATOR_MUL : '*'|'/'; // <-- using rules instead of terminals
OPERATOR_ADD : '+'|'-';

--> parentheses are not serialized

When operators are specific meta classes which are referenced, it also
fails. The following DSL illustrates that. It adds the possibility to
define operators and reference them.

Evaluation:
(ops+=Operator ';') *
'exp' exp=Expression
;

Expression : Addition;

Addition returns Expression :
Multiplication ({Operation.left=current} op=[Operator|OPERATOR_ADD]
right=Multiplication) *
;

Multiplication returns Expression :
Literal ({Operation.left=current} op=[Operator|OPERATOR_MUL]
right=Literal) *
;

Literal returns Expression:
'(' Expression ')'
| {IntLiteral} value=INT
;

terminal OPERATOR_MUL : '*'|'/';
terminal OPERATOR_ADD : '+'|'-';

Operator :
'op' name=(OPERATOR_ADD|OPERATOR_MUL)
;

--> fails with "org.eclipse.xtext.conversion.ValueConverterException:
The value '*' is an invalid OPERATOR_ADD"

Regards,

Cédric

Le 29/11/2010 16:36, Sebastian Zarnekow a écrit :
> That was to fast. The expression should read:
>
> Addition returns Expression :
> Multiplication ({Operation.left=current} op=('+'|'-')
> right=Multiplication/* <- !!*/) * /* <- !! */
> ;
>
> Regards,
> Sebastian
Re: [Xtext] Serializing an expression DSL with parentheses and different operator priorities [message #642082 is a reply to message #642081] Mon, 29 November 2010 18:10 Go to previous messageGo to next message
Cédric Vidal is currently offline Cédric VidalFriend
Messages: 101
Registered: July 2009
Senior Member
Note that I'm using Xtext 1.0.0, it might work in Xtext 1.0.1

Le 29/11/2010 19:08, Cédric Vidal a écrit :
> Hi Sebastian, thanx for the clarification, indeed, my grammar was
> erroneous in respect to associativity. But sadly, the problem is
> elsewhere. I've investigated and narrowed the issue.
>
> The following grammar serializes correctly:
>
> Expression : Addition;
>
> Addition returns Expression :
> Multiplication ({Operation.left=current} op=OPERATOR_ADD
> right=Multiplication) *
> ;
>
> Multiplication returns Expression :
> Literal ({Operation.left=current} op=OPERATOR_MUL right=Literal) *
> ;
>
> Literal returns Expression:
> '(' Expression ')'
> | {IntLiteral} value=INT
> ;
>
> terminal OPERATOR_MUL : '*'|'/';
> terminal OPERATOR_ADD : '+'|'-';
>
> When using rules instead of terminals, serialization fails to include
> parentheses:
>
> Expression : Addition;
>
> Addition returns Expression :
> Multiplication ({Operation.left=current} op=OPERATOR_ADD
> right=Multiplication) *
> ;
>
> Multiplication returns Expression :
> Literal ({Operation.left=current} op=OPERATOR_MUL right=Literal) *
> ;
>
> Literal returns Expression:
> '(' Expression ')'
> | {IntLiteral} value=INT
> ;
>
> OPERATOR_MUL : '*'|'/'; // <-- using rules instead of terminals
> OPERATOR_ADD : '+'|'-';
>
> --> parentheses are not serialized
>
> When operators are specific meta classes which are referenced, it also
> fails. The following DSL illustrates that. It adds the possibility to
> define operators and reference them.
>
> Evaluation:
> (ops+=Operator ';') *
> 'exp' exp=Expression
> ;
>
> Expression : Addition;
>
> Addition returns Expression :
> Multiplication ({Operation.left=current} op=[Operator|OPERATOR_ADD]
> right=Multiplication) *
> ;
>
> Multiplication returns Expression :
> Literal ({Operation.left=current} op=[Operator|OPERATOR_MUL]
> right=Literal) *
> ;
>
> Literal returns Expression:
> '(' Expression ')'
> | {IntLiteral} value=INT
> ;
>
> terminal OPERATOR_MUL : '*'|'/';
> terminal OPERATOR_ADD : '+'|'-';
>
> Operator :
> 'op' name=(OPERATOR_ADD|OPERATOR_MUL)
> ;
>
> --> fails with "org.eclipse.xtext.conversion.ValueConverterException:
> The value '*' is an invalid OPERATOR_ADD"
>
> Regards,
>
> Cédric
>
> Le 29/11/2010 16:36, Sebastian Zarnekow a écrit :
>> That was to fast. The expression should read:
>>
>> Addition returns Expression :
>> Multiplication ({Operation.left=current} op=('+'|'-')
>> right=Multiplication/* <- !!*/) * /* <- !! */
>> ;
>>
>> Regards,
>> Sebastian
>
Re: [Xtext] Serializing an expression DSL with parentheses and different operator priorities [message #642084 is a reply to message #642081] Mon, 29 November 2010 18:23 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3118
Registered: July 2009
Senior Member
Hi Cedric,

would you please be so kind and file your reproducable sample as a
bugzilla? Thanks.

Regards,
Sebastian
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 29.11.10 19:08, schrieb Cédric Vidal:
> Hi Sebastian, thanx for the clarification, indeed, my grammar was
> erroneous in respect to associativity. But sadly, the problem is
> elsewhere. I've investigated and narrowed the issue.
>
> The following grammar serializes correctly:
>
> Expression : Addition;
>
> Addition returns Expression :
> Multiplication ({Operation.left=current} op=OPERATOR_ADD
> right=Multiplication) *
> ;
>
> Multiplication returns Expression :
> Literal ({Operation.left=current} op=OPERATOR_MUL right=Literal) *
> ;
>
> Literal returns Expression:
> '(' Expression ')'
> | {IntLiteral} value=INT
> ;
>
> terminal OPERATOR_MUL : '*'|'/';
> terminal OPERATOR_ADD : '+'|'-';
>
> When using rules instead of terminals, serialization fails to include
> parentheses:
>
> Expression : Addition;
>
> Addition returns Expression :
> Multiplication ({Operation.left=current} op=OPERATOR_ADD
> right=Multiplication) *
> ;
>
> Multiplication returns Expression :
> Literal ({Operation.left=current} op=OPERATOR_MUL right=Literal) *
> ;
>
> Literal returns Expression:
> '(' Expression ')'
> | {IntLiteral} value=INT
> ;
>
> OPERATOR_MUL : '*'|'/'; // <-- using rules instead of terminals
> OPERATOR_ADD : '+'|'-';
>
> --> parentheses are not serialized
>
> When operators are specific meta classes which are referenced, it also
> fails. The following DSL illustrates that. It adds the possibility to
> define operators and reference them.
>
> Evaluation:
> (ops+=Operator ';') *
> 'exp' exp=Expression
> ;
>
> Expression : Addition;
>
> Addition returns Expression :
> Multiplication ({Operation.left=current} op=[Operator|OPERATOR_ADD]
> right=Multiplication) *
> ;
>
> Multiplication returns Expression :
> Literal ({Operation.left=current} op=[Operator|OPERATOR_MUL]
> right=Literal) *
> ;
>
> Literal returns Expression:
> '(' Expression ')'
> | {IntLiteral} value=INT
> ;
>
> terminal OPERATOR_MUL : '*'|'/';
> terminal OPERATOR_ADD : '+'|'-';
>
> Operator :
> 'op' name=(OPERATOR_ADD|OPERATOR_MUL)
> ;
>
> --> fails with "org.eclipse.xtext.conversion.ValueConverterException:
> The value '*' is an invalid OPERATOR_ADD"
>
> Regards,
>
> Cédric
>
> Le 29/11/2010 16:36, Sebastian Zarnekow a écrit :
>> That was to fast. The expression should read:
>>
>> Addition returns Expression :
>> Multiplication ({Operation.left=current} op=('+'|'-')
>> right=Multiplication/* <- !!*/) * /* <- !! */
>> ;
>>
>> Regards,
>> Sebastian
>
Re: [Xtext] Serializing an expression DSL with parentheses and different operator priorities [message #642107 is a reply to message #642084] Mon, 29 November 2010 20:53 Go to previous message
Cédric Vidal is currently offline Cédric VidalFriend
Messages: 101
Registered: July 2009
Senior Member
Sure, here you go:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=331378


Le 29/11/2010 19:23, Sebastian Zarnekow a écrit :
> Hi Cedric,
>
> would you please be so kind and file your reproducable sample as a
> bugzilla? Thanks.
>
> Regards,
> Sebastian
Previous Topic:ImportedNamespaceAwareLocalScopeProvider
Next Topic:How to run MWE2 code generator programmatically in the eclipse IDE plugin
Goto Forum:
  


Current Time: Tue Apr 16 19:24:18 GMT 2024

Powered by FUDForum. Page generated in 0.09421 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top