[Xtext] Grammar - Enhancing the infered model for language expressions [message #660925] |
Tue, 22 March 2011 11:38  |
Eclipse User |
|
|
|
Hi,
I create this new thread to resolve another problem I have on my
grammar. I have expressions in my language. The EBNF-like Xtext format
forces me to define my expression syntax in a non-left-recursive way.
Also I have operator precedence and this forces me to have different
rules for the different operators.
However from a generation point of view I would be glad to common
supertypes for UnaryOperationExpressions and BinaryExpressions.
Here is the ideal type hierarchy I would like:
BinaryExpression (left:Expression, right:Expression)
- ConditionalExpression (operator: enum ConditionalOperator)
- EqualityExpression (operator: enum EqualityOperator)
- RelationalExpression (operator: enum RelationalOperator)
- ArithmeticExpression (operator: enum ConditionalOperator)
Here is my actual grammar fragment for expressions:
<code>
Expression returns Expression:
{If} 'if' condition=ConditionalOrExpression 'then' then=Expression
'else' else=Expression |
{Let} 'let' definitions+=VariableDefinition ("," definitions
+=VariableDefinition)* 'in' result=Expression |
{Map} 'map' value=AccessExpression ('->' to=[ecore::EClassifier|
QualifiedName])? |
ConditionalOrExpression;
VariableDefinition returns VariableDefinition:
name=ID ':=' value=Expression;
ConditionalOrExpression returns Expression:
ConditionalAndExpression ({ConditionalExpression.left=current}
operator='or' right=ConditionalAndExpression)*;
ConditionalAndExpression returns Expression:
EqualityExpression ({ConditionalExpression.left=current} operator='and'
right=EqualityExpression)*;
enum ConditionalOperator:
Or='or' | And='and';
EqualityExpression returns Expression:
RelationalExpression ({EqualityExpression.left=current}
operator=EqualityOperator right=RelationalExpression)?;
enum EqualityOperator:
Equal='=' | Different='!=';
RelationalExpression returns Expression:
AdditiveExpression ({RelationalExpression.left=current}
operator=RelationalOperator right=AdditiveExpression)?;
enum RelationalOperator:
Less='<' | Greater='>' | LessOrEqual='<=' | GreaterOrEqual='=>';
AdditiveExpression returns Expression:
MultiplicativeExpression ({ArithmeticExpression.left=current}
operator=('+' | '-') right=MultiplicativeExpression)*;
MultiplicativeExpression returns Expression:
UnaryExpression ({ArithmeticExpression.left=current} operator=('*' |
'/') right=UnaryExpression)*;
enum ArithmeticOperator:
Add='+' | Substract='-' | Multiply='*' | Divide='/';
UnaryExpression returns Expression:
{UnaryExpression} operator=UnaryOperator operand=AccessExpression |
AccessExpression;
enum UnaryOperator:
Negation='!' | Minus='-';
AccessExpression returns Expression:
PrimaryExpression ({FeatureAccess.object=current} '.'
feature=[ecore::EStructuralFeature]
({Invocation.access=current} '(' (parameters+=Expression (","
parameters+=Expression)*)? ')')?)*;
PrimaryExpression returns Expression:
'(' Expression ')' | ObjectExpression | Literal |
{FunctionCall} name=ID '(' (parameters+=Expression ("," parameters
+=Expression)*)? ')';
ObjectExpression returns Expression:
{Source} 'source' |
{Extent} 'extent' |
{VariableUse} variable=[VariableDefinition|ID];
Literal returns Expression:
{BooleanLiteral} value=BooleanLiteralValue |
{IntegerLiteral} value=INT |
{FloatLiteral} value=FLOAT |
{StringLiteral} value=STRING;
enum BooleanLiteralValue:
False='false' | True='true';
FLOAT hidden():
INT '.' INT;
</code>
Here are my problems:
1) I can't use the ConditionalOperator enum as my ConditionalExpressions
must be expressed as two separate rules to ensure operator precedence.
I guess Xtext misses something like access a particular enum value.
Something like MyEnum.literalName. That would enable something like:
ConditionalOrExpression returns Expression:
ConditionalAndExpression ({ConditionalExpression.left=current}
operator=ConditionalOperator.Or right=ConditionalAndExpression)*;
Is there an other way to do the same thing ?
2) I can't have a common ConditionalExpression supertype. I tried to add
an 'artificial' rule (as said in the Xtext user guide) but this creates
a warning in ANTLR as this obviously creates a left-recusion even if the
rule is never called from anywhere.
ConditionalExpression:
ConditionalOrExpression | ConditionalAndExpression;
3) I can't pull up left and right feature although there are everywhere
of type Expression. I also tried to add an 'artificial' rule but this
doesn't pull up the left and right features as expected.
BinaryExpression:
ConditionalOrExpression | ConditionalAndExpression | EqualityExpression
| RelationalExpression | AdditiveExpression | MultiplicativeExpression;
The user guide says that it should pull up the common features (end of
section 3.3.2) or am I misunderstanding something ??
4) I will need to enhance my type checking (I use xtext-typesystem which
by the way would be good to integrate in Xtext) I will need to further
differentiate Multiplication from Division (as a Division needs to be of
the type of the left operand). But from a generation point of view I
really need to have my model to have common supertypes.
Infered model enhancement through Xtend looks attractive but I have no
hints in how modifying the type hierarchy (feature pull-ups and feature
types string->enum modification) would break what Xtext has generated.
Thanks for your help. Didier.
|
|
|
Re: [Xtext] Grammar - Enhancing the infered model for language expressions [message #660954 is a reply to message #660925] |
Tue, 22 March 2011 13:45   |
Eclipse User |
|
|
|
At some point you most likely want to shift to an external model instead
of generating it from the grammar. Using an imported model gives you
full control.
- henrik
On 3/22/11 12:38 PM, Didier Villevalois wrote:
> Hi,
>
> I create this new thread to resolve another problem I have on my
> grammar. I have expressions in my language. The EBNF-like Xtext format
> forces me to define my expression syntax in a non-left-recursive way.
> Also I have operator precedence and this forces me to have different
> rules for the different operators.
>
> However from a generation point of view I would be glad to common
> supertypes for UnaryOperationExpressions and BinaryExpressions.
>
> Here is the ideal type hierarchy I would like:
>
> BinaryExpression (left:Expression, right:Expression)
> - ConditionalExpression (operator: enum ConditionalOperator)
> - EqualityExpression (operator: enum EqualityOperator)
> - RelationalExpression (operator: enum RelationalOperator)
> - ArithmeticExpression (operator: enum ConditionalOperator)
>
> Here is my actual grammar fragment for expressions:
>
> <code>
> Expression returns Expression:
> {If} 'if' condition=ConditionalOrExpression 'then' then=Expression
> 'else' else=Expression |
> {Let} 'let' definitions+=VariableDefinition ("," definitions
> +=VariableDefinition)* 'in' result=Expression |
> {Map} 'map' value=AccessExpression ('->' to=[ecore::EClassifier|
> QualifiedName])? |
> ConditionalOrExpression;
>
> VariableDefinition returns VariableDefinition:
> name=ID ':=' value=Expression;
>
> ConditionalOrExpression returns Expression:
> ConditionalAndExpression ({ConditionalExpression.left=current}
> operator='or' right=ConditionalAndExpression)*;
>
> ConditionalAndExpression returns Expression:
> EqualityExpression ({ConditionalExpression.left=current} operator='and'
> right=EqualityExpression)*;
>
> enum ConditionalOperator:
> Or='or' | And='and';
>
> EqualityExpression returns Expression:
> RelationalExpression ({EqualityExpression.left=current}
> operator=EqualityOperator right=RelationalExpression)?;
>
> enum EqualityOperator:
> Equal='=' | Different='!=';
>
> RelationalExpression returns Expression:
> AdditiveExpression ({RelationalExpression.left=current}
> operator=RelationalOperator right=AdditiveExpression)?;
>
> enum RelationalOperator:
> Less='<' | Greater='>' | LessOrEqual='<=' | GreaterOrEqual='=>';
>
> AdditiveExpression returns Expression:
> MultiplicativeExpression ({ArithmeticExpression.left=current}
> operator=('+' | '-') right=MultiplicativeExpression)*;
>
> MultiplicativeExpression returns Expression:
> UnaryExpression ({ArithmeticExpression.left=current} operator=('*' |
> '/') right=UnaryExpression)*;
>
> enum ArithmeticOperator:
> Add='+' | Substract='-' | Multiply='*' | Divide='/';
>
> UnaryExpression returns Expression:
> {UnaryExpression} operator=UnaryOperator operand=AccessExpression |
> AccessExpression;
>
> enum UnaryOperator:
> Negation='!' | Minus='-';
>
> AccessExpression returns Expression:
> PrimaryExpression ({FeatureAccess.object=current} '.'
> feature=[ecore::EStructuralFeature]
> ({Invocation.access=current} '(' (parameters+=Expression (","
> parameters+=Expression)*)? ')')?)*;
>
> PrimaryExpression returns Expression:
> '(' Expression ')' | ObjectExpression | Literal |
> {FunctionCall} name=ID '(' (parameters+=Expression ("," parameters
> +=Expression)*)? ')';
>
> ObjectExpression returns Expression:
> {Source} 'source' |
> {Extent} 'extent' |
> {VariableUse} variable=[VariableDefinition|ID];
>
> Literal returns Expression:
> {BooleanLiteral} value=BooleanLiteralValue |
> {IntegerLiteral} value=INT |
> {FloatLiteral} value=FLOAT |
> {StringLiteral} value=STRING;
>
> enum BooleanLiteralValue:
> False='false' | True='true';
>
> FLOAT hidden():
> INT '.' INT;
> </code>
>
>
> Here are my problems:
>
> 1) I can't use the ConditionalOperator enum as my ConditionalExpressions
> must be expressed as two separate rules to ensure operator precedence.
>
> I guess Xtext misses something like access a particular enum value.
> Something like MyEnum.literalName. That would enable something like:
>
> ConditionalOrExpression returns Expression:
> ConditionalAndExpression ({ConditionalExpression.left=current}
> operator=ConditionalOperator.Or right=ConditionalAndExpression)*;
>
> Is there an other way to do the same thing ?
>
> 2) I can't have a common ConditionalExpression supertype. I tried to add
> an 'artificial' rule (as said in the Xtext user guide) but this creates
> a warning in ANTLR as this obviously creates a left-recusion even if the
> rule is never called from anywhere.
>
> ConditionalExpression:
> ConditionalOrExpression | ConditionalAndExpression;
>
> 3) I can't pull up left and right feature although there are everywhere
> of type Expression. I also tried to add an 'artificial' rule but this
> doesn't pull up the left and right features as expected.
>
> BinaryExpression:
> ConditionalOrExpression | ConditionalAndExpression | EqualityExpression
> | RelationalExpression | AdditiveExpression | MultiplicativeExpression;
>
> The user guide says that it should pull up the common features (end of
> section 3.3.2) or am I misunderstanding something ??
>
> 4) I will need to enhance my type checking (I use xtext-typesystem which
> by the way would be good to integrate in Xtext) I will need to further
> differentiate Multiplication from Division (as a Division needs to be of
> the type of the left operand). But from a generation point of view I
> really need to have my model to have common supertypes.
>
>
> Infered model enhancement through Xtend looks attractive but I have no
> hints in how modifying the type hierarchy (feature pull-ups and feature
> types string->enum modification) would break what Xtext has generated.
>
> Thanks for your help. Didier.
>
|
|
|
Re: [Xtext] Grammar - Enhancing the infered model for language expressions [message #661015 is a reply to message #660954] |
Tue, 22 March 2011 17:44   |
Eclipse User |
|
|
|
Hi Henrik,
Henrik Lindberg wrote on Tue, 22 March 2011 09:45 | At some point you most likely want to shift to an external model instead
of generating it from the grammar. Using an imported model gives you
full control.
|
OK I understand that maintaining my own model would enable me to have the needed type hierarchy and have features at the needed level in this hierarchy.
However, I think I still can't use values from the same enum in different parser rules. Am I wrong ?
> 1) I can't use the ConditionalOperator enum as my ConditionalExpressions
> must be expressed as two separate rules to ensure operator precedence.
>
> I guess Xtext misses something like access a particular enum value.
> Something like MyEnum.literalName. That would enable something like:
>
> ConditionalOrExpression returns Expression:
> ConditionalAndExpression ({ConditionalExpression.left=current}
> operator=ConditionalOperator.Or right=ConditionalAndExpression)*;
>
> Is there an other way to do the same thing ?
|
|
|
Re: [Xtext] Grammar - Enhancing the infered model for language expressions [message #661023 is a reply to message #661015] |
Tue, 22 March 2011 18:14   |
Eclipse User |
|
|
|
On 3/22/11 6:44 PM, Didier Villevalois wrote:
> Henrik Lindberg wrote on Tue, 22 March 2011 09:45
>> At some point you most likely want to shift to an external model
>> instead of generating it from the grammar. Using an imported model
>> gives you full control.
>
>
> OK I understand that maintaining my own model would enable me to have
> the needed type hierarchy and have features at the needed level in this
> hierarchy.
>
> However, I think I still can't use values from the same enum in
> different parser rules. Am I wrong ?
>
Yes, but you need two different rules - both returning the same Enum
type. Something like:
Add returns MyModel::Operator : op = '+' ;
Mul returns MyModel::Operator : op = '*' ;
or somesuch... But I don't know how well that works with the Xtext
grammar, if it needs to understand that it is an Enum or not. I ran into
some issues when using Enums in Eclipse b3, and ended up writing some
helpers (enum data type). For operators I used strings (and validation).
I know that others use references/links for the same thing.
- henrik
|
|
|
Re: [Xtext] Grammar - Enhancing the infered model for language expressions [message #661294 is a reply to message #661023] |
Thu, 24 March 2011 01:03   |
Eclipse User |
|
|
|
Hi,
use the following and a value converter instead:
Add returns MyModel::Operator : '+' ;
Mul returns MyModel::Operator : '*' ;
See the docs for details on the IValueConverter.
Regards,
Sebastian
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com
Am 22.03.11 19:14, schrieb Henrik Lindberg:
> On 3/22/11 6:44 PM, Didier Villevalois wrote:
>> Henrik Lindberg wrote on Tue, 22 March 2011 09:45
>>> At some point you most likely want to shift to an external model
>>> instead of generating it from the grammar. Using an imported model
>>> gives you full control.
>>
>>
>> OK I understand that maintaining my own model would enable me to have
>> the needed type hierarchy and have features at the needed level in this
>> hierarchy.
>>
>> However, I think I still can't use values from the same enum in
>> different parser rules. Am I wrong ?
>>
>
> Yes, but you need two different rules - both returning the same Enum
> type. Something like:
>
> Add returns MyModel::Operator : op = '+' ;
> Mul returns MyModel::Operator : op = '*' ;
>
> or somesuch... But I don't know how well that works with the Xtext
> grammar, if it needs to understand that it is an Enum or not. I ran into
> some issues when using Enums in Eclipse b3, and ended up writing some
> helpers (enum data type). For operators I used strings (and validation).
> I know that others use references/links for the same thing.
>
> - henrik
|
|
|
Re: [Xtext] Grammar - Enhancing the infered model for language expressions [message #1847761 is a reply to message #661294] |
Sat, 06 November 2021 05:05  |
Eclipse User |
|
|
|
Has anyone found a solution to OP's question #3 in the meantime? (i.e., pulling up common features in an expression grammar)
I struggle with a very similar issue right now. My grammar contains an expression grammar like this:
Expression: Partition;
Partition returns Expression:
Qualification ({Partition.dispatchObject=current} '??' => cases += ClosureWithParameters+)*;
Qualification returns Expression:
Disjunction ({Qualification.left=current} operator=QualificationOperator right=Qualification)?;
Disjunction returns Expression:
Conjunction ({Disjunction.left=current} operator=DisjunctionOperator right=Conjunction)*;
// ... about 10 to 15 additional productions with the same left/operator/right structure...
Exponentiation returns Expression:
Prefixation ({Exponentiation.left=current} operator=ExponentiationOperator right=Prefixation)*;
Prefixation returns Expression:
{Prefixation}
operator=(PrefixOperator|PlusAdditionOperator|MinusAdditionOperator|PlusMinusAdditionOperator|MinusPlusAdditionOperator)
operand=Prefixation | Invocation;
Invocation returns Expression:
Primary
(=>({Navigation.object=current} '.'? member=[MethodDeclaration])
|=>({Invocation.target=current} arguments=Arguments))*;
In other words, most, but not all, of the expression grammar productions have a left/operator/right structure. I tried to extract common features into its own type by adding a production like this:
BinaryOperation: Qualification | Disjunction | ... | Exponentation;
This produces the desired hierarchy structure but does not pull up the left/right or operator features (BinaryOperation has no features of its own at all).
As a result, I have a significant amount of code duplication in the scoping, validation, and code generation areas of my code. For example, even though the code generation is identical for all binary operators, I have 15 or 20 similar methods that extract left/operator/right and pass it to a common method. I'd really like to get rid of these redundant methods as well as similar sets of methods in several other areas.
Is there a way in Xtext today to extract common features of types that appear in an expression grammar?
|
|
|
Powered by
FUDForum. Page generated in 0.04994 seconds