Eclipse Community Forums: TMF (Xtext) » extending Xbase expression syntax and possible ambiguities

Home » Modeling » TMF (Xtext) » extending Xbase expression syntax and possible ambiguities

extending Xbase expression syntax and possible ambiguities [message #759137]

Sat, 26 November 2011 12:36

Eclipse User

Hi

In my DSL I've extended the XEpression syntax with statements such as, say,

environment=[Environment] left=(XExpression) relation=RelationSymbol
right=(XExpression)

where RelationSymbol is a datatype rule as follows:

RelationSymbol:
'::' | ':' | '<:' | ':>' | '<<' | '>>' |
'<|' | '|>' | '<-' | '->'
;

however, this will conflict Xbase syntax for instance for static access;
in particular, if I have something like

var MyClass s
G |- s :: s

there's a parse error (which I could avoid by writing)

var MyClass s
G |- (s) :: s

but simply cannot parse

var MyClass s
G |- s <- s

or

var MyClass s
G |- s -> s

any suggestion, please?

thanks in advance
Lorenzo

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
ICQ# lbetto, 16080134 (GNU/Linux User # 158233)
HOME: http://www.lorenzobettini.it MUSIC: http://www.purplesucker.com
http://www.myspace.com/supertrouperabba
BLOGS: http://tronprog.blogspot.com http://longlivemusic.blogspot.com
http://www.gnu.org/software/src-highlite
http://www.gnu.org/software/gengetopt
http://www.gnu.org/software/gengen http://doublecpp.sourceforge.net

Re: extending Xbase expression syntax and possible ambiguities [message #759192 is a reply to message #759137]

Sun, 27 November 2011 09:02

Eclipse User

Could you give the entire grammar? Right now I can't see where the '|-' is coming from. And it depends on where you "plugged" the XExpression syntax whether this result in an unambiguous grammar: do you get any ANTLR warnings/errors?

Re: extending Xbase expression syntax and possible ambiguities [message #759202 is a reply to message #759192]

Sun, 27 November 2011 10:45

Eclipse User

On 11/27/2011 03:02 PM, Meinte Boersma wrote:
> Could you give the entire grammar? Right now I can't see where the '|-'
> is coming from. And it depends on where you "plugged" the XExpression
> syntax whether this result in an unambiguous grammar: do you get any
> ANTLR warnings/errors?

Sure,

I get no warning/error from Antlr;

let me first highlight the important parts (then there will be the
complete grammar)

XExpressionInsideBlock returns xbase::XExpression:
RuleInvocation |
XVariableDeclaration | XExpression;

RuleInvocation:
environment=[Environment] jugmentSymbol=JudgmentSymbol
left=XExpression relationSymbol=RelationSymbol right=XExpression
;

Environment: name=ID;

JudgmentSymbol:
('||'|'|')('-'|'~'|'='|'>')+
;

RelationSymbol:
'::' | ':' | '<:' | ':>' | '<<' | '>>' |
'<|' | '|>' | '<-' | '->'
;

RuleConclusionElement:
RuleParameter | ExpressionInConclusion
;

RuleParameter:
parameter=FullJvmFormalParameter
;

ExpressionInConclusion:
expression=XExpression
;

this is a snippet of a test case

rule EClassEObject derives
G |- EClass eClass : EObject object
from {
// some expressions from Xbase
'foo' == new String() + 'bar'.toFirstUpper
G |- object.eClass :: eClass
G |- (eClass) :: eClass // we need () otherwise interpreted as static
access
G |- (eClass) :: object.eClass.name == [ s | s.toFirstLower ].apply('foo')
val EClass eC = EcoreFactory::eINSTANCE.createEClass()
eC.name = 'MyEClass'
eClass == eC
G |- object.eClass <: eClass
G |- object.eClass :> eClass
G |- object.eClass << eClass
G |- object.eClass >> eClass
G |- object.eClass <| eClass
G |- object.eClass |> eClass
//G |- (object.eClass) -> (eClass)
G |- (object.eClass) <- (eClass)
G ||- object.eClass >> eClass
}

for the moment, the only part that cannot be parsed is the commented
one, using ->

here's the complete grammar (I have backtracking enabled)

grammar it.xtypes.dsl.Xtypes with org.eclipse.xtext.xbase.Xbase

generate xtypes "http://xtypes.sf.net/Xtypes"

import "http://www.eclipse.org/xtext/xbase/Xbase" as xbase

XtypesTypeSystem:
'typesystem' name=QualifiedName
imports += Import*
('judgments' '{'
judgmentDescriptions += JudgmentDescription+
'}')?
rules+=Rule*
;

Import:
'import' importedNamespace = QualifiedNameWithWildcard
;

JudgmentDescription:
name=ID
judgmentsymbol = STRING
left=JudgmentType
relation = STRING
right=JudgmentType
;

JudgmentType:
jvmTypeReference=JvmTypeReference
;

QualifiedNameWithWildcard:
QualifiedName '.*'?
;

Rule:
Axiom | RuleWithPremises;

Axiom:
'axiom' name=ID conclusion=RuleConclusion;

RuleWithPremises:
'rule' name=ID
'derives'
conclusion=RuleConclusion
'from' premises=XBlockExpression
;

RuleConclusion:
environment=Environment
judgmentSymbol=JudgmentSymbol
left=RuleConclusionElement
relationSymbol=RelationSymbol
right=RuleConclusionElement
;

XExpressionInsideBlock returns xbase::XExpression:
RuleInvocation |
XVariableDeclaration | XExpression;

RuleInvocation:
environment=[Environment] jugmentSymbol=JudgmentSymbol
left=XExpression relationSymbol=RelationSymbol right=XExpression
;

Environment: name=ID;

JudgmentSymbol:
('||'|'|')('-'|'~'|'='|'>')+
;

RelationSymbol:
'::' | ':' | '<:' | ':>' | '<<' | '>>' |
'<|' | '|>' | '<-' | '->'
;

RuleConclusionElement:
RuleParameter | ExpressionInConclusion
;

RuleParameter:
parameter=FullJvmFormalParameter
;

ExpressionInConclusion:
expression=XExpression
;

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
ICQ# lbetto, 16080134 (GNU/Linux User # 158233)
HOME: http://www.lorenzobettini.it MUSIC: http://www.purplesucker.com
http://www.myspace.com/supertrouperabba
BLOGS: http://tronprog.blogspot.com http://longlivemusic.blogspot.com
http://www.gnu.org/software/src-highlite
http://www.gnu.org/software/gengetopt
http://www.gnu.org/software/gengen http://doublecpp.sourceforge.net

Re: extending Xbase expression syntax and possible ambiguities [message #759203 is a reply to message #759202]

Sun, 27 November 2011 10:55

Eclipse User

Since it's Sunday afternoon and I'm too lazy to plug your grammar into an Eclipse, I'll just trigger on "backtracking is switched on": is there a specific reason to do that? Backtracking disables any reporting of ANTLR warnings/errors, but it doesn't mean your grammar is unambiguous - far from it, probably. You might want to switch if off again and run the generated InternalXtypesParser.g file through ANTLR Works to try and iron out the ambiguities without having to resort to backtracking.

Re: extending Xbase expression syntax and possible ambiguities [message #759383 is a reply to message #759203]

Mon, 28 November 2011 09:07

Eclipse User

On 11/27/2011 04:55 PM, Meinte Boersma wrote:
> Since it's Sunday afternoon and I'm too lazy to plug your grammar into

no problem, you had asked for the grammar ;)

> an Eclipse, I'll just trigger on "backtracking is switched on": is there
> a specific reason to do that? Backtracking disables any reporting of
> ANTLR warnings/errors, but it doesn't mean your grammar is unambiguous -
> far from it, probably. You might want to switch if off again and run the
> generated InternalXtypesParser.g file through ANTLR Works to try and
> iron out the ambiguities without having to resort to backtracking.

backtracking is on due to

RuleConclusionElement:
RuleParameter | ExpressionInConclusion
;

RuleParameter:
parameter=FullJvmFormalParameter
;

ExpressionInConclusion:
expression=XExpression
;

to deal with (FullJvmFormalParameter | XExpression)

probably I'll use a keyword to distinguish between parameter and
expression...

cheers
Lorenzo

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
ICQ# lbetto, 16080134 (GNU/Linux User # 158233)
HOME: http://www.lorenzobettini.it MUSIC: http://www.purplesucker.com
http://www.myspace.com/supertrouperabba
BLOGS: http://tronprog.blogspot.com http://longlivemusic.blogspot.com
http://www.gnu.org/software/src-highlite
http://www.gnu.org/software/gengetopt
http://www.gnu.org/software/gengen http://doublecpp.sourceforge.net

Re: extending Xbase expression syntax and possible ambiguities [message #759391 is a reply to message #759383]

Mon, 28 November 2011 09:39

Eclipse User

OK, I should have said I was actually too lazy to even look at the grammar in detail Wink

Using a keyword to get rid of the ambiguity is one solution, but others exist: you could make FullJvmFormalParameter a subtype of XExpression in the sense that it's a valid XExpression in its own right. The distinction between RuleParameter and ExpressionInConclusion would then be a derived one, depending on the precise type and form/structure of the XExpression.

Re: extending Xbase expression syntax and possible ambiguities [message #759589 is a reply to message #759391]

Tue, 29 November 2011 04:50

Eclipse User

On 11/28/2011 03:39 PM, Meinte Boersma wrote:
> OK, I should have said I was actually too lazy to even look at the
> grammar in detail ;)
>

no problem :)

> Using a keyword to get rid of the ambiguity is one solution, but others
> exist: you could make FullJvmFormalParameter a subtype of XExpression in
> the sense that it's a valid XExpression in its own right. The
> distinction between RuleParameter and ExpressionInConclusion would then
> be a derived one, depending on the precise type and form/structure of
> the XExpression.

FullJvmFormalParameter does not seem to be a subtype of XExpression:

FullJvmFormalParameter returns types::JvmFormalParameter:
parameterType=JvmTypeReference name=ValidID;

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
ICQ# lbetto, 16080134 (GNU/Linux User # 158233)
HOME: http://www.lorenzobettini.it MUSIC: http://www.purplesucker.com
http://www.myspace.com/supertrouperabba
BLOGS: http://tronprog.blogspot.com http://longlivemusic.blogspot.com
http://www.gnu.org/software/src-highlite
http://www.gnu.org/software/gengetopt
http://www.gnu.org/software/gengen http://doublecpp.sourceforge.net

Re: extending Xbase expression syntax and possible ambiguities [message #759601 is a reply to message #759589]

Tue, 29 November 2011 05:24

Eclipse User

Your grammar has left-recursion at the following rules (or so Xtext/ANTLR tells me): XExpressionInsideBlock, JudgmentSymbol, RuleConclusionElement. The thing is that something like calling the xbase::XExpression rule is really a bit like Pandora's box: once it's open, it's hard to close. You get a lot of stuff for free, but you also get a lot of parser rules (which are tried) for free and you have to deal with the ensuing left-recursion.

I'd suggest to left-factor the grammar: it's an agitation in the buttocks alright, but fixing all kinds of grammar problems you might have after that is so much easier and you really can't rely all that much on backtracking without a very good test suite checking whether the parser does what it's supposed to do. Left-factoring means that you have to find out how a certain parser rule might call itself, probably with some other calls inbetween, and re-writing that rule in such a way that it will always consume a token before trying to match something else (and performing a tree rewrite if it does match something else).

Re: extending Xbase expression syntax and possible ambiguities [message #759937 is a reply to message #759601]

Wed, 30 November 2011 10:46

Eclipse User

On 11/29/2011 11:24 AM, Meinte Boersma wrote:
> Your grammar has left-recursion at the following rules (or so
> Xtext/ANTLR tells me): XExpressionInsideBlock, JudgmentSymbol,
> RuleConclusionElement. The thing is that something like calling the
> xbase::XExpression rule is really a bit like Pandora's box: once it's
> open, it's hard to close. You get a lot of stuff for free, but you also
> get a lot of parser rules (which are tried) for free and you have to
> deal with the ensuing left-recursion.
>
> I'd suggest to left-factor the grammar: it's an agitation in the
> buttocks alright, but fixing all kinds of grammar problems you might
> have after that is so much easier and you really can't rely all that
> much on backtracking without a very good test suite checking whether the
> parser does what it's supposed to do. Left-factoring means that you have
> to find out how a certain parser rule might call itself, probably with
> some other calls inbetween, and re-writing that rule in such a way that
> it will always consume a token before trying to match something else
> (and performing a tree rewrite if it does match something else).

Hi

I then refactored my grammar using predicates (at the end the most
important part), after removing backtracking.

I had to give using '::' as a relation symbol since it would be between
two XExpressions, and as you said "calling the
> xbase::XExpression rule is really a bit like Pandora's box" ;)

I had also to give up on '->' but for the moment I'll live with that too :)

thanks
Lore

XExpressionInsideBlock returns xbase::XExpression:
RuleInvocation |
XVariableDeclaration | XExpression;

RuleInvocation:
=>environment=[Environment] jugmentSymbol=JudgmentSymbol
left=XExpression
relationSymbol=RelationSymbol
right=XExpression
;

Environment: name=ID;

JudgmentSymbol:
'||-' | '|-' | '||~' | '|~' | '||=' | '|=' | '||>' | '|>'
;

RelationSymbol:
':' | '<:' | ':>' | '<<' | '>>' |
'<|' | '|>' | '<-' | '-->' | '<~' | '~>';

RuleConclusionElement:
=>(RuleParameter) | ExpressionInConclusion
;

RuleParameter:
parameter=FullJvmFormalParameter
;

ExpressionInConclusion:
expression=XExpression

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
ICQ# lbetto, 16080134 (GNU/Linux User # 158233)
HOME: http://www.lorenzobettini.it MUSIC: http://www.purplesucker.com
http://www.myspace.com/supertrouperabba
BLOGS: http://tronprog.blogspot.com http://longlivemusic.blogspot.com
http://www.gnu.org/software/src-highlite
http://www.gnu.org/software/gengetopt
http://www.gnu.org/software/gengen http://doublecpp.sourceforge.net

Previous Topic:	How to write Content Assist
Next Topic:	multiple generators - Xtext2, xtend2

Goto Forum:

-=] Back to Top [=-

Current Time: Wed Jul 23 07:41:43 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter