Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Need help getting started on a grammar
Need help getting started on a grammar [message #1219861] Fri, 06 December 2013 10:20 Go to next message
Jordan Wallet is currently offline Jordan WalletFriend
Messages: 16
Registered: December 2013
Junior Member
I've been going at the documentation for a good 4 hours now and I'm just about at the end of my rope. I'm trying to write a simple grammar that resembles a programming language and I can't get all the warnings/errors To go away.

The problem is that I can get any specific one to go away, so I don't know what's wrong with my approach. I feel like I lack some fundamental concept in A few examples of the problems I've run into:

The documentation never explicitly states what a "feature" (assigned value) is. What are they?

The documentation does not properly explain the difference between terminal rules and data type rules. No, really! For example, search the page for "above mentioned reasons", and then try to find reasons above it; you won't find any.

I'm not sure why I would want to, because the tutorials were written from a Java perspective and glossed over data types entirely, but I cannot set return types on terminals:
terminal NUMBER returns ecore::EInt:
Quote:
- Couldn't resolve reference to AbstractMetamodelDeclaration 'ecore'
- Cannot create datatype EInt

This makes me suspect that my project setup is broken, but I just went with the default, so I don't know what gives.

And then this gives me a 'Cannot change type twice within a rule' error:
ArithmeticExpression:
	Variable'++' |
	Variable'--' |
	((Variable | ParensExpression | NUMBER) ARITHMETIC_OPERATOR ArithmeticExpression)
;
But I'm not explicitly declaring any types (the return type above was just illustrative, it's usually not there), so shouldn't everything just be EStrings or inferred?

If anyone can shed any light on any if these issues, it would be greatly appreciated. (Incidentally, since these sorts of topics usually elicit "what are you trying to do?" types of responses: ultimately my goal, if I can get XText working at all, is an editor with a syntax highlighter and error checker.)
Re: Need help getting started on a grammar [message #1219890 is a reply to message #1219861] Fri, 06 December 2013 14:34 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
Hi
Welcome to Xtext

for the expressions there is Sven's blog
http://blog.efftinge.de/2010/08/parsing-expressions-with-xtext.html?m=1


In general your grammar should contain assignments


For the ecore problem . how does your import look like? Have a
look/copy and paste from terminals grammar (crtl click on the with
xxx in the top of the grammar)

--
Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext at itemis dot de


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Need help getting started on a grammar [message #1219923 is a reply to message #1219890] Fri, 06 December 2013 18:59 Go to previous messageGo to next message
Jordan Wallet is currently offline Jordan WalletFriend
Messages: 16
Registered: December 2013
Junior Member
Thanks for the response!

The link suggests I may be taking the wrong approach here. My background with grammars is that the starting rule is always at the root of the parse tree, but the example given has the root (Addition) ending up with only terminals as children and with a parent. I find this very confusing. That said, the post doesn't actually have much to say about what assignments do or what all my Type errors mean.

The top of the file is simply the default that was (mostly) automatically generated:
grammar org.xtext.example.mytestlang.MyTestLang with org.eclipse.xtext.common.Terminals

generate myTestLang "LinksNotAllowedInThisForumPostBecauseIAmANewUser_ButTheLanguageWasJustInventedSoThereIsNoURIAnyway"
Re: Need help getting started on a grammar [message #1219925 is a reply to message #1219923] Fri, 06 December 2013 19:30 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
Hi,
i meant: have a look at the terminal grammar. then you can see that
you have to add a import "http://www.eclipse.org/emf/2002/Ecore" as ecore


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Need help getting started on a grammar [message #1219926 is a reply to message #1219925] Fri, 06 December 2013 19:55 Go to previous messageGo to next message
Jordan Wallet is currently offline Jordan WalletFriend
Messages: 16
Registered: December 2013
Junior Member
That's already there, but... it gives a 404 (go ahead and click on your own link). I think we found one of the problems! Has this resource moved somewhere else I wonder? I can't tell just by looking at the EMF page what the URI should be here...

[Updated on: Fri, 06 December 2013 19:56]

Report message to a moderator

Re: Need help getting started on a grammar [message #1219927 is a reply to message #1219926] Fri, 06 December 2013 20:06 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
Hi,

the uri is just a name(space) not an url you can open with your browser

your grammar should look like

grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals

generate myDsl "http://www.xtext.org/example/mydsl/MyDsl"

import "http://www.eclipse.org/emf/2002/Ecore" as ecore

Model:
	greetings+=Greeting*;
	
Greeting:
	'Hello' name=STRING count=MyInt 'times' '!';
	
MyInt returns ecore::EInt:
	INT
;	



and it should work out of the box


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Need help getting started on a grammar [message #1219928 is a reply to message #1219927] Fri, 06 December 2013 20:27 Go to previous messageGo to next message
Jordan Wallet is currently offline Jordan WalletFriend
Messages: 16
Registered: December 2013
Junior Member
Well, thanks, that solves setting return types on terminals to things in ecore, but really that was just because I thought it might make some of my errors go away (I don't actually try to set any types, since I'm fine with everything just being a string).

I still have a lot of "Cannot find compatible type for feature" errors, which I don't understand since I'm only referring to terminals with implicit types, which I understand to mean they're all ecore::EString. I also still don't quite understand what features are, since I have yet to find anything on the internet that explicitly defines them.
Re: Need help getting started on a grammar [message #1219931 is a reply to message #1219928] Fri, 06 December 2013 21:09 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
Hi,

sorry without any reproducable grammar it is hard to say anything.

Feature is a term of ecore
(E) Classes consist of (EStrutural)Features. These features can be (E)Attributes (that have a simple type like EString or EInt or MyEnum) and (E)References. References can be contained (Children) or not contained (Cross References)


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Need help getting started on a grammar [message #1219938 is a reply to message #1219931] Fri, 06 December 2013 22:57 Go to previous messageGo to next message
Jordan Wallet is currently offline Jordan WalletFriend
Messages: 16
Registered: December 2013
Junior Member
To what end do I assign features in my rules? It's fine to say that it's a thing in another thing, but I can't find any explanation of how they are used in parsing or anything.

My errors all seem to arise when things can branch to a terminal or another rule. For example as I posted before
ArithmeticExpression:
	Variable'++' |
	Variable'--' |
	((Variable | ParensExpression | NUMBER) ARITHMETIC_OPERATOR ArithmeticExpression)
;


Or something of the form
RuleOne:
	RuleTwo | RuleThree | TERMINAL
;

Which gives a non-assignment warning, but then gives a "cannot find Type" error if I do something like
RuleOne:
	result=(RuleTwo | RuleThree | TERMINAL)
;

Again, these all eventually reach endpoints with implicit return types, so I don't see why there's any problem with types.

EDIT: Here's a more concrete example:
StringClause:
	clause=(LocalVariable | STRING_TERMINAL)
;

LocalVariable:
	'#'symbol=SIMPLE_NAME
;

terminal fragment ALPHANUM:
	('a'..'z'|'A'..'Z'|'0'..'9'|'_')
;

terminal SIMPLE_NAME:
	('a'..'z')(ALPHANUM)*
;

terminal fragment ESCAPED_CHAR:
	'\\'('n'|'t'|'\\'|'\"'|'\'')
;

terminal STRING_TERMINAL:
	'"'(ESCAPED_CHAR | !('\''|'"'))*'"' |
	'\''(ESCAPED_CHAR | !('\''|'"'))*'\''
;


The first rule gives a "cannot find type" error.

[Updated on: Fri, 06 December 2013 23:21]

Report message to a moderator

Re: Need help getting started on a grammar [message #1219949 is a reply to message #1219938] Sat, 07 December 2013 08:57 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
Hi,

you mix up terminals and non terminals

StringClause:
	clause=(LocalVariable | STRING_TERMINAL)
;

LocalVariable:
	'#' value=SIMPLE_NAME
;

terminal fragment ALPHANUM:
	('a'..'z'|'A'..'Z'|'0'..'9'|'_')
;

terminal SIMPLE_NAME:
	('a'..'z')(ALPHANUM)*
;

terminal fragment ESCAPED_CHAR:
	'\\'('n'|'t'|'\\'|'\"'|'\'')
;

terminal STRING_TERMINAL:
	'"'(ESCAPED_CHAR | !('\''|'"'))*'"' |
	'\''(ESCAPED_CHAR | !('\''|'"'))*'\''
;


or

StringClause:
	child=LocalVariable | value=STRING_TERMINAL
;

LocalVariable:
	'#' SIMPLE_NAME
;

terminal fragment ALPHANUM:
	('a'..'z'|'A'..'Z'|'0'..'9'|'_')
;

terminal SIMPLE_NAME:
	('a'..'z')(ALPHANUM)*
;

terminal fragment ESCAPED_CHAR:
	'\\'('n'|'t'|'\\'|'\"'|'\'')
;

terminal STRING_TERMINAL:
	'"'(ESCAPED_CHAR | !('\''|'"'))*'"' |
	'\''(ESCAPED_CHAR | !('\''|'"'))*'\''
;


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Sat, 07 December 2013 09:18]

Report message to a moderator

Re: Need help getting started on a grammar [message #1219973 is a reply to message #1219949] Sat, 07 December 2013 19:10 Go to previous messageGo to next message
Gary Worsham is currently offline Gary WorshamFriend
Messages: 176
Registered: September 2013
Senior Member
Hello.

I am far from an expert, in fact I have just made a couple of working grammars in the past couple of months. I can tell you in general where I had some problems that sound similar to what you are facing.

#1 Try to avoid creating a bunch of terminal rules. Almost every one I tried backfired and I was better off to make it into a data type rule. There's a certain attraction to trying to create terminal rules as they are the earliest point at which to accomplish something, but at least in my project they usually had unintended side effects.

#2 In the case where a rule does not require an assignment, I had to add "Actions" to make them work properly - an example is:

Before:
IsElse: '@else';

After:
IsElse: {IsElse} '@else';

Good luck! Although I have really accomplished a lot with Xtext and Xtend, I don't have a good idea of what's going on behind the scenes. It being fairly new, there are not a ton of different examples either.
Re: Need help getting started on a grammar [message #1220277 is a reply to message #1219973] Tue, 10 December 2013 19:07 Go to previous messageGo to next message
Jordan Wallet is currently offline Jordan WalletFriend
Messages: 16
Registered: December 2013
Junior Member
Hi, thanks for the replies. They are appreciated, I just didn't have time for this over the weekend.

Dietrich:
I'm not sure what you're trying to tell me. What do these two alternatives do? Why doesn't mine work?

Worsham:
As I mentioned in the original post, other than their syntax I don't understand the difference between terminal rules and data type rules.
I also don't know if a rule requires an assignment because I don't know what the assignments are for. There doesn't seem to be anything in the grammar documentation about them.
Re: Need help getting started on a grammar [message #1220278 is a reply to message #1220277] Tue, 10 December 2013 19:12 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
Hi,

you say:

a StringClause has a child (eobject) clause that is either a primitive String-Literal or a LocalVariable eobject.
this is not allowed in ecore.

a feature has to be an attribute (string, int, ...)
or a (contained) reference (object)

i recomment some basic reading on eclipse modeling framework and ecore.


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Need help getting started on a grammar [message #1220279 is a reply to message #1220278] Tue, 10 December 2013 19:21 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
P.S.

think of java 4 (no autoboxing)
you can assign an object to a primitive and vice versa


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Need help getting started on a grammar [message #1220307 is a reply to message #1220277] Wed, 11 December 2013 01:48 Go to previous messageGo to next message
Gary Worsham is currently offline Gary WorshamFriend
Messages: 176
Registered: September 2013
Senior Member
Jordan Wallet wrote on Tue, 10 December 2013 14:07

Worsham:
As I mentioned in the original post, other than their syntax I don't understand the difference between terminal rules and data type rules.
I also don't know if a rule requires an assignment because I don't know what the assignments are for. There doesn't seem to be anything in the grammar documentation about them.


Terminal rules execute immediately and are not context dependent. Whereas with a data type rule, you tell it what sorts of things are expected there. Terminal rules can't "back up" if they hit something unexpected, whereas data type rules can.

I use assignments as parameters. For example, one of my DSLs is a DSP assembler. It takes statements like:

RDAX REG0, 0.5

After I've parsed this my code generator in Xtend can say, "oh it's an RDAX", so then I can use the assigned values of "REG0" and "0.5" for whatever I want to.

Don't know if it helps. I've only been using these things for a couple months and at the beginning I was pretty baffled.
Re: Need help getting started on a grammar [message #1220314 is a reply to message #1220307] Wed, 11 December 2013 05:17 Go to previous messageGo to next message
Jordan Wallet is currently offline Jordan WalletFriend
Messages: 16
Registered: December 2013
Junior Member
So far I haven't found anything that explains EMF without a lot of curious presumptions that one is already somewhat familiar with EMF; but even so, things that bother to explain anything are rare. Mostly all I've found are references, and learning a system from a giant list-form API reference, though technically possible within a lifetime, is impractical to even attempt. If you know any good resources, links would be appreciated.

I have trouble eliminating terminals because the "char range" operator (..) and "not" (!) are not recognized in non-terminal rules, and that suggests to me that it's probably not designed to have entire alphabets in non-terminal rules.

I tried pushing terminal rules down a level so that no rule directly yields terminals and non-terminals, but there was no effect.
Re: Need help getting started on a grammar [message #1220431 is a reply to message #1220314] Wed, 11 December 2013 17:21 Go to previous messageGo to next message
Gary Worsham is currently offline Gary WorshamFriend
Messages: 176
Registered: September 2013
Senior Member
Lars Vogel has written a lot of very useful tutorials.
http://www.vogella.com/articles/EclipseEMF/article.html

Some other EMF page I bookmarked (about Dynamic EMF):
http://www.ibm.com/developerworks/library/os-eclipse-dynamicemf/

[Updated on: Wed, 11 December 2013 17:26]

Report message to a moderator

Re: Need help getting started on a grammar [message #1221165 is a reply to message #1220431] Tue, 17 December 2013 02:44 Go to previous messageGo to next message
Jordan Wallet is currently offline Jordan WalletFriend
Messages: 16
Registered: December 2013
Junior Member
Thanks for the links, I have been sure to bookmark the first one for future reference. However, I didn't see anything that was applicable to the problems I've been having. I understand less than ever why I need assignments in my grammar, and my new insight into EMF doesn't provide any clues about my "cannot find type" errors. Even if two types are possible (and I'm still not clear on why they are without explicit assignment), it seems like there has to be some kind of automatic falling-back to higher up the inheritance hierarchy, or the system couldn't theoretically parse anything at all; so why does it do fine at some levels but not others? I'm still missing something...
Re: Need help getting started on a grammar [message #1227849 is a reply to message #1221165] Sun, 05 January 2014 20:22 Go to previous messageGo to next message
Jordan Wallet is currently offline Jordan WalletFriend
Messages: 16
Registered: December 2013
Junior Member
I'm still trying to find a solution for my DSL and I was wondering if Xtext is appropriate. I don't need something that compiles in Java, I just want something that can e.g. give me warnings about undeclared variables. Would another tool (e.g. LiClipse) be simpler to get up and running for such a purpose? I haven't made any headway at all with Xtext so far.

[Updated on: Sun, 05 January 2014 20:22]

Report message to a moderator

Re: Need help getting started on a grammar [message #1227858 is a reply to message #1227849] Sun, 05 January 2014 20:59 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
Hi,

without knowing any specific problem it is hard to help.

~Christian


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Need help getting started on a grammar [message #1229544 is a reply to message #1227858] Thu, 09 January 2014 17:21 Go to previous messageGo to next message
Jordan Wallet is currently offline Jordan WalletFriend
Messages: 16
Registered: December 2013
Junior Member
Looking back on the thread, especially where I've posted code blocks, I don't know how I can be more specific than I already have been. Is there something I've forgotten to mention that would help? If so it is almost surely something I've forgotten to address myself.
Re: Need help getting started on a grammar [message #1229546 is a reply to message #1229544] Thu, 09 January 2014 17:26 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
Hi,

the problem is: this grammar language is at it is. it has rules how to specify parser rules that create object,
data type parser rules and lexer rules (terminals)

and you cannot mix up these. this is what i tried to explain.


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Need help getting started on a grammar [message #1254757 is a reply to message #1229546] Sun, 23 February 2014 19:42 Go to previous messageGo to next message
Jordan Wallet is currently offline Jordan WalletFriend
Messages: 16
Registered: December 2013
Junior Member
I'm back! Sorry to necro a thread after such a long absence, but I ate up my window of free time just trying to get this one issue resolved.

So if terminals cannot be derived from non-terminals, why are character ranges disabled in non-terminals? It seems really odd to need to define every single letter and numeral seperately.

EDIT: Now the issues are coming back to me. I just eliminated terminal rules and I still get "cannot find type" errors. Mixing the rules doesn't seem to be the issue.

On a related note, if anyone knows of any documentation on XText other than the official documentation, I'd be very grateful to know about it.

[Updated on: Sun, 23 February 2014 19:52]

Report message to a moderator

Re: Need help getting started on a grammar [message #1254767 is a reply to message #1254757] Sun, 23 February 2014 19:59 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
How does your current grammar look like? and what is the error message

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Need help getting started on a grammar [message #1254771 is a reply to message #1254767] Sun, 23 February 2014 20:01 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
p.S: did you have a look at lorenzo bettinis book. Implementing Domain-Specific Languages with Xtext and Xtend

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Need help getting started on a grammar [message #1254772 is a reply to message #1254771] Sun, 23 February 2014 20:03 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13628
Registered: July 2009
Senior Member
and there is http://xtextcasts.org/ and tons of other blogs and stuff

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Need help getting started on a grammar [message #1255325 is a reply to message #1254771] Mon, 24 February 2014 10:20 Go to previous messageGo to next message
Lorenzo Bettini is currently offline Lorenzo BettiniFriend
Messages: 1770
Registered: July 2009
Location: Firenze, Italy
Senior Member
On 23/02/2014 21:01, Christian Dietrich wrote:
> p.S: did you have a look at lorenzo bettinis book. Implementing
> Domain-Specific Languages with Xtext and Xtend

Thanks Christian, I was about to suggest that :)

Hope this helps
Lorenzo

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Xtext Book:
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book


Re: Need help getting started on a grammar [message #1256580 is a reply to message #1255325] Tue, 25 February 2014 15:30 Go to previous message
Jordan Wallet is currently offline Jordan WalletFriend
Messages: 16
Registered: December 2013
Junior Member
Oh I see, this was all an elaborate plan to convince me to buy a book! Razz Well it worked; I'll give it a shot. Thanks.
Previous Topic:Editor related issues in xtext
Next Topic:How to get rid of Tycho's downloading of p2.indexes on every build
Goto Forum:
  


Current Time: Thu Oct 22 14:34:10 GMT 2020

Powered by FUDForum. Page generated in 0.02725 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top