Eclipse Community Forums: TMF (Xtext) » Need help getting started on a grammar

Home » Modeling » TMF (Xtext) » Need help getting started on a grammar

Need help getting started on a grammar [message #1219861]

Fri, 06 December 2013 05:20

Eclipse User

I've been going at the documentation for a good 4 hours now and I'm just about at the end of my rope. I'm trying to write a simple grammar that resembles a programming language and I can't get all the warnings/errors To go away.

The problem is that I can get any specific one to go away, so I don't know what's wrong with my approach. I feel like I lack some fundamental concept in A few examples of the problems I've run into:

The documentation never explicitly states what a "feature" (assigned value) is. What are they?

The documentation does not properly explain the difference between terminal rules and data type rules. No, really! For example, search the page for "above mentioned reasons", and then try to find reasons above it; you won't find any.

I'm not sure why I would want to, because the tutorials were written from a Java perspective and glossed over data types entirely, but I cannot set return types on terminals:

terminal NUMBER returns ecore::EInt:

Quote:

- Couldn't resolve reference to AbstractMetamodelDeclaration 'ecore'
- Cannot create datatype EInt

This makes me suspect that my project setup is broken, but I just went with the default, so I don't know what gives.

And then this gives me a 'Cannot change type twice within a rule' error:

ArithmeticExpression:
	Variable'++' |
	Variable'--' |
	((Variable | ParensExpression | NUMBER) ARITHMETIC_OPERATOR ArithmeticExpression)
;

But I'm not explicitly declaring any types (the return type above was just illustrative, it's usually not there), so shouldn't everything just be EStrings or inferred?

If anyone can shed any light on any if these issues, it would be greatly appreciated. (Incidentally, since these sorts of topics usually elicit "what are you trying to do?" types of responses: ultimately my goal, if I can get XText working at all, is an editor with a syntax highlighter and error checker.)

Re: Need help getting started on a grammar [message #1219890 is a reply to message #1219861]

Fri, 06 December 2013 09:34

Eclipse User

Hi
Welcome to Xtext

for the expressions there is Sven's blog
http://blog.efftinge.de/2010/08/parsing-expressions-with-xtext.html?m=1

In general your grammar should contain assignments

For the ecore problem . how does your import look like? Have a
look/copy and paste from terminals grammar (crtl click on the with
xxx in the top of the grammar)

--
Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext at itemis dot de

Re: Need help getting started on a grammar [message #1219923 is a reply to message #1219890]

Fri, 06 December 2013 13:59

Eclipse User

Thanks for the response!

The link suggests I may be taking the wrong approach here. My background with grammars is that the starting rule is always at the root of the parse tree, but the example given has the root (Addition) ending up with only terminals as children and with a parent. I find this very confusing. That said, the post doesn't actually have much to say about what assignments do or what all my Type errors mean.

The top of the file is simply the default that was (mostly) automatically generated:

grammar org.xtext.example.mytestlang.MyTestLang with org.eclipse.xtext.common.Terminals

generate myTestLang "LinksNotAllowedInThisForumPostBecauseIAmANewUser_ButTheLanguageWasJustInventedSoThereIsNoURIAnyway"

Re: Need help getting started on a grammar [message #1219925 is a reply to message #1219923]

Fri, 06 December 2013 14:30

Eclipse User

Hi,
i meant: have a look at the terminal grammar. then you can see that
you have to add a import "http://www.eclipse.org/emf/2002/Ecore" as ecore

Re: Need help getting started on a grammar [message #1219926 is a reply to message #1219925]

Fri, 06 December 2013 14:55

Eclipse User

That's already there, but... it gives a 404 (go ahead and click on your own link). I think we found one of the problems! Has this resource moved somewhere else I wonder? I can't tell just by looking at the EMF page what the URI should be here...

[Updated on: Fri, 06 December 2013 14:56] by Moderator

Re: Need help getting started on a grammar [message #1219927 is a reply to message #1219926]

Fri, 06 December 2013 15:06

Eclipse User

Hi,

the uri is just a name(space) not an url you can open with your browser

your grammar should look like

grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals

generate myDsl "http://www.xtext.org/example/mydsl/MyDsl"

import "http://www.eclipse.org/emf/2002/Ecore" as ecore

Model:
	greetings+=Greeting*;
	
Greeting:
	'Hello' name=STRING count=MyInt 'times' '!';
	
MyInt returns ecore::EInt:
	INT
;

and it should work out of the box

Re: Need help getting started on a grammar [message #1219928 is a reply to message #1219927]

Fri, 06 December 2013 15:27

Eclipse User

Well, thanks, that solves setting return types on terminals to things in ecore, but really that was just because I thought it might make some of my errors go away (I don't actually try to set any types, since I'm fine with everything just being a string).

I still have a lot of "Cannot find compatible type for feature" errors, which I don't understand since I'm only referring to terminals with implicit types, which I understand to mean they're all ecore::EString. I also still don't quite understand what features are, since I have yet to find anything on the internet that explicitly defines them.

Re: Need help getting started on a grammar [message #1219931 is a reply to message #1219928]

Fri, 06 December 2013 16:09

Eclipse User

Hi,

sorry without any reproducable grammar it is hard to say anything.

Feature is a term of ecore
(E) Classes consist of (EStrutural)Features. These features can be (E)Attributes (that have a simple type like EString or EInt or MyEnum) and (E)References. References can be contained (Children) or not contained (Cross References)

Re: Need help getting started on a grammar [message #1219938 is a reply to message #1219931]

Fri, 06 December 2013 17:57

Eclipse User

To what end do I assign features in my rules? It's fine to say that it's a thing in another thing, but I can't find any explanation of how they are used in parsing or anything.

My errors all seem to arise when things can branch to a terminal or another rule. For example as I posted before

ArithmeticExpression:
	Variable'++' |
	Variable'--' |
	((Variable | ParensExpression | NUMBER) ARITHMETIC_OPERATOR ArithmeticExpression)
;

Or something of the form

RuleOne:
	RuleTwo | RuleThree | TERMINAL
;

Which gives a non-assignment warning, but then gives a "cannot find Type" error if I do something like

RuleOne:
	result=(RuleTwo | RuleThree | TERMINAL)
;

Again, these all eventually reach endpoints with implicit return types, so I don't see why there's any problem with types.

EDIT: Here's a more concrete example:

StringClause:
	clause=(LocalVariable | STRING_TERMINAL)
;

LocalVariable:
	'#'symbol=SIMPLE_NAME
;

terminal fragment ALPHANUM:
	('a'..'z'|'A'..'Z'|'0'..'9'|'_')
;

terminal SIMPLE_NAME:
	('a'..'z')(ALPHANUM)*
;

terminal fragment ESCAPED_CHAR:
	'\\'('n'|'t'|'\\'|'\"'|'\'')
;

terminal STRING_TERMINAL:
	'"'(ESCAPED_CHAR | !('\''|'"'))*'"' |
	'\''(ESCAPED_CHAR | !('\''|'"'))*'\''
;

The first rule gives a "cannot find type" error.

[Updated on: Fri, 06 December 2013 18:21] by Moderator

Re: Need help getting started on a grammar [message #1219949 is a reply to message #1219938]

Sat, 07 December 2013 03:57

Eclipse User

Hi,

you mix up terminals and non terminals

StringClause:
	clause=(LocalVariable | STRING_TERMINAL)
;

LocalVariable:
	'#' value=SIMPLE_NAME
;

terminal fragment ALPHANUM:
	('a'..'z'|'A'..'Z'|'0'..'9'|'_')
;

terminal SIMPLE_NAME:
	('a'..'z')(ALPHANUM)*
;

terminal fragment ESCAPED_CHAR:
	'\\'('n'|'t'|'\\'|'\"'|'\'')
;

terminal STRING_TERMINAL:
	'"'(ESCAPED_CHAR | !('\''|'"'))*'"' |
	'\''(ESCAPED_CHAR | !('\''|'"'))*'\''
;

StringClause:
	child=LocalVariable | value=STRING_TERMINAL
;

LocalVariable:
	'#' SIMPLE_NAME
;

terminal fragment ALPHANUM:
	('a'..'z'|'A'..'Z'|'0'..'9'|'_')
;

terminal SIMPLE_NAME:
	('a'..'z')(ALPHANUM)*
;

terminal fragment ESCAPED_CHAR:
	'\\'('n'|'t'|'\\'|'\"'|'\'')
;

terminal STRING_TERMINAL:
	'"'(ESCAPED_CHAR | !('\''|'"'))*'"' |
	'\''(ESCAPED_CHAR | !('\''|'"'))*'\''
;

[Updated on: Sat, 07 December 2013 04:18] by Moderator

Re: Need help getting started on a grammar [message #1219973 is a reply to message #1219949]

Sat, 07 December 2013 14:10

Eclipse User

Hello.

I am far from an expert, in fact I have just made a couple of working grammars in the past couple of months. I can tell you in general where I had some problems that sound similar to what you are facing.

#1 Try to avoid creating a bunch of terminal rules. Almost every one I tried backfired and I was better off to make it into a data type rule. There's a certain attraction to trying to create terminal rules as they are the earliest point at which to accomplish something, but at least in my project they usually had unintended side effects.

#2 In the case where a rule does not require an assignment, I had to add "Actions" to make them work properly - an example is:

Before:
IsElse: '@else';

After:
IsElse: {IsElse} '@else';

Good luck! Although I have really accomplished a lot with Xtext and Xtend, I don't have a good idea of what's going on behind the scenes. It being fairly new, there are not a ton of different examples either.

Re: Need help getting started on a grammar [message #1220277 is a reply to message #1219973]

Tue, 10 December 2013 14:07

Eclipse User

Hi, thanks for the replies. They are appreciated, I just didn't have time for this over the weekend.

Dietrich:
I'm not sure what you're trying to tell me. What do these two alternatives do? Why doesn't mine work?

Worsham:
As I mentioned in the original post, other than their syntax I don't understand the difference between terminal rules and data type rules.
I also don't know if a rule requires an assignment because I don't know what the assignments are for. There doesn't seem to be anything in the grammar documentation about them.

Re: Need help getting started on a grammar [message #1220278 is a reply to message #1220277]

Tue, 10 December 2013 14:12

Eclipse User

Hi,

you say:

a StringClause has a child (eobject) clause that is either a primitive String-Literal or a LocalVariable eobject.
this is not allowed in ecore.

a feature has to be an attribute (string, int, ...)
or a (contained) reference (object)

i recomment some basic reading on eclipse modeling framework and ecore.

Re: Need help getting started on a grammar [message #1220279 is a reply to message #1220278]

Tue, 10 December 2013 14:21

Eclipse User

P.S.

think of java 4 (no autoboxing)
you can assign an object to a primitive and vice versa

Re: Need help getting started on a grammar [message #1220307 is a reply to message #1220277]

Tue, 10 December 2013 20:48

Eclipse User

Jordan Wallet wrote on Tue, 10 December 2013 14:07

Worsham:
As I mentioned in the original post, other than their syntax I don't understand the difference between terminal rules and data type rules.
I also don't know if a rule requires an assignment because I don't know what the assignments are for. There doesn't seem to be anything in the grammar documentation about them.

Terminal rules execute immediately and are not context dependent. Whereas with a data type rule, you tell it what sorts of things are expected there. Terminal rules can't "back up" if they hit something unexpected, whereas data type rules can.

I use assignments as parameters. For example, one of my DSLs is a DSP assembler. It takes statements like:

RDAX REG0, 0.5

After I've parsed this my code generator in Xtend can say, "oh it's an RDAX", so then I can use the assigned values of "REG0" and "0.5" for whatever I want to.

Don't know if it helps. I've only been using these things for a couple months and at the beginning I was pretty baffled.

Re: Need help getting started on a grammar [message #1220314 is a reply to message #1220307]

Wed, 11 December 2013 00:17

Eclipse User

So far I haven't found anything that explains EMF without a lot of curious presumptions that one is already somewhat familiar with EMF; but even so, things that bother to explain anything are rare. Mostly all I've found are references, and learning a system from a giant list-form API reference, though technically possible within a lifetime, is impractical to even attempt. If you know any good resources, links would be appreciated.

I have trouble eliminating terminals because the "char range" operator (..) and "not" (!) are not recognized in non-terminal rules, and that suggests to me that it's probably not designed to have entire alphabets in non-terminal rules.

I tried pushing terminal rules down a level so that no rule directly yields terminals and non-terminals, but there was no effect.

Re: Need help getting started on a grammar [message #1220431 is a reply to message #1220314]

Wed, 11 December 2013 12:21

Eclipse User

Lars Vogel has written a lot of very useful tutorials.
http://www.vogella.com/articles/EclipseEMF/article.html

Some other EMF page I bookmarked (about Dynamic EMF):
http://www.ibm.com/developerworks/library/os-eclipse-dynamicemf/

[Updated on: Wed, 11 December 2013 12:26] by Moderator

Re: Need help getting started on a grammar [message #1221165 is a reply to message #1220431]

Mon, 16 December 2013 21:44

Eclipse User

Thanks for the links, I have been sure to bookmark the first one for future reference. However, I didn't see anything that was applicable to the problems I've been having. I understand less than ever why I need assignments in my grammar, and my new insight into EMF doesn't provide any clues about my "cannot find type" errors. Even if two types are possible (and I'm still not clear on why they are without explicit assignment), it seems like there has to be some kind of automatic falling-back to higher up the inheritance hierarchy, or the system couldn't theoretically parse anything at all; so why does it do fine at some levels but not others? I'm still missing something...

Re: Need help getting started on a grammar [message #1227849 is a reply to message #1221165]

Sun, 05 January 2014 15:22

Eclipse User

I'm still trying to find a solution for my DSL and I was wondering if Xtext is appropriate. I don't need something that compiles in Java, I just want something that can e.g. give me warnings about undeclared variables. Would another tool (e.g. LiClipse) be simpler to get up and running for such a purpose? I haven't made any headway at all with Xtext so far.

[Updated on: Sun, 05 January 2014 15:22] by Moderator

Re: Need help getting started on a grammar [message #1227858 is a reply to message #1227849]

Sun, 05 January 2014 15:59

Eclipse User

Hi,

without knowing any specific problem it is hard to help.

~Christian

Re: Need help getting started on a grammar [message #1229544 is a reply to message #1227858]

Thu, 09 January 2014 12:21

Eclipse User

Looking back on the thread, especially where I've posted code blocks, I don't know how I can be more specific than I already have been. Is there something I've forgotten to mention that would help? If so it is almost surely something I've forgotten to address myself.

Re: Need help getting started on a grammar [message #1229546 is a reply to message #1229544]

Thu, 09 January 2014 12:26

Eclipse User

Hi,

the problem is: this grammar language is at it is. it has rules how to specify parser rules that create object,
data type parser rules and lexer rules (terminals)

and you cannot mix up these. this is what i tried to explain.

Re: Need help getting started on a grammar [message #1254757 is a reply to message #1229546]

Sun, 23 February 2014 14:42

Eclipse User

I'm back! Sorry to necro a thread after such a long absence, but I ate up my window of free time just trying to get this one issue resolved.

So if terminals cannot be derived from non-terminals, why are character ranges disabled in non-terminals? It seems really odd to need to define every single letter and numeral seperately.

EDIT: Now the issues are coming back to me. I just eliminated terminal rules and I still get "cannot find type" errors. Mixing the rules doesn't seem to be the issue.

On a related note, if anyone knows of any documentation on XText other than the official documentation, I'd be very grateful to know about it.

[Updated on: Sun, 23 February 2014 14:52] by Moderator

Re: Need help getting started on a grammar [message #1254767 is a reply to message #1254757]

Sun, 23 February 2014 14:59

Eclipse User

How does your current grammar look like? and what is the error message

Re: Need help getting started on a grammar [message #1254771 is a reply to message #1254767]

Sun, 23 February 2014 15:01

Eclipse User

p.S: did you have a look at lorenzo bettinis book. Implementing Domain-Specific Languages with Xtext and Xtend

Re: Need help getting started on a grammar [message #1254772 is a reply to message #1254771]

Sun, 23 February 2014 15:03

Eclipse User

and there is http://xtextcasts.org/ and tons of other blogs and stuff

Re: Need help getting started on a grammar [message #1255325 is a reply to message #1254771]

Mon, 24 February 2014 05:20

Eclipse User

On 23/02/2014 21:01, Christian Dietrich wrote:
> p.S: did you have a look at lorenzo bettinis book. Implementing
> Domain-Specific Languages with Xtext and Xtend

Thanks Christian, I was about to suggest that :)

Hope this helps
Lorenzo

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Xtext Book:
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book

Re: Need help getting started on a grammar [message #1256580 is a reply to message #1255325]

Tue, 25 February 2014 10:30

Eclipse User

Oh I see, this was all an elaborate plan to convince me to buy a book! Razz

Well it worked; I'll give it a shot. Thanks.

Previous Topic:	Editor related issues in xtext
Next Topic:	How to get rid of Tycho's downloading of p2.indexes on every build

Goto Forum:

-=] Back to Top [=-

Current Time: Wed Jul 23 19:40:11 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter