Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Number/String syntax errors in Xtext(I am learning Xtext's rule syntax and running into some problems)
Number/String syntax errors in Xtext [message #1015865] Sun, 03 March 2013 23:50 Go to next message
Robert Brown III is currently offline Robert Brown IIIFriend
Messages: 35
Registered: July 2009
Member
Greetings:

I am attempting to create a syntax that contains numbers and strings, as in the following:

Date: 2 Mar 2013

I have created a set of rules that more- or- less accommodate this:

Date:
'Date': DAY Month YEAR
;

terminal DAY:
('1'..'9') | (('1'..'3')('0'..'9'))
;

Month:
name = ('Jan'|'Feb'|'Mar'|'Apr)
;

terminal YEAR
('0'..'2)('0'..'9')('0'..'9')('0'..'9')
;

This set of rules seems to have a number of problems:

The Date rule, as written, seems to generate an error: "Cannot change type twice within a rule". I have no idea what that means, except that Xtext will apparently not allow numbers and choice strings in the same rule.

If I remove the Month rule from Date, Xtext compiles, but the syntax doesn't seem to work right. The DAY rule is supposed to provide a choice between a single digit from 1 to 9 and a two- digit number, but for some reason it only accepts a two- digit number. So while I can enter a line like:

Date: 12 2013

is accepted but a date like:

Date: 2 2013

is not.

Have I found a bug in the rules that breaks numbers in terminal rules? Or is there something missing that causes the ignoring of the '|' in my numbers? Also: what on Earth does the "Cannot change type twice within a rule" error mean and how can I fix it???

Someone please advise.
Re: Number/String syntax errors in Xtext [message #1015866 is a reply to message #1015865] Mon, 04 March 2013 00:32 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik LindbergFriend
Messages: 2500
Registered: July 2009
Senior Member
You have overlapping terminals: YEAR and DAY. Use data rules instead and
INT terminal, Then validate for day and year to ensure they are in range.

Also, you seem to be missing assignment.

Try something like:

Date : 'Date' ':' d=Day m=Month y=Year ;

Day : INT ;
Month : 'Jan'|'Feb'|'Mar'|'Apr' ;
Year : INT ;

Then add validation/check to the rules Day and Year to ensure the
integer is in range.

Hope that helps.
Regards
- henrik

On 2013-04-03 24:50, Robert Brown III wrote:
> Greetings:
>
> I am attempting to create a syntax that contains numbers and strings, as
> in the following:
>
> Date: 2 Mar 2013
>
> I have created a set of rules that more- or- less accommodate this:
>
> Date:
> 'Date': DAY Month YEAR
> ;
>
> terminal DAY:
> ('1'..'9') | (('1'..'3')('0'..'9'))
> ;
>
> Month:
> name = ('Jan'|'Feb'|'Mar'|'Apr)
> ;
>
> terminal YEAR
> ('0'..'2)('0'..'9')('0'..'9')('0'..'9')
> ;
>
> This set of rules seems to have a number of problems:
>
> The Date rule, as written, seems to generate an error: "Cannot
> change type twice within a rule". I have no idea what that means, except
> that Xtext will apparently not allow numbers and choice strings in the
> same rule.
>
> If I remove the Month rule from Date, Xtext compiles, but the syntax
> doesn't seem to work right. The DAY rule is supposed to provide a choice
> between a single digit from 1 to 9 and a two- digit number, but for some
> reason it only accepts a two- digit number. So while I can enter a line
> like:
>
> Date: 12 2013
>
> is accepted but a date like:
>
> Date: 2 2013
>
> is not.
>
> Have I found a bug in the rules that breaks numbers in terminal rules?
> Or is there something missing that causes the ignoring of the '|' in my
> numbers? Also: what on Earth does the "Cannot change type twice within a
> rule" error mean and how can I fix it???
>
> Someone please advise.
>
Re: Number/String syntax errors in Xtext [message #1015881 is a reply to message #1015866] Mon, 04 March 2013 07:21 Go to previous messageGo to next message
Robert Brown III is currently offline Robert Brown IIIFriend
Messages: 35
Registered: July 2009
Member
Thank you, Henrik. Your post has been very helpful...

I do have a problem, though. You knew to use INT and the "m=" and "d=" syntax for creating the rule(s) that I wanted. Unfortunately, there is no way I could have known about these things because they are not described anywhere in the tutorials or even in the reference documentation.

For example, I did not know that INT existed until you mentioned it in your post. I also was unaware of the validation capability that you mention, and have no idea how to implement it for my grammer.

The tutorials on the Xtext site give great examples of using the rules syntax, but the information they provide is very limited and the explanations are incomplete. Other documentation provides some good overall concepts (though without concrete context which makes their explanations largely useless), but nowhere have I been able to find a concrete explanation of all the various things that you can put into a grammar. In other words, there really isn't a complete explanation of the elements of rule syntax. No one at the Xtext site seems to want to describe the use of INT, or how to generally use the elements of a grammar (for example: when to put together a rule like

Date: 'Date': DAY Month YEAR

and when to use

Date: 'Date' ':' d=Day m=Month y=Year;

And no one even mentions validation, never mind how to create a validator!

Could you or someone provide information on how to do the validation you are talking about? Or better: could you point me to a place where I can learn how to do validation? Also: is there a place that you can point me to s site that actually describes, in a straightforward manner, things like the use of INT, and the construction of actual datatype, terminal, and features?

Again, thanks for your help.
Re: Number/String syntax errors in Xtext [message #1015895 is a reply to message #1015881] Mon, 04 March 2013 08:22 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 6485
Registered: July 2009
Senior Member
Hi,

have a look at

http://www.eclipse.org/Xtext/documentation.html#_15
http://www.eclipse.org/Xtext/documentation.html#syntax (Data Type Rules Subsection)
http://www.eclipse.org/Xtext/documentation.html#valueconverter
http://www.eclipse.org/Xtext/documentation.html#validation
Re: Number/String syntax errors in Xtext [message #1016009 is a reply to message #1015895] Mon, 04 March 2013 15:20 Go to previous messageGo to next message
Robert Brown III is currently offline Robert Brown IIIFriend
Messages: 35
Registered: July 2009
Member
Thanks, Christian, but I've already been through these links. They are the reasons for my complaints about the reference documentation.

The various descriptions talk a lot (too much, in my opinion) about the source classes and not enough about the actual rules syntax. For example: there is no mention of INT anywhere that I can find in the documentation.

My point here is that I have not found anything in the documentation -- including the reference documentation -- that would have enabled me to find the information that Henrik gave me in this Topic. That is the sort of thing I am asking for.

I need some documentation that provides straightforward information about the rules language. Whether this documentation goes into it (in more detail than the reference documentation), or consists of lots and lots of tutorials and examples doesn't matter. I need to get detailed information on the actual rules language, not on the various classes that implement it.
Re: Number/String syntax errors in Xtext [message #1016068 is a reply to message #1016009] Mon, 04 March 2013 19:39 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 6485
Registered: July 2009
Senior Member
Hi,

first i want to state that i am NOT a Xtext Committer so i am not the guy to blame.
Documentation is always a hard thing and it is impossbile to make it right for everyone.
For Xtext in particular there a tons of examples and blog articles out there beside the reference documentation.

Regarding your Specific Problem:

The existence of a supergrammar (with the INT rule) should come clear by the heavy use of it in the examples.besides it is stated explicitely in the docs

Quote:

The rule DataType starts with a keyword 'datatype', followed by an identifier which is parsed by a rule called ID. The rule ID is defined in the super grammar org.eclipse.xtext.common.Terminals and parses a single word, a.k.a identifier. You can navigate to the declaration by using F3 on the rule call. The value returned by the call to ID is assigned (=) to the feature name.


Quote:
In the secret compartments example there are no explicitly defined terminal rules, since it only uses the ID rule which is inherited from the grammar org.eclipse.xtext.common.Terminals (cf. Grammar Mixins). Therein the ID rule is defined as follows:


so if you have a look at the examples there is made a lot of use of ID (explained) and the use of INT should then be a simple adoption

there is a expressive section about how the parsing works (parser background if of course helpful)
the basic result is
Quote:

Rules
Basically parsing can be separated in the following phases.
1 Lexing
2 Parsing


from the difference with lexing an parsing a "data type rule" is explained as a matter of choice

Quote:

Data Type Rules
Data type rules are parsing-phase rules, which create instances of EDataType (src) instead of EClass (src). Thinking about it, one may discover that they are quite similar to terminal rules. However, the nice thing about data type rules is that they are actually parser rules and are therefore

(1) context sensitive


In your acutal example

Date: 'Date': DAY Month YEAR

and when to use 

Date: 'Date' ':' d=Day m=Month y=Year;


the first is a datatype rule and the second a normal parser rule.
both solve your problem anyway.

In general there is a lot on the grammar language in the docs - maybe more tutorials would help in your case
if you are not the "reference doc guy" to find out what to apply when.

And of course you can always file a bugzilla if you are missing a particular thing in the docs.

Or use the forum. we are willingly to answer almost any question.

~Christian
Re: Number/String syntax errors in Xtext [message #1016091 is a reply to message #1016068] Mon, 04 March 2013 22:35 Go to previous messageGo to next message
Robert Brown III is currently offline Robert Brown IIIFriend
Messages: 35
Registered: July 2009
Member
Quote:

first i want to state that i am NOT a Xtext Committer so i am not the guy to blame.
Documentation is always a hard thing and it is impossbile to make it right for everyone.


Believe me: I am not blaming you for the documentation. I was just pointing out that I had already been through the documentation that you pointed me to, and for the reasons I explained it did not help me. Apologies if you thought I was assigning blame...

Quote:

The existence of a supergrammar (with the INT rule) should come clear by the heavy use of it in the examples.besides it is stated explicitely in the docs

...

so if you have a look at the examples there is made a lot of use of ID (explained) and the use of INT should then be a simple adoption


Those quotes you provided in your answer do not "explicitly" describe anything. The section you took them from talks about a supergrammar without giving any details about it. There is no mention of INT or how to use it. Furthermore, any examples that used INT were not found by me -- at least not on the Xtext site.

As for the use of INT being clear from the way ID is explained: perhaps, but not necessarily. It is in all caps, meaning it is likely a terminal rule, which I could not have known was standard unless someone told me. Even so, since I did not know that INT existed,(having not seen it in any example so far) it is somewhat difficult to infer its use from anything.

Quote:

the first is a datatype rule and the second a normal parser rule.
both solve your problem anyway.


No, actually the second did not, because using it helped generate the error that caused me to come here.

Quote:

In general there is a lot on the grammar language in the docs - maybe more tutorials would help in your case
if you are not the "reference doc guy" to find out what to apply when.


Perhaps so, though not for the reasons you suggest. Actually, I have no problem using references. I just don't learn from them -- especially if they do not tell me anything useful.

Quote:

Or use the forum. we are willingly to answer almost any question.


My reason for asking for different doco was to avoid bugging you guys all the time with what will probably be dumb questions. You have helped a great deal, though, because you have let me know that where documentation is concerned I cannot expect to see much better than I have seen and, ultimately, the best way to learn this language is to keep finding examples and asking questions about them here when necessary.

For this I thank you. I am not being sarcastic. I want to learn this grammar language and you have indicated to me what I will need to do to learn it.

Thanks for your assistance.
Re: Number/String syntax errors in Xtext [message #1016117 is a reply to message #1016091] Tue, 05 March 2013 06:52 Go to previous message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 6485
Registered: July 2009
Senior Member
Hmmmm i fear Xtext is not Docucmentable in that way. since you can nearly change everything a "there is and so use/change it" docs would be millions of lines.
never the less if there some basic concepts like the INT rule and you feel it is not documented well enhough please file a bugzilla and state what could be done better.

so the expectation is: the docs says: there is an ID in terminals. so you go to terminals and see whats there besides ID.
but as said before: This is a matter of taste (and time) an how and what to document.


regarding your particular problem i think we both did not get the problem or are not talking with each other.

when i speak of a datatype rule and if you have a look at the docs you see that it contains no assignment. your actually does

so i was thinking of something like

MyDate returns ecore::EDate:
	INT "." INT "." INT
;


or

Date: "Date:" value=MyDate;
MyDate: Day Month Year;
Day : INT ;
Month : 'Jan'|'Feb'|'Mar'|'Apr' ;
Year : INT ;

[Updated on: Tue, 05 March 2013 06:53]

Report message to a moderator

Previous Topic:Serialization assert in Xtext tester fails with unclear message
Next Topic:Scoping
Goto Forum:
  


Current Time: Sun Nov 23 03:19:48 GMT 2014

Powered by FUDForum. Page generated in 0.04174 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software