Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Terminal Rule Bug?  () 1 Vote
Terminal Rule Bug? [message #804089] Wed, 22 February 2012 07:21 Go to next message
Heinz Huber is currently offline Heinz HuberFriend
Messages: 45
Registered: February 2012
Member
Hi,

i'm trying to build my own language which will parse my Configuration Files.
Everything went fine, but now i stuck.

My Configuration Files got different Names for Clients in the network. I started to write checks (terminals) to approve that these conventions are okay.

The Clients got Names in the following formats:
Test_L_1_Sender
Test_Testclient_Sender
Statustest_Sender
TEST01_Sender
NULL_Sender
ABC_to_TEST2

My first Idea was to create a terminal rule like the following one:
terminal SENDERNAME:
	('a'..'z'|'A'..'Z'|'_'|'0'..'9')+ 
;

Here i got the following Error "The following token definitions can never be matched because prior tokens match the same input: RULE_INT". Changing '0'..'9' into INT doesn't make it better.

After a long time with my best friend Google i found the Information, that terminal rules aren't as good as they seem. So i created a "normal" rule like the following:
SenderName:
	  (BUCHUNTER|INT)+
;

terminal BUCHUNTER:
	('a'..'z'|'A'..'Z'|'_')+
;

This one works better, but he is expecting a number now oO
Names without a number work very fine ... but at the moment i add a number to my Name a error shows up again! The Error which is created in the editor plugin is the following: "required (...)+ loop did not match anything at input 'Test_L_1_Sender'"! Changing (BUCHUNTER|INT)+ to (INT|BUCHUNTER)+ in the rule doesn't change the situation.

I don't really know how i can locate the problem, so i can fix this!
The following Code is the way i used the rule/terminal. Maybe this is my mistake?
	'Name' '=' name=SenderName 


Hope someone can help!



[Updated on: Wed, 22 February 2012 07:29]

Report message to a moderator

Re: Terminal Rule Bug? [message #804098 is a reply to message #804089] Wed, 22 February 2012 07:37 Go to previous messageGo to next message
Vlad Dumitrescu is currently offline Vlad DumitrescuFriend
Messages: 431
Registered: July 2009
Location: Gothenburg
Senior Member
Hi!

I think you want

terminal BUCHUNTER:
	'a'..'z'|'A'..'Z'|'_'
;


best regards,
Vlad
Re: Terminal Rule Bug? [message #804111 is a reply to message #804098] Wed, 22 February 2012 08:03 Go to previous messageGo to next message
Heinz Huber is currently offline Heinz HuberFriend
Messages: 45
Registered: February 2012
Member
Hi,

thanks for your help, but the error is still there.
I changed the terminal rule like you said, but this doesn't help me!

The error is still the following: "required (...)+ loop did not match anything at input 'Test_L_1_Sender'".
All Clients which are in my editor testfile are marked at the "Name = ..." field with the same error message!

Do you have another idea? Smile
Re: Terminal Rule Bug? [message #804119 is a reply to message #804111] Wed, 22 February 2012 08:18 Go to previous messageGo to next message
Vlad Dumitrescu is currently offline Vlad DumitrescuFriend
Messages: 431
Registered: July 2009
Location: Gothenburg
Senior Member
Hi,

Can your names start with a digit? If not, then you can use

terminal SENDERNAME:
	('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*
;


If yes, then I think you can't use the default terminals at the same time, you will have to define your own.

regards,
Vlad
Re: Terminal Rule Bug? [message #804132 is a reply to message #804089] Wed, 22 February 2012 08:36 Go to previous messageGo to next message
Alexander Nittka is currently offline Alexander NittkaFriend
Messages: 1193
Registered: July 2009
Senior Member
Hi,

how about a datatype rule

SenderName hidden(): (INT)? ID;

You should not define terminals if they are not absolutely necessary, as overlapping terminals are a hell. If a name must have a very particular form, it might be better to allow "anything" remotely OK in the grammar (using existing terminals, own datatype rules) and then validate the name. This allows you to give meaningful error messages as well (e.g. "there must be at least 3 underscores in a sender name").

Alex
Re: Terminal Rule Bug? [message #804142 is a reply to message #804119] Wed, 22 February 2012 08:51 Go to previous messageGo to next message
Heinz Huber is currently offline Heinz HuberFriend
Messages: 45
Registered: February 2012
Member
Hi,

no they can't start with digits.
While i was trying to fix this bug i got a nearly same rule you got.

I included your rule into my Code and got another problem.
Now the Clientnames which got a digit in their names work fine! But the other ones doesn't!
The parser is expecting a number in every name! I get an error while there is no number in the names, but when I insert one, the error is gone.

The Error is the following: "mismatched input 'TESThi_Sender' expecting RULE_SENDERNAME".
In my opinion your rule right to generate Names like this one! But in my generated Plugin it isn't.

If it matters im using the ANTLR Parser v3.2.0 with the latest Eclipse (Eclipse IDE for Java Developers 1.4.1.20110909-1818) and the XText (Xtext SDK 2.2.1.v201112130541) -> This one are the latest i think ... or is there a bugfix oder update i missed?

Thanks again for your Help Vlad!
I hope we can fix this Smile



Re: Terminal Rule Bug? [message #804145 is a reply to message #804132] Wed, 22 February 2012 08:56 Go to previous messageGo to next message
Heinz Huber is currently offline Heinz HuberFriend
Messages: 45
Registered: February 2012
Member
Do you have an example for me?
I don't really know what this means for my code ...

Or is there something in the documentation of Xtext?


Alexander Nittka wrote on Wed, 22 February 2012 03:36
Hi,

how about a datatype rule

SenderName hidden(): (INT)? ID;

You should not define terminals if they are not absolutely necessary, as overlapping terminals are a hell. If a name must have a very particular form, it might be better to allow "anything" remotely OK in the grammar (using existing terminals, own datatype rules) and then validate the name. This allows you to give meaningful error messages as well (e.g. "there must be at least 3 underscores in a sender name").

Alex

Re: Terminal Rule Bug? [message #804220 is a reply to message #804145] Wed, 22 February 2012 10:30 Go to previous messageGo to next message
Heinz Huber is currently offline Heinz HuberFriend
Messages: 45
Registered: February 2012
Member
Okay Alexander i got what you meant.
The way with validations sounds clever in my ears.

But now i've got a new little problem.
I created a rule with ANY_OTHER like the following:
SenderName:
	ANY_OTHER+
;

This should make it possible to insert everything, right? But it doesn't!
Now every Name I insert is wrong and doesn't fit ... but why? oO

In my validation class I will insert a new method which will check the names and throw a error if necessary.

Could someone please help me again ... ^^
Thanks in advance Smile
Re: Terminal Rule Bug? [message #804284 is a reply to message #804220] Wed, 22 February 2012 12:29 Go to previous messageGo to next message
Alexander Nittka is currently offline Alexander NittkaFriend
Messages: 1193
Registered: July 2009
Senior Member
Hi,

lexing is greedy, tokens are made as long as possible. That is ANY_OTHER will match only if no other terminal rule matches. That is, make the datatype rule something like
SenderName hidden(): (INT|ID|ANY_OTHER)+;

Alex


Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext@itemis.de
Re: Terminal Rule Bug? [message #804297 is a reply to message #804284] Wed, 22 February 2012 12:49 Go to previous messageGo to next message
Heinz Huber is currently offline Heinz HuberFriend
Messages: 45
Registered: February 2012
Member
Hi,

thanks again, but this changes nothing.
The bug is still there!

I directly copied the thing you wrote into my code and tried it, but there is still the same bug.
Maybe it's just magic or something like this oO

To be sure:
SenderName hidden(): (INT|ID|ANY_OTHER)+;

Sender:
	'Sender%'
	'Name=' name=SenderName
	('IgnoreQueueFull=' ('true'|'false'))?
	insert=Queueaddress
;


That's the part of the code with the Sendername.
Another question which i got is -> Why is it necessary to write insert=Queueaddress?
Queueaddress is another rule of my language. Is this right? Or is the bug generated there? When i eleminate the "insert=" a error shows up which tells me: "An unassigned rule call is not allowed, when the 'current' was already created."

Hope you got another idea Smile

[Updated on: Wed, 22 February 2012 12:52]

Report message to a moderator

Re: Terminal Rule Bug? [message #804330 is a reply to message #804297] Wed, 22 February 2012 13:47 Go to previous messageGo to next message
Alexander Nittka is currently offline Alexander NittkaFriend
Messages: 1193
Registered: July 2009
Senior Member
Hi,

have you tried simply to write Sender: ... name=ID...

If this does not work, is it possible for you to provide the full grammar along with a sample model file you expect to be correct but which has a parse error? As to your second question. The grammar describes the syntax of the file to be parsed but also how to instantiate the objects from the file.
Sender becomes an object of type Sender, it has an attribute name (due to name=SenderName).
Queueaddress is also an "object definition", but if you do not assign it to a feature/attribute (i.e. you leave out the "insert="-part), the model construction does not work. You already have a Sender-Object, and now it is unclear what to do with the Queueaddress-Object.

Alex



Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext@itemis.de
Re: Terminal Rule Bug? [message #804347 is a reply to message #804330] Wed, 22 February 2012 14:15 Go to previous messageGo to next message
Heinz Huber is currently offline Heinz HuberFriend
Messages: 45
Registered: February 2012
Member
Hi,

I tried the simple name=ID thing long time ago.
The bug was there too Sad

I attached the .xtext file and one file with some examples of my grammar.

Names of Sender could be like the following:
Test_L_1_Sender
Test_Testclient_Sender
Statustest_Sender
TEST01_Sender
NULL_Sender
ABC_to_TEST2

The bug I got is located in the Name= field at the Sender Element. The Listener works very fine!
The error message is: "required (...)+ loop did not match anything at input 'Test_Sender'". This is only at the elements without a number! If i add a number, the entry name is fine.

Hope you can find something Smile
Re: Terminal Rule Bug? [message #804447 is a reply to message #804347] Wed, 22 February 2012 16:31 Go to previous messageGo to next message
Vlad Dumitrescu is currently offline Vlad DumitrescuFriend
Messages: 431
Registered: July 2009
Location: Gothenburg
Senior Member
Hi,

Your terminals BUCHUNTER, BUCHUNTERPUNKT and CONNECTION conflict with ID: one of them is chosen instead of ID. You should make them also non-terminals.

regards,
Vlad
Re: Terminal Rule Bug? [message #804552 is a reply to message #804347] Wed, 22 February 2012 19:14 Go to previous messageGo to next message
Alexander Nittka is currently offline Alexander NittkaFriend
Messages: 1193
Registered: July 2009
Senior Member
Hi,

there are some nasty details about the names in your grammar. As already noted, most of your terminals overlap which causes the problem. Also, CONNECTION contains the # which marks the start of a single line comment, not very nice.

Test.Listener reuses the keyword Listener, a similar problem you will have with the single character keywords a, m, t etc.

Most urgent is the definition of a CONNECTION. Is it always "\{something\}" or is it just the example?
Are spaces allowed between the + and Listener at the beginning of a Listener? Usually I am all for splitting up keywords, but for you to get started '+Listener', '+Sender', '+QueueAddress' might be an option (otherwise you will have problems using Listener within a name; you will have to enumerate every keyword allowed in datatype rule like QueueName). Currently, I would suggest the following grammar to start with.

Model:
	(elements += AbstractElement)* 
;

AbstractElement:
	Listener|Sender
;

Listener:
	'+Listener' '%'
	'Name' '=' name=ID
	insert=Queueaddress
	'-'
;

Sender:
	'+Sender' '%'
	'Name' '=' name=ID
	('IgnoreQueueFull' '=' (ignoreQueueFull?='true' | 'false'))?
	insert=Queueaddress
	'-'
;


Queueaddress:
	'+QueueAddress'
	'Class' '=' class=ID
	('+Connection'
	'a' '=' a = Connection
	'm' '=' m = Connection
	't' '=' t = Connection
	'h' '=' h = Connection
	'p' '=' p = Connection
	'c' '=' c = Connection
	'-')? '+Queue'
	('n' '=' n = QueueName)?
	'-''-'
;

QueueName:
	(ID|'.'|INT)+ 	
;

terminal Connection: '\\{'->'\\}';

terminal SL_COMMENT :
'#' !('\n'|'\r')* ('\r'? '\n')?;

terminal ML_COMMENT: 
	'*' -> '*'
;
Re: Terminal Rule Bug? [message #804951 is a reply to message #804552] Thu, 23 February 2012 07:15 Go to previous messageGo to next message
Heinz Huber is currently offline Heinz HuberFriend
Messages: 45
Registered: February 2012
Member
Hi,

thanks for your nice work!
This one works perfect! Smile

The CONNECTION Element isn't always like this. It could also be a direct IP.
So it doesn't has do start with the "\" and end with the "\" Elements.

I will try to finish my grammar now. Hope this works without any problems Smile

Thanks again! Very Happy
Re: Terminal Rule Bug? [message #808215 is a reply to message #804951] Mon, 27 February 2012 14:42 Go to previous messageGo to next message
Heinz Huber is currently offline Heinz HuberFriend
Messages: 45
Registered: February 2012
Member
Hi Guys,

the past days i got no problems while working on my project.
It is nearly finished now, but i found another problem.

In my configuration files it is necessary to have elements like "+Name". Name could be anything which is included in the terminal ID. (A-Z, a-z, _, 0-9). My first idea was to just do the following:

Destination:
	'+'name=ID
	'Sender' '=' sender=[Sender|ID]
	'-'
;


Now he is expecting a whitespace between + and the specific name. But i don't really know why!

My next idea was to create a rule for this like the following one:

DestinationName: 
'+' ID+;


This rule isn't solving my problem, too. I tried everything i could imagin, but nothing helped. The error messages are like: "Multiple markers at this line
- no viable alternative at character 'F' - mismatched input 'AFC' expecting '-'". And when i'm adding the whitespace between the + and the Name, the error is gone! But this kind of Tags isn't possible for my language.

The interesting thing about all this is, that there are 2 words which are correct (without the whitespace) for the parser. These ones are: "W*" (* stands for any possible insertion) and "Uninteresting". All other names are wrong!

Does anyone has a idea where the problem is?
I just want to create Elements like:
+Test
+Element
+Name
+Hello
+Whatever

Thanks in advance for helping me again ... Smile
Re: Terminal Rule Bug? [message #809112 is a reply to message #808215] Tue, 28 February 2012 13:48 Go to previous messageGo to next message
Alexander Nittka is currently offline Alexander NittkaFriend
Messages: 1193
Registered: July 2009
Senior Member
Hi,

at this point the keyword definitions starting with "+" fire back. The lexer is greedy and as there are keywords that start with "+" and are longer than one character, you get the mismatched input error.
Try replacing '+Listener' and others by '+''Listener'. Of course, no Listener is a keyword and you will have to explicitly allow it as a name (as ID will not fire anymore). So if Listener is a valid name, you do something like

... name=MyId...

MyId: 'Listener'|...|ID;

Alex
Re: Terminal Rule Bug? [message #809818 is a reply to message #809112] Wed, 29 February 2012 09:48 Go to previous message
Heinz Huber is currently offline Heinz HuberFriend
Messages: 45
Registered: February 2012
Member
Thanks again! Your hint worked very well! Smile
My grammar is finished Very Happy
Now i'm creating validations, corretion hints and all that stuff!

Thanks again Alex!

[Updated on: Wed, 29 February 2012 09:48]

Report message to a moderator

Previous Topic:custom content assist help needed
Next Topic:[Xtext] Enabling user customization for formatting
Goto Forum:
  


Current Time: Thu Sep 26 03:56:02 GMT 2024

Powered by FUDForum. Page generated in 0.03828 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top