Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » [Newbie] Problem with grammar
[Newbie] Problem with grammar [message #892861] Sat, 30 June 2012 17:16 Go to next message
Xavier Coulon is currently offline Xavier CoulonFriend
Messages: 58
Registered: July 2009
Member
Hello,

I'm trrying to specify a grammar to support the Byteman rules language.
I'm starting with a subset of the language, as described here:

Each 'rule' has the following form:
- zero or more comment lines which must start with '#' (whatever the other chars)
- a rule name defined after the 'RULE' keyword. The rule name can be anything but must at least contain a non-space char.
- a 'ENDRULE' line to indicate the end of the rule definition.

eg:
# a comment
# another comment
RULE bar
ENDRULE


Here's the .xtext I have for now (excluding the usual headers 'grammar ... with ... generate ...' )

ByteManRule:
	COMMENT*
	'RULE' name=RULE_NAME LINE_BREAK
	'ENDRULE' LINE_BREAK*
	;

terminal CHAR: ('a'..'z'|'A'..'Z');

terminal RULE_NAME:
	(CHAR|INT) (CHAR|INT|' '|'\t')*
;

terminal LINE_BREAK: ('\r'? '\n');

terminal COMMENT: '#' -> LINE_BREAK; 


When I generate the XText artifacts, I get the following errors:
...
error(208): ...InternalBytemanDsl.g:174:1: The following token definitions can never be matched because prior tokens match the same input: RULE_INT
2839 [main] INFO  or.validation.JavaValidatorFragment  - generating Java-based EValidator API
error(208): ...InternalBytemanDsl.g:349:1: The following token definitions can never be matched because prior tokens match the same input: RULE_INT
...


And in the Eclipse runtime, using the XText Editor, I get an error marker at the 'RULE...' line:
"Missing 'RULE' at 'RULE bar'"
(Strangely, I have a marker in the editor, but nothing in the project explorer nor in the Problems view)

Can you help me ?
Thanks
Best regards,
Xavier
Re: [Newbie] Problem with grammar [message #892862 is a reply to message #892861] Sat, 30 June 2012 18:00 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 31158
Registered: July 2009
Senior Member
Xavier,

I think you need to show more of your grammar. It seems likely that
your RULE_INT accepts the same characters as RULE_NAME. I.e., "1" can
be both an int and a name, which is a problem.


On 30/06/2012 7:16 PM, Xavier Coulon wrote:
> Hello,
>
> I'm trrying to specify a grammar to support the Byteman rules language.
> I'm starting with a subset of the language, as described here:
>
> Each 'rule' has the following form:
> - zero or more comment lines which must start with '#' (whatever the
> other chars) - a rule name defined after the 'RULE' keyword. The rule
> name can be anything but must at least contain a non-space char. - a
> 'ENDRULE' line to indicate the end of the rule definition.
>
> eg:
>
> # a comment
> # another comment
> RULE bar
> ENDRULE
>
>
> Here's the .xtext I have for now (excluding the usual headers 'grammar
> ... with ... generate ...' )
>
>
> ByteManRule:
> COMMENT*
> 'RULE' name=RULE_NAME LINE_BREAK
> 'ENDRULE' LINE_BREAK*
> ;
>
> terminal CHAR: ('a'..'z'|'A'..'Z');
>
> terminal RULE_NAME:
> (CHAR|INT) (CHAR|INT|' '|'\t')*
> ;
>
> terminal LINE_BREAK: ('\r'? '\n');
>
> terminal COMMENT: '#' -> LINE_BREAK;
>
> When I generate the XText artifacts, I get the following errors:
>
> ..
> error(208): ...InternalBytemanDsl.g:174:1: The following token
> definitions can never be matched because prior tokens match the same
> input: RULE_INT
> 2839 [main] INFO or.validation.JavaValidatorFragment - generating
> Java-based EValidator API
> error(208): ...InternalBytemanDsl.g:349:1: The following token
> definitions can never be matched because prior tokens match the same
> input: RULE_INT
> ..
>
>
> And in the Eclipse runtime, using the XText Editor, I get an error
> marker at the 'RULE...' line: "Missing 'RULE' at 'RULE bar'"
> (Strangely, I have a marker in the editor, but nothing in the project
> explorer nor in the Problems view)
>
> Can you help me ?
> Thanks
> Best regards,
> Xavier
Re: [Newbie] Problem with grammar [message #892865 is a reply to message #892862] Sat, 30 June 2012 19:03 Go to previous messageGo to next message
Alexander Nittka is currently offline Alexander NittkaFriend
Messages: 1193
Registered: July 2009
Senior Member
Hi,

do not define terminals if not absolutely necessary. Use datatype rules instead. Overlapping terminals cause major problems.

Name:(CHAR|INT)(CHAR|INT|' '|'\t')*;

Alex
Re: [Newbie] Problem with grammar [message #892866 is a reply to message #892865] Sat, 30 June 2012 19:14 Go to previous messageGo to next message
Xavier Coulon is currently offline Xavier CoulonFriend
Messages: 58
Registered: July 2009
Member
@Ed: as I wrote before, I showed the whole grammar (except the "grammar..." and "generate..." lines). I think the RULE_INT in question comes from the org.eclipse.xtext.common.terminals.

@Alex: thanks, I'll try this way.

Thanks for your help
Best regards,
Xavier
Re: [Newbie] Problem with grammar [message #892872 is a reply to message #892866] Sat, 30 June 2012 22:12 Go to previous messageGo to next message
Xavier Coulon is currently offline Xavier CoulonFriend
Messages: 58
Registered: July 2009
Member
Hello again,

Moving a little forward, here's my (new) current grammar:

bytemanDsl:
	rules+=(ByteManRule)*;

ByteManRule:
	(COMMENT)*
	'RULE' name=RuleName 
	'ENDRULE'
	;
	
terminal CHAR: ('a'..'z'|'A'..'Z');

terminal COMMENT: ('#' -> ('\r'? '\n'));

RuleName:
	(CHAR|INT|' '|'\t')+
;


In the example below, I now get a single error that I can't explain:
# foo
RULE f o o
ENDRULE

# bar
RULE bar
ENDRULE


The validator gives an error on 'bar' (at line 'RULE bar') with the given message:
extraneous input 'bar', expecting ENDRULE


Any idea ?
Thanks very much
Best regards,
Xavier
Re: [Newbie] Problem with grammar [message #892883 is a reply to message #892872] Sun, 01 July 2012 09:13 Go to previous messageGo to next message
Alexander Nittka is currently offline Alexander NittkaFriend
Messages: 1193
Registered: July 2009
Senior Member
Hi,

I assume you use the default terminal grammar, i.e. you import the definition of the ID terminal rule. This is why "f o o" works (your CHAR rule has higher priority for single letters) but the ID rule will win for "bar" (a longer token can be produced). Two of the possible solutions:
-copy the terminal rules you need from the default grammar to your grammar and dont import that
-overwrite the ID-rule with something appropriate in your use case.

Alex
Re: [Newbie] Problem with grammar [message #892915 is a reply to message #892883] Sun, 01 July 2012 20:44 Go to previous message
Xavier Coulon is currently offline Xavier CoulonFriend
Messages: 58
Registered: July 2009
Member
Alex,

That's awesome ! I did follow your first advice and got the expected result with a nice grammar.

Thanks very much for your help

Best regards,
Xavier
Previous Topic:Do I have to "reinvent" things like interfaces with methods and impl classes?
Next Topic:[Solved] Xtext 2.2.1 project build stucks at "Validate resources"
Goto Forum:
  


Current Time: Sat May 30 09:44:08 GMT 2020

Powered by FUDForum. Page generated in 0.07438 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top