Eclipse Community Forums: TMF (Xtext) » [Newbie] Problem with grammar

Home » Modeling » TMF (Xtext) » [Newbie] Problem with grammar

[Newbie] Problem with grammar [message #892861]

Sat, 30 June 2012 13:16

Eclipse User

Hello,

I'm trrying to specify a grammar to support the Byteman rules language.
I'm starting with a subset of the language, as described here:

Each 'rule' has the following form:
- zero or more comment lines which must start with '#' (whatever the other chars)
- a rule name defined after the 'RULE' keyword. The rule name can be anything but must at least contain a non-space char.
- a 'ENDRULE' line to indicate the end of the rule definition.

eg:

# a comment
# another comment
RULE bar
ENDRULE

Here's the .xtext I have for now (excluding the usual headers 'grammar ... with ... generate ...' )

ByteManRule:
	COMMENT*
	'RULE' name=RULE_NAME LINE_BREAK
	'ENDRULE' LINE_BREAK*
	;

terminal CHAR: ('a'..'z'|'A'..'Z');

terminal RULE_NAME:
	(CHAR|INT) (CHAR|INT|' '|'\t')*
;

terminal LINE_BREAK: ('\r'? '\n');

terminal COMMENT: '#' -> LINE_BREAK;

When I generate the XText artifacts, I get the following errors:

...
error(208): ...InternalBytemanDsl.g:174:1: The following token definitions can never be matched because prior tokens match the same input: RULE_INT
2839 [main] INFO  or.validation.JavaValidatorFragment  - generating Java-based EValidator API
error(208): ...InternalBytemanDsl.g:349:1: The following token definitions can never be matched because prior tokens match the same input: RULE_INT
...

And in the Eclipse runtime, using the XText Editor, I get an error marker at the 'RULE...' line:
"Missing 'RULE' at 'RULE bar'"
(Strangely, I have a marker in the editor, but nothing in the project explorer nor in the Problems view)

Can you help me ?
Thanks
Best regards,
Xavier

Re: [Newbie] Problem with grammar [message #892862 is a reply to message #892861]

Sat, 30 June 2012 14:00

Eclipse User

Xavier,

I think you need to show more of your grammar. It seems likely that
your RULE_INT accepts the same characters as RULE_NAME. I.e., "1" can
be both an int and a name, which is a problem.

On 30/06/2012 7:16 PM, Xavier Coulon wrote:
> Hello,
>
> I'm trrying to specify a grammar to support the Byteman rules language.
> I'm starting with a subset of the language, as described here:
>
> Each 'rule' has the following form:
> - zero or more comment lines which must start with '#' (whatever the
> other chars) - a rule name defined after the 'RULE' keyword. The rule
> name can be anything but must at least contain a non-space char. - a
> 'ENDRULE' line to indicate the end of the rule definition.
>
> eg:
>
> # a comment
> # another comment
> RULE bar
> ENDRULE
>
>
> Here's the .xtext I have for now (excluding the usual headers 'grammar
> ... with ... generate ...' )
>
>
> ByteManRule:
> COMMENT*
> 'RULE' name=RULE_NAME LINE_BREAK
> 'ENDRULE' LINE_BREAK*
> ;
>
> terminal CHAR: ('a'..'z'|'A'..'Z');
>
> terminal RULE_NAME:
> (CHAR|INT) (CHAR|INT|' '|'\t')*
> ;
>
> terminal LINE_BREAK: ('\r'? '\n');
>
> terminal COMMENT: '#' -> LINE_BREAK;
>
> When I generate the XText artifacts, I get the following errors:
>
> ..
> error(208): ...InternalBytemanDsl.g:174:1: The following token
> definitions can never be matched because prior tokens match the same
> input: RULE_INT
> 2839 [main] INFO or.validation.JavaValidatorFragment - generating
> Java-based EValidator API
> error(208): ...InternalBytemanDsl.g:349:1: The following token
> definitions can never be matched because prior tokens match the same
> input: RULE_INT
> ..
>
>
> And in the Eclipse runtime, using the XText Editor, I get an error
> marker at the 'RULE...' line: "Missing 'RULE' at 'RULE bar'"
> (Strangely, I have a marker in the editor, but nothing in the project
> explorer nor in the Problems view)
>
> Can you help me ?
> Thanks
> Best regards,
> Xavier

Re: [Newbie] Problem with grammar [message #892865 is a reply to message #892862]

Sat, 30 June 2012 15:03

Eclipse User

Hi,

do not define terminals if not absolutely necessary. Use datatype rules instead. Overlapping terminals cause major problems.

Name:(CHAR|INT)(CHAR|INT|' '|'\t')*;

Alex

Re: [Newbie] Problem with grammar [message #892866 is a reply to message #892865]

Sat, 30 June 2012 15:14

Eclipse User

@Ed: as I wrote before, I showed the whole grammar (except the "grammar..." and "generate..." lines). I think the RULE_INT in question comes from the org.eclipse.xtext.common.terminals.

@Alex: thanks, I'll try this way.

Thanks for your help
Best regards,
Xavier

Re: [Newbie] Problem with grammar [message #892872 is a reply to message #892866]

Sat, 30 June 2012 18:12

Eclipse User

Hello again,

Moving a little forward, here's my (new) current grammar:

bytemanDsl:
	rules+=(ByteManRule)*;

ByteManRule:
	(COMMENT)*
	'RULE' name=RuleName 
	'ENDRULE'
	;
	
terminal CHAR: ('a'..'z'|'A'..'Z');

terminal COMMENT: ('#' -> ('\r'? '\n'));

RuleName:
	(CHAR|INT|' '|'\t')+
;

In the example below, I now get a single error that I can't explain:

# foo
RULE f o o
ENDRULE

# bar
RULE bar
ENDRULE

The validator gives an error on 'bar' (at line 'RULE bar') with the given message:

extraneous input 'bar', expecting ENDRULE

Any idea ?
Thanks very much
Best regards,
Xavier

Re: [Newbie] Problem with grammar [message #892883 is a reply to message #892872]

Sun, 01 July 2012 05:13

Eclipse User

Hi,

I assume you use the default terminal grammar, i.e. you import the definition of the ID terminal rule. This is why "f o o" works (your CHAR rule has higher priority for single letters) but the ID rule will win for "bar" (a longer token can be produced). Two of the possible solutions:
-copy the terminal rules you need from the default grammar to your grammar and dont import that
-overwrite the ID-rule with something appropriate in your use case.

Alex

Re: [Newbie] Problem with grammar [message #892915 is a reply to message #892883]

Sun, 01 July 2012 16:44

Eclipse User

Alex,

That's awesome ! I did follow your first advice and got the expected result with a nice grammar.

Thanks very much for your help

Best regards,
Xavier

Previous Topic:	Do I have to "reinvent" things like interfaces with methods and impl classes?
Next Topic:	[Solved] Xtext 2.2.1 project build stucks at "Validate resources"

Goto Forum:

-=] Back to Top [=-

Current Time: Sun Jul 27 12:02:26 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter