Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » (xtext) Grammar's problem
(xtext) Grammar's problem [message #1791939] Sun, 08 July 2018 14:51 Go to next message
Erick Santana Rodrigues is currently offline Erick Santana RodriguesFriend
Messages: 1
Registered: July 2018
Junior Member
Hello,

I'm trying to specify a grammar to the go language, but I'm having a few problems and I don't know how to correct it or even why it is happening.

This is my grammar until now:

Model:
	elements+=int_lit;
	
int_lit:
	decimal_lit | octal_lit | hex_lit
;

decimal_lit:
	DECIMAL_DIGIT (DECIMAL_DIGIT)*
;

octal_lit:
	"0" OCTAL_DIGIT*
;

hex_lit:
	"0" ( "x" | "X" ) HEX_DIGIT HEX_DIGIT*
;
	
terminal IDENTIFIER: 
	LETTER (LETTER | DECIMAL_DIGIT)*
;

terminal A_F: 
	"A" .. "F" | "a" .. "f" 
;

terminal LETTER:
	A_F | 'g' .. 'z' | 'G' .. 'Z' | "_"
;

terminal HEX_DIGIT:
	DECIMAL_DIGIT | A_F
;

terminal OCTAL_DIGIT:
	('0'..'7')
;

terminal ESCAPED_CHAR:
	"\\" ( "a" | "b" | "f" | "n" | "r" | "t" | "v" | "\\" | "\'" | "\"" )
;

terminal DECIMAL_DIGIT:
	OCTAL_DIGIT | '8' | '9';

terminal BINARY_OP:
	"||" | "&&" | REL_OP | ADD_OP | MUL_OP
;

terminal REL_OP:
	"==" | "!=" | "<" | "<=" | ">" | ">="
;

terminal ADD_OP:
	"+" | "-" | "|" | "^"
;

terminal MUL_OP:
	"*" | "/" | "%" | "<<" | ">>" | "&" | "&^"
;

terminal UNARY_OP:
	"+" | "-" | "!" | "^" | "*" | "&" | "<-"
;	


These are the errors I get after running it:

error(208): ../org.xtext.example.mydsl/src-gen/org/xtext/example/mydsl/parser/antlr/internal/InternalMyDsl.g:283:1: The following token definitions can never be matched because prior tokens match the same input: RULE_OCTAL_DIGIT,RULE_DECIMAL_DIGIT
error(208): ../org.xtext.example.mydsl.ide/src-gen/org/xtext/example/mydsl/ide/contentassist/antlr/internal/InternalMyDsl.g:468:1: The following token definitions can never be matched because prior tokens match the same input: RULE_OCTAL_DIGIT,RULE_DECIMAL_DIGIT

Can someone explain me why this is happening and how can I prevent this type of error?

P.S: Sorry for any mistakes, english is not my mother language.
Re: (xtext) Grammar's problem [message #1792124 is a reply to message #1791939] Thu, 12 July 2018 04:29 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12235
Registered: July 2009
Senior Member
Xtext uses ANTLR.
ANTLR does parsing in two steps. first lexing. then parsing
lexer must be context free.

since you use terminal rules inside each other a terminal rule written first can match a terminal rule written second if it is a 1:1 reuse
- you might have a look at "terminal fragments"
- you might not do everything with different terminal rules
- you might have a look at the concept of terminal rules

=> there is no easy solution without putting work into it. you might have a look at other grammars like xbase to see what they do.



Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: (xtext) Grammar's problem [message #1792137 is a reply to message #1792124] Thu, 12 July 2018 06:25 Go to previous message
Karsten Thoms is currently offline Karsten ThomsFriend
Messages: 614
Registered: July 2009
Location: Dortmund, Germany
Senior Member

If you like to debug the ANTLR grammar refer to this blog post: https://blogs.itemis.com/en/debugging-xtext-grammars-what-to-do-when-your-language-is-ambiguous

A simple rule of thumb: Use as few terminal rules as possible. Make higher order patterns of tokens to a datatype parser rule.

Previous Topic:Handling terminals without assigned feature, for code generation
Next Topic:Restoring the model to a saved state
Goto Forum:
  


Current Time: Fri Sep 21 14:02:00 GMT 2018

Powered by FUDForum. Page generated in 0.02073 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top