Skip to main content



      Home
Home » Modeling » TMF (Xtext) » (xtext) Grammar's problem
(xtext) Grammar's problem [message #1791939] Sun, 08 July 2018 10:51 Go to next message
Eclipse UserFriend
Hello,

I'm trying to specify a grammar to the go language, but I'm having a few problems and I don't know how to correct it or even why it is happening.

This is my grammar until now:

Model:
	elements+=int_lit;
	
int_lit:
	decimal_lit | octal_lit | hex_lit
;

decimal_lit:
	DECIMAL_DIGIT (DECIMAL_DIGIT)*
;

octal_lit:
	"0" OCTAL_DIGIT*
;

hex_lit:
	"0" ( "x" | "X" ) HEX_DIGIT HEX_DIGIT*
;
	
terminal IDENTIFIER: 
	LETTER (LETTER | DECIMAL_DIGIT)*
;

terminal A_F: 
	"A" .. "F" | "a" .. "f" 
;

terminal LETTER:
	A_F | 'g' .. 'z' | 'G' .. 'Z' | "_"
;

terminal HEX_DIGIT:
	DECIMAL_DIGIT | A_F
;

terminal OCTAL_DIGIT:
	('0'..'7')
;

terminal ESCAPED_CHAR:
	"\\" ( "a" | "b" | "f" | "n" | "r" | "t" | "v" | "\\" | "\'" | "\"" )
;

terminal DECIMAL_DIGIT:
	OCTAL_DIGIT | '8' | '9';

terminal BINARY_OP:
	"||" | "&&" | REL_OP | ADD_OP | MUL_OP
;

terminal REL_OP:
	"==" | "!=" | "<" | "<=" | ">" | ">="
;

terminal ADD_OP:
	"+" | "-" | "|" | "^"
;

terminal MUL_OP:
	"*" | "/" | "%" | "<<" | ">>" | "&" | "&^"
;

terminal UNARY_OP:
	"+" | "-" | "!" | "^" | "*" | "&" | "<-"
;	


These are the errors I get after running it:

error(208): ../org.xtext.example.mydsl/src-gen/org/xtext/example/mydsl/parser/antlr/internal/InternalMyDsl.g:283:1: The following token definitions can never be matched because prior tokens match the same input: RULE_OCTAL_DIGIT,RULE_DECIMAL_DIGIT
error(208): ../org.xtext.example.mydsl.ide/src-gen/org/xtext/example/mydsl/ide/contentassist/antlr/internal/InternalMyDsl.g:468:1: The following token definitions can never be matched because prior tokens match the same input: RULE_OCTAL_DIGIT,RULE_DECIMAL_DIGIT

Can someone explain me why this is happening and how can I prevent this type of error?

P.S: Sorry for any mistakes, english is not my mother language.
Re: (xtext) Grammar's problem [message #1792124 is a reply to message #1791939] Thu, 12 July 2018 00:29 Go to previous messageGo to next message
Eclipse UserFriend
Xtext uses ANTLR.
ANTLR does parsing in two steps. first lexing. then parsing
lexer must be context free.

since you use terminal rules inside each other a terminal rule written first can match a terminal rule written second if it is a 1:1 reuse
- you might have a look at "terminal fragments"
- you might not do everything with different terminal rules
- you might have a look at the concept of terminal rules

=> there is no easy solution without putting work into it. you might have a look at other grammars like xbase to see what they do.

Re: (xtext) Grammar's problem [message #1792137 is a reply to message #1792124] Thu, 12 July 2018 02:25 Go to previous message
Eclipse UserFriend
If you like to debug the ANTLR grammar refer to this blog post: https://blogs.itemis.com/en/debugging-xtext-grammars-what-to-do-when-your-language-is-ambiguous

A simple rule of thumb: Use as few terminal rules as possible. Make higher order patterns of tokens to a datatype parser rule.

Previous Topic:Handling terminals without assigned feature, for code generation
Next Topic:Restoring the model to a saved state
Goto Forum:
  


Current Time: Sun Jun 22 11:46:22 EDT 2025

Powered by FUDForum. Page generated in 0.03815 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top