Eclipse Community Forums: TMF (Xtext) » Problem with simple grammar

Home

Home » Modeling » TMF (Xtext) » Problem with simple grammar

Problem with simple grammar [message #871408]

Sun, 13 May 2012 08:39

Eclipse User

Hi,

I am trying to add support for a web language called ObjectiveJ and I am starting with the following grammar :

compilation_unit:
	objj+=objectivej*
;

objectivej:   
	preprocessor_declaration
	| typeDeclaration* 
    ;
 
 preprocessor_declaration:
	'@import' file_specification
	;
	
 file_specification: 
 	('<'|'"')(ID ('/' | '\\' | '.')?)+ ('>' | '"')
 ;
 
typeDeclaration:
	class_implementation
	| category_implementation
;

class_implementation:
	'@implementation' 
	(
		class_name ( ':' superclass_name )
		( instance_variables )?
	)
	'@end';

category_implementation:
	'@implementation'
	(
		class_name '(' category_name ')' 
		( instance_variables )?
	)
	'@end';
	
class_name: 
	ID
	;
	
superclass_name:
	ID
	;

category_name:
	ID
	;
	
instance_variables:
	'{' instance_variable_declaration '}';

instance_variable_declaration:	
	ANY_OTHER
	;

My goal is first to be able to parse the following code :

@import <Foundation/CPObject.j>
@import <AppKit/CPView.j>
@import "MyClass.j"

@implementation Address : CPObject
{
   CPString name;
   CPString city;
}

// a line comment

@end

But when I try to generate the parser I get some errors like :
warning(200): ../org.cappuccino.lang/src-gen/org/cappuccino/lang/parser/antlr/internal/InternalObjectiveJ.g:98:1: Decision can match input such as "{EOF, '@import', '@implementation'}" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
error(201): ../org.cappuccino.lang/src-gen/org/cappuccino/lang/parser/antlr/internal/InternalObjectiveJ.g:98:1: The following alternatives can never be matched: 2

warning(200): ../org.cappuccino.lang/src-gen/org/cappuccino/lang/parser/antlr/internal/InternalObjectiveJ.g:142:1: Decision can match input such as "'@implementation'" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): ../org.cappuccino.lang/src-gen/org/cappuccino/lang/parser/antlr/internal/InternalObjectiveJ.g:119:1: Decision can match input such as "'@import'" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input

What am I doing wrong in my grammar ?

Thanks

Re: Problem with simple grammar [message #871410 is a reply to message #871408]

Sun, 13 May 2012 09:44

Eclipse User

Hi,

the problem is that your grammar is ambigous. one reason for that seems to be that you hate assignments (=,+=,?=) and (nearly) dont you them at all.

compilation_unit:
	objj+=objectivej*
;

objectivej:   
	preprocessor_declaration
	| typeDeclaration 
    ;
 
 preprocessor_declaration:
	'@import' file=file_specification
	;
	
 file_specification: 
 	('<'|'"')(ID ('/' | '\\' | '.')?)+ ('>' | '"')
 ;
 
typeDeclaration:
	class_implementation
	| category_implementation
;

class_implementation:
	'@implementation' 
	(
		name=class_name ( ':' superclass_name=superclass_name )
		( instance_variables )?
	)
	'@end';

category_implementation:
	'@implementation'
	(
		name=class_name '(' category_name=category_name ')' 
		( instance_variables=instance_variables )?
	)
	'@end';
	
class_name: 
	ID
	;
	
superclass_name:
	ID
	;

category_name:
	ID
	;
	
instance_variables:
	'{' instance_variable_declaration '}';

instance_variable_declaration:	
	ANY_OTHER
	;

p.s: if i read the word preprocessor in conjunction with Xtext it is time to run away as fast as i can.

besides this you dont seem to care about Xtext naming conventions.

~Christian

Re: Problem with simple grammar [message #871427 is a reply to message #871410]

Sun, 13 May 2012 12:06

Eclipse User

Thanks Christian you are helping a lot.

Re: Problem with simple grammar [message #871435 is a reply to message #871427]

Sun, 13 May 2012 13:06

Eclipse User

Someone else has an idéal about how to fix it?An example is better than 100 words
Thanks

Re: Problem with simple grammar [message #871437 is a reply to message #871410]

Sun, 13 May 2012 13:28

Eclipse User

See below...

> compilation_unit:
> objj+=objectivej*
> ;
>
> objectivej: preprocessor_declaration
> | typeDeclaration ;
>
> preprocessor_declaration:
> '@import' file=file_specification
> ;
>
> file_specification: ('<'|'"')(ID ('/' | '\\' | '.')?)+ ('>' | '"')
> ;
>
> typeDeclaration:
> class_implementation
> | category_implementation
> ;
>
> class_implementation:
> '@implementation' (
Ambiguous - the category_implementation also starts with the token
'@implementation'

> name=class_name ( ':' superclass_name=superclass_name )
> ( instance_variables )?
you probably want to assign the result of instance_variables
e.g. instance_variables = instance_variables.
> )
> '@end';
>
> category_implementation:
> '@implementation'
Ambiguous - clashes with class_implementation

> (
> name=class_name '(' category_name=category_name ')' (
> instance_variables=instance_variables )?
> )
> '@end';
>
> class_name: ID
> ;
>
> superclass_name:
> ID
> ;
>
> category_name:
> ID
> ;
>
The previous three rules don't do much for you - you can simply assign
ID directly instead of having a DataRule that is equal to a terminal. It
would make sense if you have data type conversion associated with the
data rules.

> instance_variables:
> '{' instance_variable_declaration '}';
>
> instance_variable_declaration:
> ANY_OTHER
> ;
>
This looks odd - is "instance_variables" really just one character
(ANY_OTHER) enclosed in braces? You probably meant something else. Also,
whatever result that is parsed, you probably want to assign it to
something. e.g.

instance_variables :
'{' instance_variable_declrations += instance_variable_declaration+ '}'
;
instance_variable_declaration : ANY_OTHER+ ;

But that is just a wild guess, and kind of odd, as ANY_OTHER means non
INT, ID, STRING, keyword. If you need help here, you need to describe
how it is supposed to work.

Hope that helps you a bit.
Regards
- henrik

Re: Problem with simple grammar [message #871463 is a reply to message #871437]

Sun, 13 May 2012 18:26

Eclipse User

Ok so actually I think it's not as easy as I thought, I might need to spend years to learn this new syntax while I was just get used to antlr.
I rewrote the grammar from scratch and now I am starting with this :

Objectivej:
  (types+=Type)*;

Type:
  ImportDecl 
  //| TypeDecl
  ;

ImportDecl:
	"@import" name=FileSpecification
	;

FileSpecification: 
 	('<'|'"')(ID ('/' | '\\' | '.')?)+ ('>' | '"')
 	;

But when I try it doesn't recognize the import using " but only the ones with <> :

@import <Foundation/CPObject.j> //OK
@import <AppKit/CPView.j> // OK
@import "MyClass.j" // Error : no viable input

And where can I find some real language grammar (c, java, c#, pascal, basic, ...) ?

[Updated on: Sun, 13 May 2012 18:29] by Moderator

Re: Problem with simple grammar [message #871467 is a reply to message #871463]

Sun, 13 May 2012 20:15

Eclipse User

You need to post the complete grammar with all declarations in order for
anyone to help with what is going on.

My guess here is that you use the default terminals, and it has a
definition of String that will eat the double quote.

So...
FileSpecification : STRING | ('<' (ID ('/' | '\\' | '.')?)+ '>') ;

Would do the trick - you have to check that the FileSpecification is
valid though. (Simply add a @Check method in your validation class).

Good move to start with a really small grammar... makes it much easier
to understand where problems come from.

Regards
- henrik

On 2012-14-05 24:26, smartmobili Mising name wrote:
> Ok so actually I think it's not as easy as I thought, I might need to
> spend years to learn this new syntax while I was just get used to antlr.
> I rewrote the grammar from scratch and now I am starting with this :
>
>
> Objectivej:
> (types+=Type)*;
>
> Type:
> ImportDecl //| TypeDecl
> ;
>
> ImportDecl:
> "@import" name=FileSpecification
> ;
>
> FileSpecification: ('<'|'"')(ID ('/' | '\\' | '.')?)+ ('>' | '"')
> ;
>
>
> But when I try it doesn't recognize the import using " instead <> :
>
> @import <Foundation/CPObject.j> //OK
> @import <AppKit/CPView.j> // OK
> @import "MyClass.j" // Error : no viable input
>
>
>

Re: Problem with simple grammar [message #915888 is a reply to message #871467]

Tue, 18 September 2012 02:52

Eclipse User

The terminals are to come from lexer which is normally tightly coupled with parser. So parser is a simple crawler on a state-machine (FA). SM is created by parser generators (see the tables generated by yacc for a grammar). For an SR parser, during the S (shift) the parser wants a token and that is provided by lexer. So parser does not know how to read the file or tokenize - it is the job of lexer and pre-processor (if applicable for the language) to supply the terminal tokens (in the form of an integer - for eg IDENTIFIER means 10, while means 11 ...) during the shift operation of the crawler. Normally the lex spec files ends with .l where you have to specify the regular grammar for your lexer so that the file is tokenized as per the lexical specs of the language in question.

Crawler gets into issues, when conflicts happen as crawler simply goes by the state tables (normally there would be 3 tables, shift-states, reduce states, reduce-maps - if my memory is right !). So the parser generator has to produce the tables in the right way for the language in question. For that you have to have unambiguous grammar. Normally SR conflict is resolved by taking the S path, RR is resolved by the 1st production of the grammar among the applicable productions for top of the state-stack.

The theory is simple discrete mathematics. If you have interest you can learn much faster.

Regards
Veda

Previous Topic:	How to get objects from a resource from within the ProposalProvider ?
Next Topic:	15 Minute Tutorial Extended

Goto Forum:

-=] Back to Top [=-

Current Time: Wed Jul 23 06:17:21 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter