Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Problem with simple grammar
Problem with simple grammar [message #871408] Sun, 13 May 2012 12:39 Go to next message
smartmobili Missing name is currently offline smartmobili Missing name
Messages: 5
Registered: July 2009
Junior Member
Hi,

I am trying to add support for a web language called ObjectiveJ and I am starting with the following grammar :

compilation_unit:
	objj+=objectivej*
;

objectivej:   
	preprocessor_declaration
	| typeDeclaration* 
    ;
 
 preprocessor_declaration:
	'@import' file_specification
	;
	
 file_specification: 
 	('<'|'"')(ID ('/' | '\\' | '.')?)+ ('>' | '"')
 ;
 
typeDeclaration:
	class_implementation
	| category_implementation
;

class_implementation:
	'@implementation' 
	(
		class_name ( ':' superclass_name )
		( instance_variables )?
	)
	'@end';

category_implementation:
	'@implementation'
	(
		class_name '(' category_name ')' 
		( instance_variables )?
	)
	'@end';
	
class_name: 
	ID
	;
	
superclass_name:
	ID
	;

category_name:
	ID
	;
	
instance_variables:
	'{' instance_variable_declaration '}';

instance_variable_declaration:	
	ANY_OTHER
	;


My goal is first to be able to parse the following code :

@import <Foundation/CPObject.j>
@import <AppKit/CPView.j>
@import "MyClass.j"

@implementation Address : CPObject
{
   CPString name;
   CPString city;
}

// a line comment

@end


But when I try to generate the parser I get some errors like :
warning(200): ../org.cappuccino.lang/src-gen/org/cappuccino/lang/parser/antlr/internal/InternalObjectiveJ.g:98:1: Decision can match input such as "{EOF, '@import', '@implementation'}" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
error(201): ../org.cappuccino.lang/src-gen/org/cappuccino/lang/parser/antlr/internal/InternalObjectiveJ.g:98:1: The following alternatives can never be matched: 2

warning(200): ../org.cappuccino.lang/src-gen/org/cappuccino/lang/parser/antlr/internal/InternalObjectiveJ.g:142:1: Decision can match input such as "'@implementation'" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): ../org.cappuccino.lang/src-gen/org/cappuccino/lang/parser/antlr/internal/InternalObjectiveJ.g:119:1: Decision can match input such as "'@import'" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input

What am I doing wrong in my grammar ?

Thanks
Re: Problem with simple grammar [message #871410 is a reply to message #871408] Sun, 13 May 2012 13:44 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian Dietrich
Messages: 6396
Registered: July 2009
Senior Member
Hi,

the problem is that your grammar is ambigous. one reason for that seems to be that you hate assignments (=,+=,?=) and (nearly) dont you them at all.

compilation_unit:
	objj+=objectivej*
;

objectivej:   
	preprocessor_declaration
	| typeDeclaration 
    ;
 
 preprocessor_declaration:
	'@import' file=file_specification
	;
	
 file_specification: 
 	('<'|'"')(ID ('/' | '\\' | '.')?)+ ('>' | '"')
 ;
 
typeDeclaration:
	class_implementation
	| category_implementation
;

class_implementation:
	'@implementation' 
	(
		name=class_name ( ':' superclass_name=superclass_name )
		( instance_variables )?
	)
	'@end';

category_implementation:
	'@implementation'
	(
		name=class_name '(' category_name=category_name ')' 
		( instance_variables=instance_variables )?
	)
	'@end';
	
class_name: 
	ID
	;
	
superclass_name:
	ID
	;

category_name:
	ID
	;
	
instance_variables:
	'{' instance_variable_declaration '}';

instance_variable_declaration:	
	ANY_OTHER
	;


p.s: if i read the word preprocessor in conjunction with Xtext it is time to run away as fast as i can.

besides this you dont seem to care about Xtext naming conventions.

~Christian


Re: Problem with simple grammar [message #871427 is a reply to message #871410] Sun, 13 May 2012 16:06 Go to previous messageGo to next message
smartmobili Missing name is currently offline smartmobili Missing name
Messages: 5
Registered: July 2009
Junior Member
Thanks Christian you are helping a lot.
Re: Problem with simple grammar [message #871435 is a reply to message #871427] Sun, 13 May 2012 17:06 Go to previous messageGo to next message
smartmobili Missing name is currently offline smartmobili Missing name
Messages: 5
Registered: July 2009
Junior Member
Someone else has an idéal about how to fix it?An example is better than 100 words
Thanks
Re: Problem with simple grammar [message #871437 is a reply to message #871410] Sun, 13 May 2012 17:28 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik Lindberg
Messages: 2500
Registered: July 2009
Senior Member
See below...

> compilation_unit:
> objj+=objectivej*
> ;
>
> objectivej: preprocessor_declaration
> | typeDeclaration ;
>
> preprocessor_declaration:
> '@import' file=file_specification
> ;
>
> file_specification: ('<'|'"')(ID ('/' | '\\' | '.')?)+ ('>' | '"')
> ;
>
> typeDeclaration:
> class_implementation
> | category_implementation
> ;
>
> class_implementation:
> '@implementation' (
Ambiguous - the category_implementation also starts with the token
'@implementation'

> name=class_name ( ':' superclass_name=superclass_name )
> ( instance_variables )?
you probably want to assign the result of instance_variables
e.g. instance_variables = instance_variables.
> )
> '@end';
>
> category_implementation:
> '@implementation'
Ambiguous - clashes with class_implementation

> (
> name=class_name '(' category_name=category_name ')' (
> instance_variables=instance_variables )?
> )
> '@end';
>
> class_name: ID
> ;
>
> superclass_name:
> ID
> ;
>
> category_name:
> ID
> ;
>
The previous three rules don't do much for you - you can simply assign
ID directly instead of having a DataRule that is equal to a terminal. It
would make sense if you have data type conversion associated with the
data rules.

> instance_variables:
> '{' instance_variable_declaration '}';
>
> instance_variable_declaration:
> ANY_OTHER
> ;
>
This looks odd - is "instance_variables" really just one character
(ANY_OTHER) enclosed in braces? You probably meant something else. Also,
whatever result that is parsed, you probably want to assign it to
something. e.g.

instance_variables :
'{' instance_variable_declrations += instance_variable_declaration+ '}'
;
instance_variable_declaration : ANY_OTHER+ ;

But that is just a wild guess, and kind of odd, as ANY_OTHER means non
INT, ID, STRING, keyword. If you need help here, you need to describe
how it is supposed to work.

Hope that helps you a bit.
Regards
- henrik
Re: Problem with simple grammar [message #871463 is a reply to message #871437] Sun, 13 May 2012 22:26 Go to previous messageGo to next message
smartmobili Missing name is currently offline smartmobili Missing name
Messages: 5
Registered: July 2009
Junior Member
Ok so actually I think it's not as easy as I thought, I might need to spend years to learn this new syntax while I was just get used to antlr.
I rewrote the grammar from scratch and now I am starting with this :

Objectivej:
  (types+=Type)*;

Type:
  ImportDecl 
  //| TypeDecl
  ;

ImportDecl:
	"@import" name=FileSpecification
	;

FileSpecification: 
 	('<'|'"')(ID ('/' | '\\' | '.')?)+ ('>' | '"')
 	;


But when I try it doesn't recognize the import using " but only the ones with <> :

@import <Foundation/CPObject.j> //OK
@import <AppKit/CPView.j> // OK
@import "MyClass.j" // Error : no viable input

And where can I find some real language grammar (c, java, c#, pascal, basic, ...) ?

[Updated on: Sun, 13 May 2012 22:29]

Report message to a moderator

Re: Problem with simple grammar [message #871467 is a reply to message #871463] Mon, 14 May 2012 00:15 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik Lindberg
Messages: 2500
Registered: July 2009
Senior Member
You need to post the complete grammar with all declarations in order for
anyone to help with what is going on.

My guess here is that you use the default terminals, and it has a
definition of String that will eat the double quote.

So...
FileSpecification : STRING | ('<' (ID ('/' | '\\' | '.')?)+ '>') ;

Would do the trick - you have to check that the FileSpecification is
valid though. (Simply add a @Check method in your validation class).

Good move to start with a really small grammar... makes it much easier
to understand where problems come from.

Regards
- henrik

On 2012-14-05 24:26, smartmobili Mising name wrote:
> Ok so actually I think it's not as easy as I thought, I might need to
> spend years to learn this new syntax while I was just get used to antlr.
> I rewrote the grammar from scratch and now I am starting with this :
>
>
> Objectivej:
> (types+=Type)*;
>
> Type:
> ImportDecl //| TypeDecl
> ;
>
> ImportDecl:
> "@import" name=FileSpecification
> ;
>
> FileSpecification: ('<'|'"')(ID ('/' | '\\' | '.')?)+ ('>' | '"')
> ;
>
>
> But when I try it doesn't recognize the import using " instead <> :
>
> @import <Foundation/CPObject.j> //OK
> @import <AppKit/CPView.j> // OK
> @import "MyClass.j" // Error : no viable input
>
>
>
Re: Problem with simple grammar [message #915888 is a reply to message #871467] Tue, 18 September 2012 06:52 Go to previous message
Vedanarayanan Ganesan is currently offline Vedanarayanan Ganesan
Messages: 1
Registered: September 2012
Junior Member
The terminals are to come from lexer which is normally tightly coupled with parser. So parser is a simple crawler on a state-machine (FA). SM is created by parser generators (see the tables generated by yacc for a grammar). For an SR parser, during the S (shift) the parser wants a token and that is provided by lexer. So parser does not know how to read the file or tokenize - it is the job of lexer and pre-processor (if applicable for the language) to supply the terminal tokens (in the form of an integer - for eg IDENTIFIER means 10, while means 11 ...) during the shift operation of the crawler. Normally the lex spec files ends with .l where you have to specify the regular grammar for your lexer so that the file is tokenized as per the lexical specs of the language in question.

Crawler gets into issues, when conflicts happen as crawler simply goes by the state tables (normally there would be 3 tables, shift-states, reduce states, reduce-maps - if my memory is right !). So the parser generator has to produce the tables in the right way for the language in question. For that you have to have unambiguous grammar. Normally SR conflict is resolved by taking the S path, RR is resolved by the 1st production of the grammar among the applicable productions for top of the state-stack.

The theory is simple discrete mathematics. If you have interest you can learn much faster.

Regards
Veda
Previous Topic:How to get objects from a resource from within the ProposalProvider ?
Next Topic:15 Minute Tutorial Extended
Goto Forum:
  


Current Time: Fri Oct 31 11:42:02 GMT 2014

Powered by FUDForum. Page generated in 0.02874 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software