Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Lexer too big(Stuck with "specialStateTransition() code exceeds 65535 byte limit" error)
Lexer too big [message #778653] Fri, 13 January 2012 16:14 Go to next message
Hannes F is currently offline Hannes FFriend
Messages: 7
Registered: November 2011
Junior Member
Hello!

I have a problem with the lexer generated by ANTLR.
As of my understanding, this is related to the terminals only so I'm posting all my terminal declarations here:
// some basic fragments
terminal fragment ct : 'T';
terminal fragment digit: '0'..'9';
terminal fragment letter: 'a'..'z' | 'A'..'Z';
terminal fragment us: '_';

// white space
terminal WS: (' '|'\t'|'\r'|'\n')+;

// numbers
terminal INT returns ecore::EInt: ('0'..'9')+;
terminal exponent: ('e' | 'E') ('+' | '-')? INT;
terminal number_literal : (INT ('.' INT?)? exponent?) | ('.' INT exponent?);

// id and mlmname
terminal ID : letter (letter | us | digit)*;
terminal trm_mlmname_text : letter (letter | digit | us | '-' | '.')*;

// strings
terminal fragment simplestring: !'"';
terminal string_literal : '"' simplestring* ('""' simplestring+)* '"';

// iso date/datetime
terminal fragment fractional_seconds: ('.' INT)?;
terminal fragment time_zone: ('Z' | 'z' 
    | '+' digit digit ':' digit digit 
    | '-' digit digit ':' digit digit)?
;
terminal iso_date :  digit digit digit digit '-' digit digit '-' digit digit;
terminal iso_date_time :  iso_date ct digit digit ':' digit digit ':' digit digit fractional_seconds time_zone;

// from bracket to bracket
terminal data_mapping :  '{' -> '}';

// not two semicolons in a row
terminal trm_ttext: (!';')+ (';' (!';')+)*;

// comments
terminal ML_COMMENT : '/*' -> '*/';
terminal SL_COMMENT : '//' !('\n'|'\r')* ('\r'? '\n')?;

(Note that I'm not including any other definitions. The first line of the grammar is:
grammar arden.xtext.ArdenSyntax hidden(WS, ML_COMMENT, SL_COMMENT))

Using these terminals, the method public int specialStateTransition(int s, IntStream _input) in InternalArdenSyntaxLexer.java exceeds the 65535 byte Java Bytecode limit when compiled. It contains the state machine for the lexer.

I don't know which terminal I should change first.The method specialStateTransition() itself consists of one large case statement that gives no hints on which rule is the origin of the problem.
And also I think I need the terminals the way I've declared them.

Do you have an idea what I could do?
Any help would be appreciated!

Regards,
Hannes
Re: Lexer too big [message #778690 is a reply to message #778653] Fri, 13 January 2012 16:35 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14665
Registered: July 2009
Senior Member
Hi,

did you enable

options = {
  classSplitting=true
}


in the parser.antlr.XtextAntlrGeneratorFragment
and in the parser.antlr.XtextAntlrUiGeneratorFragment

~Christian


Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Lexer too big [message #780536 is a reply to message #778653] Wed, 18 January 2012 14:05 Go to previous message
Hannes F is currently offline Hannes FFriend
Messages: 7
Registered: November 2011
Junior Member
Thanks, Christian!

That was indeed the problem.
I put "classsplitting = true" in the XtextAntlrGeneratorFragment but I forgot to put it in the XtextAntlrUiGeneratorFragment.
Now it does compile without errors.

Regards,
Hannes
Previous Topic:[Xtext] Signature based crosslinks
Next Topic:content assist for the certain position
Goto Forum:
  


Current Time: Thu Apr 25 13:48:23 GMT 2024

Powered by FUDForum. Page generated in 0.02948 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top