Eclipse Community Forums: TMF (Xtext) » Problems with terminal rules...

Help

Home

Home » Modeling » TMF (Xtext) » Problems with terminal rules...

Show: Today's Messages :: Show Polls :: Message Navigator

Problems with terminal rules... [message #634312]

Thu, 21 October 2010 09:08

Fran Blanco

Messages: 20
Registered: October 2010

Junior Member

Hi,

I want to create a DSL and i am having some problems...

I have some kind of variables:

INT --> +- 0...9
DEC ---> +- 0...9 . 0...9
LGL ---> '0' or '1'
CHAR --> A....Z 0....9 eg. YESNO or '1234' or 'YESNO'

I have the problem with CHAR variable. I can introduce this kind of variable of different ways:

YESNO
'YESNO'
'ASDD1234'
ASD1ASD

between quotes or not...

Here is my Code:

Quote:

grammar org.xtext.prueba with org.eclipse.xtext.common.Terminals
generate prueba "http://www.xtext.org/prueba"
Model:
CLStatementInicial;

terminal ENTEROS:
('-')? ('0'..'9')+;
terminal DECIMAL:
(ENTEROS) ('.')? (ENTEROS)?;
terminal GENERAL:
('A'..'Z') ('A'..'Z' | '0'..'9')*;
terminal PARAMETRO:
('&') ('A'..'Z') ('A'..'Z' | '0'..'9')*;
terminal PROCEDURE:
('A'..'Z' | '0'..'9' | '_')+;
terminal DIRECTORIO:
('A'..'Z' | '0'..'9' | '/' | '_')+;
terminal CHAR: //Espacios da ERROR
"'"? ('A'..'Z' | '0'..'9' | '/' | '_' | '.')+ "'"?;

enum LISTATIPOSPARAMETRO:
CHAR = '*CHAR' |
DEC = '*DEC' |
LGL = '*LGL' |
INTEGER = '*INT' |
UINT = '*UINT';

//Definición estructura Valores...
NumeroINT:
Value=(ENTEROS);
NumeroDEC:
Value=(DECIMAL);
NumeroCHAR:
Value=(CHAR);
NumeroLGL:
Value=("'0'" | "'1'");
Valor:
NumeroINT |
NumeroDEC |
NumeroCHAR |
NumeroLGL;

CLStatementInicial:
DCL:
'DCL'
('VAR'? '('? (parametro=PARAMETRO) ')'?)
('TYPE'? '('? (listatipos=LISTATIPOSPARAMETRO) ')'?)
('LEN'? '('? (length=ENTEROS) (length2=ENTEROS)? ')'?)?
(valor1=NumeroLGL |('VALUE' '(' (valor=Valor) ')'))?

When I introduce a CHAR with quotes is fine, but if i dont do it, doesnt work....

What could it be the correct structure of terminal rules?
Im so lost....

Thanks you..!

B

Report message to a moderator

Re: Problems with terminal rules... [message #634330 is a reply to message #634312]

Thu, 21 October 2010 09:52

Jan Koehnlein

Messages: 760
Registered: July 2009
Location: Hamburg

Senior Member

Your terminal rules overlap, e.g. "'1'" will match LGL as well as CHAR,
"0" will match INT as well ad DEC. As the internal Antlr lexer is
context-less this can yield very strange behaviour, as the first
matching terminal rule wins. It's generally a better idea to use a
minumum set of terminal rules and cover the rest in datatype rules,
which are evaluated by the parser thus dealing with lookahead etc, e.g.

Int returns ecore::EInt :
(-|+)? INT;

Dec:
(-|+)? INT '.' INT?

terminal INT:
('0'..'9')+;

Into the bargain, your character rule matches 'a as well as a' or 'a'.
You should rather use an alternative covering either quotes on both ends
or none, as we do in the STRING rule in
/org.eclipse.xtext/src/org/eclipse/xtext/common/Terminals.xt ext

Am 21.10.10 11:08, schrieb Fran Blanco:
> Hi,
>
> I want to create a DSL and i am having some problems...
>
> I have some kind of variables:
>
> INT --> +- 0...9
> DEC ---> +- 0...9 . 0...9
> LGL ---> '0' or '1'
> CHAR --> A....Z 0....9 eg. YESNO or '1234' or 'YESNO'
>
> I have the problem with CHAR variable. I can introduce this kind of
> variable of different ways:
>
> YESNO
> 'YESNO'
> 'ASDD1234'
> ASD1ASD
>
> between quotes or not...
>
> Here is my Code:
>
> Quote:
>> grammar org.xtext.prueba with org.eclipse.xtext.common.Terminals
>> generate prueba "http://www.xtext.org/prueba"
>> Model:
>> CLStatementInicial;
>>
>> terminal ENTEROS:
>> ('-')? ('0'..'9')+;
>> terminal DECIMAL:
>> (ENTEROS) ('.')? (ENTEROS)?;
>> terminal GENERAL:
>> ('A'..'Z') ('A'..'Z' | '0'..'9')*;
>> terminal PARAMETRO:
>> ('&') ('A'..'Z') ('A'..'Z' | '0'..'9')*;
>> terminal PROCEDURE:
>> ('A'..'Z' | '0'..'9' | '_')+;
>> terminal DIRECTORIO:
>> ('A'..'Z' | '0'..'9' | '/' | '_')+;
>> terminal CHAR: //Espacios da ERROR
>> "'"? ('A'..'Z' | '0'..'9' | '/' | '_' | '.')+ "'"?;
>>
>>
>>
>> enum LISTATIPOSPARAMETRO:
>> CHAR = '*CHAR' |
>> DEC = '*DEC' |
>> LGL = '*LGL' |
>> INTEGER = '*INT' |
>> UINT = '*UINT';
>>
>> //Definición estructura Valores...
>> NumeroINT:
>> Value=(ENTEROS);
>> NumeroDEC:
>> Value=(DECIMAL);
>> NumeroCHAR:
>> Value=(CHAR);
>> NumeroLGL:
>> Value=("'0'" | "'1'");
>> Valor:
>> NumeroINT |
>> NumeroDEC |
>> NumeroCHAR |
>> NumeroLGL;
>>
>> CLStatementInicial:
>> DCL: 'DCL' ('VAR'? '('? (parametro=PARAMETRO) ')'?)
>> ('TYPE'? '('? (listatipos=LISTATIPOSPARAMETRO) ')'?)
>> ('LEN'? '('? (length=ENTEROS) (length2=ENTEROS)? ')'?)?
>> (valor1=NumeroLGL |('VALUE' '(' (valor=Valor) ')'))?
>
>
> When I introduce a CHAR with quotes is fine, but if i dont do it, doesnt
> work....
>
>
> What could it be the correct structure of terminal rules?
> Im so lost....
>
> Thanks you..!
>
> B

--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com

---
Get professional support from the Xtext committers at www.typefox.io

Report message to a moderator

Re: Problems with terminal rules... [message #634549 is a reply to message #634312]

Fri, 22 October 2010 07:39

Fran Blanco

Messages: 20
Registered: October 2010

Junior Member

Hi Jan,

Thank you for ur reply! I think i have got the solution of INT and DEC problem... Here we go:

Model:
		'PGM' 
	('PARM' '(' (parametros+=PARAMETRO)+ ')' )?
	(OtrosStatements+=Statements)*
	'ENDPGM';


terminal PARAMETRO:
('&') ('A'..'Z') ('A'..'Z' | '0'..'9')*;

DATAINT:
('-'|'+')? INT;
DATADEC:
('-'|'+')? INT '.' INT;

Valor:
DATAINT |
DATADEC;


enum LISTATIPOSPARAMETRO:
CHAR = '*CHAR' |
DEC = '*DEC' |
LGL = '*LGL' |
INTEGER = '*INT' |
UINT = '*UINT';

Statements:

	'DCL' 
	('VAR'? '('? (parametro=PARAMETRO) ')'?)
	('TYPE'? '('? (listatipos=LISTATIPOSPARAMETRO) ')'?)
	('LEN'? '('? (INT) (INT)? ')'?)?
	('VALUE' '(' (Valor) ')')?

What do you think? Its right?

After that, CHAR variable must have uppercases and numbers only. For example:

YESNO
YES123NO
'YESNO'
'YES123NO'

I m trying to put together INT rule with a uppercase letter rule, but RULE_INT fails... i have tried use a datatype too but I havent got it.. I dont know how to do it..!

Quotes issue i havent tried it yet... :S

But I have looked at terminal STRING and i dont understand the structure:

terminal STRING :
'"' ( '\\' ('b'|'t'|'n'|'f'|'r'|'"'|"'"|'\\') | !('\\'|'"') )* '"' |
"'" ( '\\' ('b'|'t'|'n'|'f'|'r'|'"'|"'"|'\\') | !('\\'|"'") )* "'";

But i could do a datatype like that:

DATACHAR2:
("'") DATACHAR ("'") |
DATACHAR;

Thank you so much again.

B

Edit:

I am trying right now and i cannot do it.

terminal UPPER:
"'" ('A'..'Z' | '0'..'9')+ "'" |
('A'..'Z' | '0'..'9')+;

The following token definitions are unreachable: RULE_INT..

[Updated on: Fri, 22 October 2010 08:18]

Report message to a moderator

Re: Problems with terminal rules... [message #634562 is a reply to message #634549]

Fri, 22 October 2010 08:53

Meinte Boersma

Messages: 434
Registered: July 2009
Location: Leiden, Netherlands

Senior Member

Yup, UPPER will consume "1234" (without the single quotes) as well, so it shadows INT. Look at the default terminals' definitions to see what the pattern looks like for things like

'1234'
A1234

etc.

Xtext blogs: executable models...again? | workshop material | custom scoping with Xtend

Report message to a moderator

Re: Problems with terminal rules... [message #634567 is a reply to message #634312]

Fri, 22 October 2010 09:25

Fran Blanco

Messages: 20
Registered: October 2010

Junior Member

Hi!

The first one, i can write it with a STRING.
The second one with ID, but I couldnt use ID rule if i would want to write 1ASD23 cuz it needs to start with a letter...

Report message to a moderator

Re: Problems with terminal rules... [message #634926 is a reply to message #634312]

Mon, 25 October 2010 07:30

Fran Blanco

Messages: 20
Registered: October 2010

Junior Member

I have been thinkin about it in the weekend and I havent got anything... I dont get to define a datatype or rule with UPPERCASE and numbers :S

I am going on with my DSL structure but i need to define this rule as soon as possible...

Any help??

Thz

B

Report message to a moderator

Re: Problems with terminal rules... [message #635078 is a reply to message #634926]

Mon, 25 October 2010 15:17

Alexander Nittka

Messages: 1193
Registered: July 2009

Senior Member

Hi,

my general comment is: Make terminal rules non-overlapping and define as few terminals as possible, overwrite the default terminals (only) where it is sensible to do so (you may have to adapt the value converters then). Try to use datatype rules as much as possible (if you don't specify a return type the string that is read from the model is stored). You can add value converters throwing exceptions if the format (e.g. for LGL) incorrect.

That is use
Lgl returns <sensibledatatype>: INT;

and in the value converter test that the INT is in fact only 0 or 1.

Write tests for terminal and datatype rules (see http://blogs.itemis.de/stundzig/archives/726 for inspiration).

Often it is easier not to put everything in the grammar (terminal rules) but have semantic validation (allowing for user friendly error messages).

Basically everything of the following rules should be datatype rules rather than terminal rules.
>> terminal ENTEROS:
>> ('-')? ('0'..'9')+;
>> terminal DECIMAL:
>> (ENTEROS) ('.')? (ENTEROS)?;
>> terminal GENERAL:
>> ('A'..'Z') ('A'..'Z' | '0'..'9')*;
>> terminal PARAMETRO:
>> ('&') ('A'..'Z') ('A'..'Z' | '0'..'9')*;
>> terminal PROCEDURE:
>> ('A'..'Z' | '0'..'9' | '_')+;
>> terminal DIRECTORIO:
>> ('A'..'Z' | '0'..'9' | '/' | '_')+;
>> terminal CHAR: //Espacios da ERROR
>> "'"? ('A'..'Z' | '0'..'9' | '/' | '_' | '.')+ "'"?;

Alex

Report message to a moderator