Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » problem for define dsl
problem for define dsl [message #908964] Thu, 06 September 2012 10:04 Go to next message
Laigle jérôme is currently offline Laigle jérôme
Messages: 26
Registered: August 2012
Junior Member
Hi,

I'm working on a DSL but i have a problem with my grammar

I want to define for example:
dcl pc%
print=1: tabv(1),(W10),(*(*)N6Z4),(Z2.Z2),(Z8.Z5,Z3,Z5),pc

So, I've started with the following construct:

CompilationUnit:
abalElements+=AbalElement*;

AbalElement:
Print|Declaration|Etiquette|Constant ;
Etiquette /*returns EtiquetteSymbol*/:
'&'name=ID
;
Print:
'print='numLog=INT ('&'etiquette=[Etiquette])?':'(datas+=Data3 ((',') datas+=Data3)* )?
;
Data3:
Expression|Directive2|Format2
;
Directive2:
Atb|Bell|Clear|Page|Home|Paint|Tab|Tabv
;
Format2:
'('inputField2=InputField2')'
;
InputField2:
CommandeID2|CommandeQualifiedNameComma|CommandeQualifiedName|CommandeIdComma
;
CommandeID2:
{CommandeID2} commande+=('E'|'U'|'W'|'Z'|'N'|'O'|'L'|'R'|'$'|'C'|'I'|'H'|'+'|'-'|'.'|'V'|'S'|'*'|'X'|'T'|'/'|'/0'|'1'|'2'|'3'|'4'|'5'|'6'|'7'|'8'|'9')*
;

Tabv:'tabv''('lignes=INT')'//('='Variable=[Symbol])?
;
Tab: 'tab''('col=INT (','lig=INT)?')'//('='Variable=[Symbol])?
;
Paint:'paint' '('text=INT (','background=INT)?')'//('='Variable=[Symbol])?
;
Home:{Home}'home'//('='Variable=[Symbol])?
;
Clear:
{Clear}'clear'//('='Variable=[Symbol])?
;
Page:
{Page}'page'//('='Variable=[Symbol])?
;
Bell:
{Bell}'bell'//('='Variable=[Symbol])?
;
Atb:
'atb''('attribut=INT')'//('='Variable=[Symbol])?
;

CommandeQualifiedNameComma:
commande=QualifiedNameDotComma
;
CommandeQualifiedName:
commande=QualifiedNameDot
;
CommandeIdComma:
commande=QualifiedNameComma
;
Declaration:
PointerDeclaration|VariableDeclaration|PointerProc//|VariableDeclarationProc//|ProcPointer
;
PointerDeclaration returns Symbol:
'ptr'name = ID type=Type /*( | ('like' variable=[Symbol |Type])) */
;

PointerProc returns Symbol:
'ptr proc' name = ID
;

VariableDeclaration returns Symbol:
'dcl' name = ID type=Type/*( | ('like' variable=[VariableDeclaration |Type]))*/
('endloc')?('[,S]')?
;
Constant returns Symbol:
'const' variable=VariableDeclaration
;
Type:
ShortNumericType | NumericType | StringType |BCDType
//|PointerProc
;

ShortNumericType:
{ShortNumericType}'#';

NumericType:
{NumericType}'%';


StringType:
{StringType}'$'('='size=INT)?
;

BCDType:
{BCDType}('='size=INT)?;
Expression :
CompareOperator;

CompareOperator returns Expression:
ConditionalOR(( {Equality2.left=current} '='|{Equality.left=current}'=='|{NoInclusion.left=current}'!='|{Different.left=current}'<>'|{Inferrior.left=current}'<'|{Superior.left=current}'>'|{SuperiorOrEqual.left=current}'>='|{InferriorOrEqual.left=current}'<=') right=ConditionalOR)*;

ConditionalOR returns Expression:
Addition (({ConditionalOR.left=current} 'or'|{ConditionalOX.left=current} 'ox'|{ConditionalAND.left=current} 'and'|{ConditionalNOT.left=current} 'not') right=Addition)*;
Addition returns Expression:
Multiplication (({Plus.left=current} '+' | {Minus.left=current} '-') right=Multiplication)*;

Multiplication returns Expression:
//CallExpression (({Multi.left=current} '*' | {Div.left=current} '/') right=CallExpression)*;
PrimaryExpression (({Multi.left=current} '*' | {Div.left=current} '/') right=PrimaryExpression)*;

//CallExpression returns Expression :
//PrimaryExpression (({Call.funcExpr=current} procedure=[Procedure] /*','*/) right=PrimaryExpression )*;
//PrimaryExpression (({Call.callExpr=current} symbol=[Symbol] /*procedure =[Procedure]*/ ('('argument+=[VariableDeclaration] (',' argument+=[VariableDeclaration])* ')')? ) right=PrimaryExpression )*;

PrimaryExpression returns Expression:
NumericInt|NumericReal|StringLiteral// |ExplicitCall
//| VariableOperande
|SymbolRef
| '(' Expression ')'
;

//ExplicitCall returns Call : {Call} explicitKeyword ?= 'call' procedure=[Procedure] ('('argument+=[VariableDeclaration] (',' argument+=[VariableDeclaration])* ')')?;

SymbolRef:
CallOrVariableSymbolRef|PauseSymbolRef|AliasSymbolRef|LikeSymbolRef
//|ProcPointer//|ExternEventCall
;

CallOrVariableSymbolRef:
('call')? symbol=[Symbol|ID] ("(" (arguments+=Expression)? ( "," arguments+=Expression)* ")")?(':erreur')?
;

PauseSymbolRef:
'pause' symbol=[Symbol|ID]//|Constant
;
AliasSymbolRef:
'alias' symbol=[Symbol|ID]
;
LikeSymbolRef:
'like' variable=[Symbol |ID]
;
StringLiteral:
stringValue=STRING
;
NumericInt:
value=INT;

NumericReal:
value=REAL;
terminal INT returns ecore::EInt: ('0'..'9')+;
terminal REAL returns ecore::EDouble : INT '.' INT;
//QualifiedName2 hidden(): ('E'|'U'|'W'|'Z'|'N'|'O'|'L'|'R'|'C'|'I'|'H'|'V'|'S'|'X'|'T'|'$'|'0'|'1'|'2'|'3'|'4'|'5'|'6'|'7'|'8'|'9')*;
terminal ID : ('a'..'z'|'A'..'Z'|'$'|'_') ('a'..'z'|'A'..'Z'|'$'|'_'|'0'..'9')*;



QualifiedName hidden(): ID '.' ID;
QualifiedName2 hidden():ID ('?' ID)+;
QualifiedNameDot hidden(): ID ('.' ID)+;
QualifiedNameComma hidden(): ID (',' ID)+;
QualifiedNameDotComma hidden(): QualifiedNameDot (','QualifiedNameDot)+;
terminal STRING :
'"' ( '\\' ('b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\') | !('\\'|'"') )* '"' |
"'" ( '\\' ('b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\') | !('\\'|"'") )* "'"
;



terminal SL_COMMENT : (';') !('\n'|'\r')* ('\r'? '\n')?;

terminal WS : (' '|'\t'|'\r'|'\n')+;

terminal ANY_OTHER: .;


But when i defined something like this i have problem
I need to put space between command but it's not that i want
And i can't define ID like E or U or...T
the same problem with INT i can't define a int value like 1 or 2 or...9
I don't know what is the solution
Maybe somebody has an idea or can point me into the right direction? Thanks in advance!

Best regards,
Jérôme
Report message to a moderator
Re: problem for define dsl [message #911879 is a reply to message #908964] Wed, 12 September 2012 13:54 Go to previous messageGo to next message
Laigle jérôme is currently offline Laigle jérôme
Messages: 26
Registered: August 2012
Junior Member
Other example

My dsl:
CompilationUnit:
abalElements+=AbalElement*;

AbalElement:
Print|Command
;

Print:
'print='numLog=INT (label=[Label])?
;

Command:
{Command} 'commandKeyword->' command=('1'|'2'|'A'|'B')
;

Label :
'&'name=ID
;

terminal INT returns ecore::EInt: ('0'..'9')+;
terminal REAL returns ecore::EDouble : INT '.' INT;
terminal ID : ('a'..'z'|'A'..'Z'|'$'|'_') ('a'..'z'|'A'..'Z'|'$'|'_'|'0'..'9')*;


terminal STRING :
'"' ( '\\' ('b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\') | !('\\'|'"') )* '"' |
"'" ( '\\' ('b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\') | !('\\'|"'") )* "'"
;

terminal SL_COMMENT : (';') !('\n'|'\r')* ('\r'? '\n')?;

terminal WS : (' '|'\t'|'\r'|'\n')+;

terminal ANY_OTHER: .;

test with xtend:

@RunWith(typeof(XtextRunner))
@InjectWith(typeof(LanguageInjectorProvider))

class test{
@Inject extension ParseHelper<CompilationUnit>



@Test def void testAsk2() {
val compilationUnit =
'''
print=1234
'''.parse

Assert::assertTrue(compilationUnit.abalElements.size>0);

val abalElement = compilationUnit.abalElements.get(0)
val print = abalElement as Print
Assert::assertEquals(1234,print.numLog)//test is ok
}
@Test def void testAsk() {
val compilationUnit =
'''
print=1
'''.parse

Assert::assertTrue(compilationUnit.abalElements.size>0);

val abalElement = compilationUnit.abalElements.get(0)//assertion failed error
val print = abalElement as Print
Assert::assertEquals(1,print.numLog)
}

}
I can't reuse a command in int or id
i don't know how i can define
Maybe somebody has an idea or can point me into the right direction? Thanks in advance!

Best regards,
Jérôme
Re: problem for define dsl [message #911926 is a reply to message #911879] Wed, 12 September 2012 15:44 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik Lindberg
Messages: 2500
Registered: July 2009
Senior Member
The problems are probably caused by overlapping terminals. If you have a
keyword '1', then you can never have an INT that is 1.

Command : {Command} 'commandKeyword' '->' command = CommandType ;

CommandType: INT | ID ;

Then you associate a value converter for CommandType that issues errors
if the string it gets is not ('1'|'2'|'A'|'B').

Alternatively, you don't add a converter, and instead just validate that
the command string is ('1'|'2'|'A'|'B'). This way you can offer a quick
fix if the user entered say 'a'.

With the suggested scheme you will not however automatically get code
completion since the framework can not guess what is acceptable, so you
need to customize that.

Small note, it is better to use separate keywords; i.e. 'print' '=' than
running them together - esp if the string contains whitespace, or if an
operator is also a free standing operator. If you separate them, it also
means that whitespace is tolerated between the tokens unless you
instruct Xtext to not hide whitespace.

Hope that helps.

Regards
- henrik


On 2012-12-09 15:54, Laigle jérôme wrote:
> Other example
>
> My dsl:
> CompilationUnit:
> abalElements+=AbalElement*;
>
> AbalElement:
> Print|Command
> ;
>
> Print:
> 'print='numLog=INT (label=[Label])? ;
>
> Command:
> {Command} 'commandKeyword->' command=('1'|'2'|'A'|'B')
> ;
>
> Label :
> '&'name=ID
> ;
>
> terminal INT returns ecore::EInt: ('0'..'9')+;
> terminal REAL returns ecore::EDouble : INT '.' INT;
> terminal ID : ('a'..'z'|'A'..'Z'|'$'|'_')
> ('a'..'z'|'A'..'Z'|'$'|'_'|'0'..'9')*;
>
>
> terminal STRING : '"' ( '\\'
> ('b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\') | !('\\'|'"') )* '"' |
> "'" ( '\\' ('b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\') |
> !('\\'|"'") )* "'"
> ;
> terminal SL_COMMENT : (';') !('\n'|'\r')* ('\r'? '\n')?;
>
> terminal WS : (' '|'\t'|'\r'|'\n')+;
>
> terminal ANY_OTHER: .;
>
> test with xtend:
>
> @RunWith(typeof(XtextRunner)) @InjectWith(typeof(LanguageInjectorProvider))
>
> class test{
> @Inject extension ParseHelper<CompilationUnit>
>
>
>
> @Test def void testAsk2() {
> val compilationUnit = '''
> print=1234
> '''.parse
>
> Assert::assertTrue(compilationUnit.abalElements.size>0);
>
> val abalElement = compilationUnit.abalElements.get(0)
> val print = abalElement as Print
> Assert::assertEquals(1234,print.numLog)//test is ok
> }
> @Test def void testAsk() {
> val compilationUnit = '''
> print=1
> '''.parse
>
> Assert::assertTrue(compilationUnit.abalElements.size>0);
>
> val abalElement =
> compilationUnit.abalElements.get(0)//assertion failed error
> val print = abalElement as Print
> Assert::assertEquals(1,print.numLog)
> }
>
> }
> I can't reuse a command in int or id
> i don't know how i can define
> Maybe somebody has an idea or can point me into the right direction?
> Thanks in advance!
>
> Best regards,
> Jérôme
Re: problem for define dsl [message #917839 is a reply to message #911926] Thu, 20 September 2012 13:39 Go to previous messageGo to next message
Laigle jérôme is currently offline Laigle jérôme
Messages: 26
Registered: August 2012
Junior Member
thank you for your help


i tried to do something to validate

i defined java validator like that:

public class LanguageJavaValidator extends AbstractLanguageJavaValidator {

String[] validCommands = {"E","Z"};

@Check
public void checkValidCommand(Command cmd) {
for(String commande:cmd.getCommandKey()){
if(!Arrays.asList(validCommands).contains(commande)){
warning("Invalid Command: "+commande, LanguagePackage.eINSTANCE.getCompilationUnit_AbalElements());
}
}

}
}


My dsl:
CompilationUnit:
abalElements+=AbalElement*;

AbalElement:
Print|Command
;

Print:
'print' '=' numLog=INT (label=[Label])?
;

Command:
{Command} 'commandKeyword' '->' commandKey+=ID
;

Label :
'&'name=ID
;


but when i realised test it's not ok

@Test def void testAsk3() {
val compilationUnit =
'''
commandKeyword-> AB
'''.parse

Assert::assertTrue(compilationUnit.abalElements.size>0);

val abalElement = compilationUnit.abalElements.get(0)
val commande = abalElement as Command
Assert::assertEquals("AB",commande.commandKey.get(0))
}

this test should be false but it's ok
i don't find many example for implement java validator maybe it's not a good defined
Best regards,
Jérôme
Re: problem for define dsl [message #918229 is a reply to message #917839] Thu, 20 September 2012 21:33 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik Lindberg
Messages: 2500
Registered: July 2009
Senior Member
On 2012-20-09 15:39, Laigle jérôme wrote:
> thank you for your help
>
>
> i tried to do something to validate
>
> i defined java validator like that:
>
> public class LanguageJavaValidator extends AbstractLanguageJavaValidator {
>
> String[] validCommands = {"E","Z"};
>
> @Check
> public void checkValidCommand(Command cmd) {
> for(String commande:cmd.getCommandKey()){
> if(!Arrays.asList(validCommands).contains(commande)){
> warning("Invalid Command: "+commande,
> LanguagePackage.eINSTANCE.getCompilationUnit_AbalElements());
> }
> }
>
> }
> }
>
>
> My dsl:
> CompilationUnit:
> abalElements+=AbalElement*;
>
> AbalElement:
> Print|Command
> ;
>
> Print:
> 'print' '=' numLog=INT (label=[Label])? ;
>
> Command:
> {Command} 'commandKeyword' '->' commandKey+=ID
> ;
>
> Label :
> '&'name=ID
> ;
>
>
> but when i realised test it's not ok
>
> @Test def void testAsk3() {
> val compilationUnit = '''
> commandKeyword-> AB
> '''.parse
>
> Assert::assertTrue(compilationUnit.abalElements.size>0);
>
> val abalElement = compilationUnit.abalElements.get(0)
> val commande = abalElement as Command
> Assert::assertEquals("AB",commande.commandKey.get(0))
> }
>
> this test should be false but it's ok
> i don't find many example for implement java validator maybe it's not a
> good defined Best regards,
> Jérôme

You need to check if there are errors. You got exactly what was parsed,
so naturally the thing you are checking contains "AB".

You need to look at how to assert that a warning is present.
It is only if there are syntax errors (can not parse this) that it will
fail on its own.

If it was not this way; you could not read the parsed (non validation
conforming input) in a semantic way, and it would be very difficult to
implement quick fixes etc.

Also, make sure your validation logic is invoked as you expect. (Set a
breakpoint, or do a printout or something).

Hope that helps.
- henrik
Re: problem for define dsl [message #921743 is a reply to message #918229] Mon, 24 September 2012 11:43 Go to previous messageGo to next message
Laigle jérôme is currently offline Laigle jérôme
Messages: 26
Registered: August 2012
Junior Member
thanks for the quick reply

i tried to parse warning
public class ParseHelper3<T extends EObject> extends ParseHelper<T> {

@Inject
private IResourceFactory resourceFactory;

@Inject
private Provider<XtextResourceSet> resourceSetProvider;

public List<Diagnostic> parseToWarning(InputStream in, URI uriToUse, Map<?, ?> options, ResourceSet resourceSet) {
Resource resource = resourceFactory.createResource(uriToUse);
resourceSet.getResources().add(resource);
try {
resource.load(in, options);
if (resource.getWarnings().size() > 0) {
return resource.getWarnings();
}
throw new IllegalStateException();
} catch (IOException e) {
throw new WrappedException(e);
}
}

public List<Diagnostic> parseToWarning(CharSequence text) throws Exception {
return parseToWarning(text, resourceSetProvider.get());
}

public List<Diagnostic> parseToWarning(CharSequence text, ResourceSet resourceSetToUse) throws Exception {
return parseToWarning(getAsStream(text), computeUnusedUri(resourceSetToUse), null, resourceSetToUse);
}

public List<Diagnostic> parseToWarning(CharSequence text, URI uriToUse, ResourceSet resourceSetToUse) throws Exception {
return parseToWarning(getAsStream(text), uriToUse, null, resourceSetToUse);
}

}

Java validator:
public class LanguageJavaValidator extends AbstractLanguageJavaValidator {

String[] validCommands = {"E","Z"};

@Check
public void checkValidCommand(Command cmd) {
for(String commande:cmd.getCommandKey()){
if(!Arrays.asList(validCommands).containsAll(Arrays.asList(commande))){
warning("Invalid Command: ", LanguagePackage.Literals.COMMAND__COMMAND_KEY);

//eINSTANCE.getCompilationUnit_AbalElements()
}
}

}
}

And my test :
class WarningTest {
@Inject extension ParseHelper3<CompilationUnit>
@Test def void testAsk3() {
val diagnostic =
parseToWarning(
'''
commandKeyword-> AB
''')

Assert::assertNotNull(diagnostic)
Assert::assertEquals(1,diagnostic.size)
}
}
@Test def void testAsk4() {
val diagnostic =
parseToWarning(
'''
commandKeyword-> E
''')

Assert::assertNotNull(diagnostic)
Assert::assertEquals(0,diagnostic.size)
}

but test failed
java.lang.NullPointerException
Maybe somebody has an idea or can point me into the right direction? Thanks in advance!

Best regards,
Jérôme

Re: problem for define dsl [message #921876 is a reply to message #921743] Mon, 24 September 2012 14:03 Go to previous message
Henrik Lindberg is currently offline Henrik Lindberg
Messages: 2500
Registered: July 2009
Senior Member
What I meant was that you need to assert that there are no warning or
errors in your unit test.

The errors and warnings you issue in your validator ends up as warnings
and errors in the Resource. How you assert these in the test depends on
how you have written your tests, if you are using the Xtext framework etc.

Regards
- henrik
Previous Topic:xText editor encoding when open and save file
Next Topic:Choose from one of multiple IJvmModelInferrer implementations at runtime
Goto Forum:
  


Current Time: Sun Oct 26 07:56:46 GMT 2014

Powered by FUDForum. Page generated in 0.11014 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software