Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Problem with Multiline Comments(Curious bug when multiline comments are being "captured" in a rule)
Problem with Multiline Comments [message #1763976] Tue, 23 May 2017 14:19 Go to next message
Veselin Drumev is currently offline Veselin DrumevFriend
Messages: 9
Registered: May 2017
Junior Member
Hello,

I am having the following problem: I need to make a "translator" that takes one script and generates another from it, so I decided I will use xtext for that. I created a grammar and everything and it is - OK - the translator works. My problems come when i need to move the comments from the old skripts to the new ones. For that purpose i am using the following:
Quote:

/* Comments */
Comments:
(SingleLine_Comment | MultiLine_Comment)
;
SingleLine_Comment:
{SingleLine_Comment}
commentLines+=SL_COMMENT*
;
MultiLine_Comment:
{MultiLine_Comment}
commentText+=ML_COMMENT*
;

The problem is that when i remove the comments from the hidden section (i just set the WS to hidden for my grammar) I get an error for parsing the multi-line comments. The single line comments are collected correctly and if i add ML_COMMENT to the hidden section the scripts are actually parsed without errors.

I have set the parser option for backtracking to true due to requirements in my grammar.

Can you please help me to find where is the problem and perhaps fix it so that i collect all type of comments correctly.

Thank you,
Veselin
Re: Problem with Multiline Comments [message #1764078 is a reply to message #1763976] Wed, 24 May 2017 14:58 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12254
Registered: July 2009
Senior Member
can you share a complete minimal reproducing sample grammar.
i am not sure where you call these comment rules, where and how you changed the hidden, and why you do backtracking (which might hide other errors btw)


Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Problem with Multiline Comments [message #1764082 is a reply to message #1764078] Wed, 24 May 2017 15:06 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12254
Registered: July 2009
Senior Member
p.s.

you know that the hidden comments are inside the node model so you may simply traverse that one to extract them.
(a more complex version of MultiLineCommentDocumentationProvider


Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Problem with Multiline Comments [message #1764172 is a reply to message #1764082] Fri, 26 May 2017 10:47 Go to previous messageGo to next message
Veselin Drumev is currently offline Veselin DrumevFriend
Messages: 9
Registered: May 2017
Junior Member
Hello,

It will be hard for me to provide you with the reproducing example since I cannot share a lot about the project.

I do understand that there is a way to read the hidden comments from the node model but can you give me an example how to find a node from the current object:

I have dispatchers for all grammar rules - so is there a way to get the node from the element somehow?

e.g.: def dispatch toText(Addition expr){.. getNodeFromElement(expr)?..}

Re: Problem with Multiline Comments [message #1764173 is a reply to message #1764082] Fri, 26 May 2017 10:48 Go to previous messageGo to next message
Veselin Drumev is currently offline Veselin DrumevFriend
Messages: 9
Registered: May 2017
Junior Member
Hello,

It will be hard for me to provide you with the reproducing example since I cannot share a lot about the project.

I do understand that there is a way to read the hidden comments from the node model but can you give me an example how to find a node from the current object:

I have dispatchers for all grammar rules - so is there a way to get the node from the element somehow?

e.g.: def dispatch toText(Addition expr){.. getNodeFromElement(expr)?..}

Re: Problem with Multiline Comments [message #1764174 is a reply to message #1764082] Fri, 26 May 2017 10:48 Go to previous messageGo to next message
Veselin Drumev is currently offline Veselin DrumevFriend
Messages: 9
Registered: May 2017
Junior Member
Hello,

It will be hard for me to provide you with the reproducing example since I cannot share a lot about the project.

I do understand that there is a way to read the hidden comments from the node model but can you give me an example how to find a node from the current object:

I have dispatchers for all grammar rules - so is there a way to get the node from the element somehow?

e.g.: def dispatch toText(Addition expr){.. getNodeFromElement(expr)?..}

Re: Problem with Multiline Comments [message #1764175 is a reply to message #1764174] Fri, 26 May 2017 11:46 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12254
Registered: July 2009
Senior Member
Will give you an example somewhen next week. Have no computer to create on
But NodeModelUtils should have methods for that

(You will need to adapt this to your usecase anyway ....)


Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Fri, 26 May 2017 11:48]

Report message to a moderator

Re: Problem with Multiline Comments [message #1764176 is a reply to message #1764175] Fri, 26 May 2017 12:24 Go to previous messageGo to next message
Veselin Drumev is currently offline Veselin DrumevFriend
Messages: 9
Registered: May 2017
Junior Member
Thanks a lot.
I will look trough the Utils and see if I could manage by myself :).
Re: Problem with Multiline Comments [message #1764177 is a reply to message #1764176] Fri, 26 May 2017 12:28 Go to previous messageGo to next message
Veselin Drumev is currently offline Veselin DrumevFriend
Messages: 9
Registered: May 2017
Junior Member
Also had some internet problems that's why there are 3 posts of the same thing :(.
Re: Problem with Multiline Comments [message #1764471 is a reply to message #1764175] Tue, 30 May 2017 11:07 Go to previous messageGo to next message
Veselin Drumev is currently offline Veselin DrumevFriend
Messages: 9
Registered: May 2017
Junior Member
Ok so I am trying to work with MultiLineCommentDocumentationProvider and it is not working :( and I am getting some null exceptions. So here is my grammar ( a bit changed but it reproduces the problem ):

Quote:

grammar com.vd.internal.MyDsl with org.eclipse.xtext.common.Terminals hidden(WS)
generate myDsl "http://www.vd.com/internal/MyDsl"

/*** MODEL ***/
ScriptModel:
firstComment=Comments
includes=Includes
declarations+=FunctionVariableDeclaration*
mainFuncComment=Comments
mainFunction=MainFunction
functionDefinitions=FunctionDefinitions
lastComments=Comments
;
/* #SDBG */
SDBG:
'#SDBG'?
;
/* #INCLUDE_DELIMITER */
terminal INCLUDE_DELIMITER:
'#INCLUDE_DELIMITER'?
;
/*** Includes ***/
Includes:
{Includes}
comments += Comments
includesList+=Include*
;
Include:
'#' 'include' libName=STRING ';'?
;

/*** FunctionVariableDeclaration ****/
FunctionVariableDeclaration:
VariableDeclarationStatement
| FunctionDeclarationStatement
;
/* Function Declarations */
FunctionDeclarationStatement:
FunctionDeclaration ';'
;
FunctionDeclaration:
comments=Comments
funType=FunctionType funName=Identifier hasBrackets?='(' paramList=FunctionParamsList? ')'
;
FunctionParamsList:
{FunctionParamsList}
parameters+=FunctionParameter (',' parameters+=FunctionParameter)*
;
FunctionParameter:
typeName=VariableType paramName=Identifier
;
/* Variable Declarations */
VariableDeclarationStatements:
{VariableDeclarationStatements}
(stmts+=VariableDeclarationStatement)+
;
VariableDeclarationStatement:
comments=Comments
varDeclr=VariableDeclaration ';'
;
VariableDeclaration:
typeName=VariableType variables=ExpressionList
;
/*** Function Definitions ***/
/* Main Function */
MainFunction:
'void' 'main' '(' ')'
comments=Comments
block = Block
;
/* Other Functions */
FunctionDefinitions:
{FunctionDefinitions}
funDefs+=FunctionDefinition*
;
FunctionDefinition:
header=FunctionDeclaration
comments=Comments
body=Block
;
Block:
{Block}
'{'
(variables=VariableDeclarationStatements)?
blockStatements+=BlockStatement*
endingComments=Comments
'}'
;
BlockStatement:
statement=CommentedStatement
;
/*** STATEMENTS ***/
Statement:
Block
| Expression ';'
| WhileStatement
| ForStatement
| DoWhileStatement ';'
| ConditionalStatement
| SwitchStatement
| ReturnStatement ';'
// | ContinueStatement ';'
// | BreakStatement ';'
| ({Statement} ';')
;
CommentedStatement:
=>comments=Comments
stmt = Statement
;
ReturnStatement:
{ReturnStatement}
'return' expr=CommentedExpression?
;
//ContinueStatement:
// {ContinueStatement}
// 'continue' label=Identifier?
//;
//BreakStatement:
// {BreakStatement}
// 'break' label=Identifier?
//;
WhileStatement:
'while' '(' expr=CommentedExpression ')' statement=CommentedStatement
;
DoWhileStatement:
'do' block=Block 'while' '(' expr=CommentedExpression ')'
;
SwitchStatement:
'switch' '(' expr=CommentedExpression ')'
'{'
cases += CaseFragment*
deflt = DefaultFragment?
comments=Comments
'}'
;
CaseFragment:
comments=Comments
'case' expr=CommentedExpression ':' stmts+=CommentedStatement*
;
DefaultFragment:
{DefaultFragment}
comments=Comments
'default' ':' stmts+=CommentedStatement*
;
/*** EXPRESSIONS ***/
CommentedExpression:
comments=Comments
expr=Expression
;
Expression:
BinaryOperationExpression
;
BinaryOperationExpression:
CalculatedConditionalExpression ({BinaryOperationExpression.leftExpression=current} =>operand=BINARY_OPERANDS rightExpression=CalculatedConditionalExpression)*
;
terminal BINARY_OPERANDS:
('+'|'-'|'/'|'*'|'%'|'||'|'&&'|'^^'|'&'|'|'|'^'|'=='|'!='|'<='|'>='|'<'|'>'|'='|'+='|'-='|'/='|'*='|'%='|'&='|'|='|'^='|'<<'|'>>'|'<<='|'>>=')
;
CalculatedConditionalExpression:
PostfixExpression ({CalculatedConditionalExpression.condExpr=current}=>'?' trueExpr=PostfixExpression ':' falseExpr=PostfixExpression)?
;
PostfixExpression:
PrefixExpression ({PostfixExpression.expr=current} postfix+=SINGULAR_OPERANDS)*
;
PrefixExpression:
(prefix+=SINGULAR_OPERANDS)* expr=TypeCastExpression
;
TypeCastExpression:
(hasParenthesis?='(' typeCast=PRIMITIVE_TYPE ')')? expr=FunctionCall
;
FunctionCall:
UnitaryExpression ({FunctionCall.expr=current} hasParenthesis?='(' paramsList=ExpressionList? ')')?
;
UnitaryExpression:
Term | ({UnitaryExpression}prefix=UNARY_OPERANDS nextExpr=Term)
;
terminal SINGULAR_OPERANDS:
'++' | '--'
;
terminal UNARY_OPERANDS:
'~' | '!' | '+' | '-' | '*'
;
/* ID Rule */
Identifier:
idName=ID
;
Term:
atom=Atom (trailer+=Trailer)*
| hasParenthesis?='(' expr=CommentedExpression ')'
| array=ArrayInitializer
;
ArrayInitializer:
{ArrayInitializer}
hasCurlyBrackets?='{' exprList=ExpressionList '}'
;
Atom:
Identifier
| Literal
;
Trailer:
{Trailer}
hasBrackets?='[' (index=CommentedExpression)? ']'
;
ConditionalStatement:
'if' '(' expr=CommentedExpression ')' statement=CommentedStatement (comments=Comments =>hasElse?='else' elseStatement=CommentedStatement)?
;
ForStatement:
'for' '(' init=ExpressionList? ';' cond=CommentedExpression? ';' upd=ExpressionList? ')' stmt=CommentedStatement
;
ExpressionList:
expr+=CommentedExpression (',' expr+=CommentedExpression)*
;
/* Comments */
Comments:
(SingleLine_Comment | MultiLine_Comment)
;
SingleLine_Comment:
{SingleLine_Comment}
commentLines+=SL_COMMENT*
;
//terminal JAVADOC: '/**' -> '*/';
//@Override
//terminal ML_COMMENT: '/*'(!'*') -> '*/';
MultiLine_Comment:
{MultiLine_Comment}
commentText+=ML_COMMENT*
;
/* Types */
FunctionType:
PRIMITIVE_TYPE | 'void'
;

VariableType:
typeName=PRIMITIVE_TYPE
;
terminal PRIMITIVE_TYPE:
'int'
| 'double'
| 'string'
;
/* Literals */
Literal:
nullLit = NULL_LITERAL
| boolLit = BOOLEAN_LITERAL
| intVal = INT_LITERAL
| doubleVal = DOUBLE_LITERAL
| stringVal = STRING
;
///* Our STRING terminal */
//@Override
//terminal STRING: '"' .* '"';
/* Null Literal */
terminal NULL_LITERAL:
'NULL_int'
| 'NULL_double'
| 'NULL_string'
;
/* Boolean Literal */
terminal BOOLEAN_LITERAL:
'TRUE'
| 'FALSE'
;
/* Integer Literal */
terminal INT_LITERAL:
UNARY_MINUS? DECIMAL_NUMERAL
;
terminal fragment DECIMAL_NUMERAL:
'0'
| INT_NON_ZERO INT?
;
terminal fragment INT_NON_ZERO:
('1'..'9')
;
/* Floating-Point Literal */
terminal DOUBLE_LITERAL:
UNARY_MINUS? DECIMAL_NUMERAL '.' INT?
| '.' INT
| DECIMAL_NUMERAL
;
terminal fragment UNARY_MINUS:
'-'
;
//TERMINALS END


I made a simple test metod for the generator:
Quote:


def static String getElemComments(EObject elem){
var MultiLineCommentDocumentationProvider commentProvider = new MultiLineCommentDocumentationProvider();

var List<INode> commentList = commentProvider.getDocumentationNodes(elem);
println(commentList.join)
println("===========")

println(commentProvider.getDocumentation(elem));

return "";
}



when i call this method for the ScriptModel.mainMethod - it throws null exception :(( here : commentProvider.getDocumentationNodes(elem);
perhaps I am not doing it the right way but I cannot find more documentation for the Provider and so.

Thanks in advance for all the help you can provide :).
Re: Problem with Multiline Comments [message #1764472 is a reply to message #1764471] Tue, 30 May 2017 11:10 Go to previous messageGo to next message
Veselin Drumev is currently offline Veselin DrumevFriend
Messages: 9
Registered: May 2017
Junior Member
Simple input that should be parsed but gives errors:
Quote:
#include "test_test"
/**
* test
*/
void main(){
//test test
int bla;
/*
big Test
*/
int mm, pp = FALSE;

bla = (a && b) || c;
mm = TRUE;
}

Re: Problem with Multiline Comments [message #1764483 is a reply to message #1764472] Tue, 30 May 2017 12:39 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12254
Registered: July 2009
Senior Member
static and new is BAD. use @Inject

Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Problem with Multiline Comments [message #1764515 is a reply to message #1764483] Tue, 30 May 2017 17:49 Go to previous messageGo to next message
Veselin Drumev is currently offline Veselin DrumevFriend
Messages: 9
Registered: May 2017
Junior Member
Thanks but how?
Quote:


@Inject
var static MultiLineCommentDocumentationProvider commentProvider;
/**
* Extracts the comments for an EObject and returns them
*
* @param the EObject element to be searched for comments
*/
def static String getElemComments(EObject elem){
if(commentProvider===null) {println("COMMENT PROVIDER IS NULL"); return ""}
var List<INode> commentList = commentProvider.getDocumentationNodes(elem);
println(commentList.join)
println("===========")

println(commentProvider.getDocumentation(elem));

return "";
}



I receive the error message that the provider is null. I understand I am probably not doing something right but don't know what.
Could you please provide me with simple example on whatever grammar where this works? or something like that. Thank you in advance.
Re: Problem with Multiline Comments [message #1764516 is a reply to message #1764515] Tue, 30 May 2017 18:09 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12254
Registered: July 2009
Senior Member
class MyDslGenerator extends AbstractGenerator {

@Inject
MultiLineCommentDocumentationProvider p

override void doGenerate(Resource resource, IFileSystemAccess2 fsa, IGeneratorContext context) {
...
}
}


Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Problem with Multiline Comments [message #1778734 is a reply to message #1764516] Wed, 20 December 2017 15:16 Go to previous messageGo to next message
David  Riobó is currently offline David RiobóFriend
Messages: 9
Registered: August 2017
Junior Member
I've encountered the same problems as Veselin when trying to use this but I am still not able to make it work.

In my case, I am coding together an application to generate documentation artifacts (to be used for the end-users) from a specific dsl script.

For this matter, I want to add some comments to the model with a specific tag that will allow me to filter the elements that will be shown in the documentation, together with a description added there in the file.

I've written

@Inject
static MultiLineCommentDocumentationProvider mlcdp;




XtextXMLDocGeneratorLocal generator = new XtextXMLDocGeneratorLocal();
new org.eclipse.emf.mwe.utils.StandaloneSetup().setPlatformUri("../");
Injector injector = new MyDSLStandaloneSetup()
.createInjectorAndDoEMFRegistration();
XtextResourceSet resourceSet = injector
.getInstance(XtextResourceSet.class);
resourceSet.addLoadOption(XtextResource.OPTION_RESOLVE_ALL,
Boolean.TRUE);
org.eclipse.emf.ecore.resource.Resource resource = resourceSet
.getResource(
URI.createURI(
"platform:documentation.mydsl"),
true);

for (TreeIterator<EObject> iterator = resource
.getAllContents(); iterator.hasNext();) {
EObject eObject = iterator.next();
System.out.println(eObject.eClass().getName());
if (mlcdp.getDocumentation(eObject) != null) {
String docu = mlcdp.getDocumentation(eObject);
}
}

but I get just NullPointerExceptions


I've used the standalonesetup since I want it to be runnable from outside the plugin as a standalone application. Is that the problem? Am I missing something else?

Thanks!

Re: Problem with Multiline Comments [message #1778735 is a reply to message #1764516] Wed, 20 December 2017 15:16 Go to previous message
David  Riobó is currently offline David RiobóFriend
Messages: 9
Registered: August 2017
Junior Member
Ups I posted twice without noticing

[Updated on: Wed, 20 December 2017 15:19]

Report message to a moderator

Previous Topic:XText Update-Site older Releases
Next Topic:unresolved jvmTypeReference as generic type parameter
Goto Forum:
  


Current Time: Wed Sep 26 15:34:41 GMT 2018

Powered by FUDForum. Page generated in 0.02548 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top