Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » parse namespaced variable names(XTEXT grammar)
parse namespaced variable names [message #1773623] Sun, 01 October 2017 07:52 Go to next message
Gang Wang is currently offline Gang WangFriend
Messages: 5
Registered: October 2017
Junior Member
I am new to Xtext, and would really appreciate if any of you can help me fix the following problem.

I want to write a DSL that would be able to parse code like the following:
  var m:a/b/c
  var c:d/e/f
  var m:h/i/j
  
  def add(
      int v1, 
     int v2)
     
 let m:h/i/j = add(v1=m:a/b/c, v2=c:d/e/f)


The idea is in the "let" statement, the left side of the equation can only get all the vars already defined in the content assistance.

I have the following grammar:
grammar org.xtext.example.Calculator with org.eclipse.xtext.common.Terminals

generate calculatorDSL "..."

Model:
	statements += Statement*;
	
Statement:
	Variable | Func | Assignment;

Variable: 'var' name=PathId; 
Func: 'def' name=ID '(' parameters += Parameter* ')'; 	 
Assignment: 'let' what = [Variable] '=' callee = [Func] '(' ')';

QualifiedName:ID ('/' ID)*;
enum PartitionType: m|c|t;
Partition: name=PartitionType ':';
Path: {Path} name=QualifiedName;
PathId: part=Partition path=Path ;

Parameter: type = ParameterType name = ID ;
ParameterType: elementType=ElementType (array ?='[' (length=INT)? ']')?; 
	
ElementType: BasicType | EntityType;
BasicType:   typeName = ('string' | 'int' | 'boolean');
EntityType:  typeName = 'composite';


In the grammar, the Assignment rule tries to use the [Variable] to get all the vars defined, so that when ctrl+space is pressed it show them up. But instead, I got an error message in the "let" statement if I try the following:
var m:this/is/a/field                
var c:this/is/a/constant 
var m:this/is/the/result

def add(
        int v1
        int v2)
// all the lines above worked as expected

let m:this/is/the/result = add()


In the above DSL code, in the let statement, an error shows "mismatched input 'm' expecting RULE_ID".

I have been trying to find solutions for hours but got nowhere. Can anyone offer some insight on how to fix it?

Thanks.
Re: parse namespaced variable names [message #1773670 is a reply to message #1773623] Mon, 02 October 2017 16:07 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 11373
Registered: July 2009
Senior Member
https://blogs.itemis.com/en/xtext-hint-identifiers-conflicting-with-keywords


Need professional support for Xtext, Xpand, EMF?
Go to: http://xtext.itemis.com
Twitter : @chrdietrich
Blog : christiandietrich.wordpress.com

[Updated on: Mon, 02 October 2017 16:08]

Report message to a moderator

Re: parse namespaced variable names [message #1773675 is a reply to message #1773670] Mon, 02 October 2017 16:50 Go to previous messageGo to next message
Gang Wang is currently offline Gang WangFriend
Messages: 5
Registered: October 2017
Junior Member
Chris,

Thanks for the reply, but I am not sure I fully understand your suggestion, or if the suggestion fits this use case.

First, I tested with using ^ to escape as suggested by the blog, it did not work as the blog suggests. Secondly, if there are slashes "/" in the var names, the content assistance does not pick them up.

I guess my question is more or less distilled to the following:

1. In my DSL, the variables are not just ID's, rather they are composed of two parts:
[namespace]:[path], where namespace is from an enum, and path contains "/".
2. When a prior defined var such as "m:a/b/c" is cross referenced later on in the DSL, I'd like the content assistance to bring up these vars smartly. Namely: when I type "m:" and ctrl-space, it would show "m:a/b/c" and "m:e/f/g" and so on. If I type "m:a" it would only show "m:a/b/c".

Can you share some insights on how to do this? Maybe with a minimal grammar?

Thanks in advance for your teaching. BTW, I have been reading your blogs and posts about Xtext, and learned a lot. Appreciate what you do for the community!


Re: parse namespaced variable names [message #1773676 is a reply to message #1773675] Mon, 02 October 2017 16:52 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 11373
Registered: July 2009
Senior Member
ID ('/' ID)

Use ValidID here as in the blog


Need professional support for Xtext, Xpand, EMF?
Go to: http://xtext.itemis.com
Twitter : @chrdietrich
Blog : christiandietrich.wordpress.com
Re: parse namespaced variable names [message #1773687 is a reply to message #1773676] Tue, 03 October 2017 03:08 Go to previous messageGo to next message
Gang Wang is currently offline Gang WangFriend
Messages: 5
Registered: October 2017
Junior Member
Chris,

Thanks again for the reply. Below is the change I made based on your suggestion, unfortunately it did not work ... I had to comment the KEYWORD part away since it does not make sense to have the keywords to be a part of the "path".

Model:
	statements += Statement*;
	
Statement:
	Variable | Func | Assignment;

Variable: 'var' name=PathId; 
Func: 'def' name=ID '(' parameters += Parameter* ')'; 	 
Assignment: 'let' what = [Variable] '=' callee = [Func] '(' ')';

//QualifiedName:ID ('/' ID)*;
QualifiedName:ValidID ('/' ValidID)*;
ValidID: ID;// | KEYWORD;
//KEYWORD: 'm:'|'t:'|'c:';

enum PartitionType: m|c|t;
Partition: name=PartitionType ':';
Path: {Path} name=QualifiedName;
PathId: part=Partition path=Path ;

Parameter: type = ParameterType name = ID ;
ParameterType: elementType=ElementType (array ?='[' (length=INT)? ']')?; 
	
ElementType: BasicType | EntityType;
BasicType:   typeName = ('string' | 'int' | 'boolean');
EntityType:  typeName = 'composite';
Re: parse namespaced variable names [message #1773689 is a reply to message #1773687] Tue, 03 October 2017 04:25 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 11373
Registered: July 2009
Senior Member
i dont get that. in your post you complain about exactly that

Model:
	statements += Statement*;
	
Statement:
	Variable | Func | Assignment;

Variable: 'var' name=PathId; 
Func: 'def' name=ID '(' parameters += Parameter* ')'; 	 
Assignment: 'let' what = [Variable|CompleteVarName] '=' callee = [Func] '(' ')';

//QualifiedName:ID ('/' ID)*;
QualifiedName:ValidID ('/' ValidID)*;
ValidID: ID | KEYWORD;
KEYWORD: 'm'|'t'|'c';

CompleteVarName:
	('m'|'t'|'c')':'ValidID ('/' ValidID)*
;

enum PartitionType: m|c|t;
Partition: name=PartitionType ':';
Path: {Path} name=QualifiedName;
PathId: part=Partition path=Path ;

Parameter: type = ParameterType name = ID ;
ParameterType: elementType=ElementType (array ?='[' (length=INT)? ']')?; 
	
ElementType: BasicType | EntityType;
BasicType:   typeName = ('string' | 'int' | 'boolean');
EntityType:  typeName = 'composite';


i ssume youalready have adapted nameprovider and or scopeprovider

e.g.

class MyDslNameProvider extends DefaultDeclarativeQualifiedNameProvider {
	
	def QualifiedName qualifiedName(Variable v) {
		QualifiedName.create(v.name.part.name.name().toLowerCase+":"+v.name.path.name)
	}
}


or

class MyDslNameProvider extends DefaultDeclarativeQualifiedNameProvider {
	
	def QualifiedName qualifiedName(Variable v) {
		QualifiedName.create(NodeModelUtils.findNodesForFeature(v, MyDslPackage.Literals.VARIABLE__NAME).head.text.trim)
	}
}


with

class MyDslRuntimeModule extends AbstractMyDslRuntimeModule {
	
	override bindIQualifiedNameProvider() {
		MyDslNameProvider
	}
	
}



Need professional support for Xtext, Xpand, EMF?
Go to: http://xtext.itemis.com
Twitter : @chrdietrich
Blog : christiandietrich.wordpress.com
Re: parse namespaced variable names [message #1773751 is a reply to message #1773689] Wed, 04 October 2017 06:39 Go to previous messageGo to next message
Gang Wang is currently offline Gang WangFriend
Messages: 5
Registered: October 2017
Junior Member
Chris,

Thanks very much for your patience, and help. I am new to the technology and your suggestions are valuable and cannot be overstated.

I was limited in my thinking to hope it can be done purely by tweaking the grammar. It hadn't occurred to me I need code up the providers. Now I kinda got it work to an extent. But I now have more questions.

1. I couldn't get the first name provider work. Particularly, I got a compiler error in the following line complaining it cannot find 'path' and 'part' for variable 'v'.
QualifiedName.create(v.name.part.name.name().toLowerCase+":"+v.name.path.name)


2. I found that QualifiedName is kinda special --- I tried to use another way such as "MyQualifiedName" in the grammar, together adjusting the code, but got problems --- "MyQualifiedName cannot be resolved to a type." Did I miss anything here?
Path: {Path} name = MyQualifiedName; //QualifiedName;
PathId: part=Partition path=Path ;

MyQualifiedName:ValidID ('/' ValidID)*;

class CalcGraphDSLNameProvider extends DefaultDeclarativeQualifiedNameProvider {
	
	def MyQualifiedName qualifiedName(Variable v) {
		MyQualifiedName.create(NodeModelUtils.findNodesForFeature(v, CalcGraphDSLPackage.Literals.VARIABLE__NAME).head.text.trim)
	}
}


3. More seriously, the hover over behavior is broken. Now if I hover over a variable, in the tool tip window, it shows just "Variable" without the variable name. If I use [Variable|STRING] rule, I can get the right behavior at hovering over i.e. "Variable m:a/b/c". of course, using STRING does not give me the right content assistance behavior as we discussed. Any insight on how I can fix it?
One thing I am guessing is that with the CompleteVarName rule, the variables in the "let" statement don't match up with the variables in the "var" statement. But why the content assistance works then? I am puzzled.

Thanks again.
Re: parse namespaced variable names [message #1773755 is a reply to message #1773751] Wed, 04 October 2017 07:49 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 11373
Registered: July 2009
Senior Member
which grammar did you use?
2 makes no sense QUalifedName from grammar and qualifiedname as xtext class have zero in common
3 dont understand might be followup of 1


Need professional support for Xtext, Xpand, EMF?
Go to: http://xtext.itemis.com
Twitter : @chrdietrich
Blog : christiandietrich.wordpress.com

[Updated on: Wed, 04 October 2017 08:36]

Report message to a moderator

Re: parse namespaced variable names [message #1773784 is a reply to message #1773755] Wed, 04 October 2017 15:10 Go to previous messageGo to next message
Gang Wang is currently offline Gang WangFriend
Messages: 5
Registered: October 2017
Junior Member
Thank again, Chris.

The grammar I am using is almost identical to the one you sent in your previous email. I am enclosing the code here, it is extracted and minimized to focus on the points we are on:
grammar com.intuit.tke.CalcGraphDSL with org.eclipse.xtext.common.Terminals
generate calcGraphDSL "http://www.intuit.com/tke/CalcGraphDSL"

Model:statements += Statement*;	
Statement: Variable | Func | Assignment;

Variable: 'var' name=PathId type=ParameterType;
//Variable: 'var' name=STRING type=ParameterType;
Func: 'def' name=ID '(' parameters += Parameter (',' parameters += Parameter)* ')'; 	 
//Assignment: 'let' what = [Variable|STRING] '=' callee = [Func] '(' inputs+=InputPair (',' inputs+=InputPair)* ')';
Assignment: 'let' what = [Variable|CompleteVarName] '=' callee = [Func] '(' inputs+=InputPair (',' inputs+=InputPair)* ')';

//InputPair: name=ID '=' field=[Variable|STRING];
InputPair: field=[Variable|CompleteVarName] '=>' name=ID ;

enum PartitionType: m|c|t;
Partition: name = PartitionType ':';
Path: {Path} name = QualifiedName; 
//PathId returns String: part=Partition path=Path ;
PathId: part=Partition path=Path ;

QualifiedName:ValidID ('/' ValidID)*;
ValidID: ID | KEYWORD;
KEYWORD: 'm'|'t'|'c';
CompleteVarName: ('m' |'t'|'c')':'ValidID ('/' ValidID)*;

Parameter: type = ParameterType name = ID ;
ParameterType: elementType=ElementType (array ?='[' (length=INT)? ']')?; 
	
ElementType: BasicType | EntityType;
BasicType:   typeName = ('string' | 'int' | 'boolean');
EntityType:  typeName = 'composite';


Can you elaborate on #2? I am a little lost here. Should I change it as the following? it still doesn't compile:
class CalcGraphDSLNameProvider extends DefaultDeclarative[b]My[/b]QualifiedNameProvider {
	
	def [b]My[/b]QualifiedName [b]myQualifiedName[/b](Variable v) {
		QualifiedName.create(NodeModelUtils.findNodesForFeature(v, CalcGraphDSLPackage.Literals.VARIABLE__NAME).head.text.trim)
	}
}

In other words, if I switch to use a different name than QualifiedName, how shall I fix the code?

Thanks.
Re: parse namespaced variable names [message #1773791 is a reply to message #1773784] Wed, 04 October 2017 16:18 Go to previous message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 11373
Registered: July 2009
Senior Member
class MyDslNameProvider extends DefaultDeclarativeQualifiedNameProvider {
	
	def QualifiedName qualifiedName(Variable v) {
		QualifiedName.create(v.name.part.name.name().toLowerCase+":"+v.name.path.name)
	}
}


=> compiles fine. no errors.
references work.

no switch you can name thing thing "Whatever" in the grammar, the nameprovider still returns QualifiedName
inherit from DefaultDeclarativeQualifiedNameProvider or IQualifiedNameProvider interface but no MyQualifiedName or Whatever.


for hover edit label provider e.g

class MyDslLabelProvider extends DefaultEObjectLabelProvider {

	@Inject
	new(AdapterFactoryLabelProvider delegate) {
		super(delegate);
	}
	
	@Inject
	IQualifiedNameProvider qnp;
	
	def text(Variable v) {
		qnp.getFullyQualifiedName(v).toString
	}

}


Need professional support for Xtext, Xpand, EMF?
Go to: http://xtext.itemis.com
Twitter : @chrdietrich
Blog : christiandietrich.wordpress.com
Previous Topic:Important: Xtext and Oxygen 1a
Next Topic:How to trigger a full re-build programmatically
Goto Forum:
  


Current Time: Wed Nov 22 01:59:49 GMT 2017

Powered by FUDForum. Page generated in 0.02081 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software