Eclipse Community Forums: TMF (Xtext) » parse namespaced variable names

Help

Home

Home » Modeling » TMF (Xtext) » parse namespaced variable names(XTEXT grammar)

Show: Today's Messages :: Show Polls :: Message Navigator

parse namespaced variable names [message #1773623]

Sun, 01 October 2017 07:52

Gang Wang

Messages: 5
Registered: October 2017

Junior Member

I am new to Xtext, and would really appreciate if any of you can help me fix the following problem.

I want to write a DSL that would be able to parse code like the following:

  var m:a/b/c
  var c:d/e/f
  var m:h/i/j
  
  def add(
      int v1, 
     int v2)
     
 let m:h/i/j = add(v1=m:a/b/c, v2=c:d/e/f)

The idea is in the "let" statement, the left side of the equation can only get all the vars already defined in the content assistance.

I have the following grammar:

grammar org.xtext.example.Calculator with org.eclipse.xtext.common.Terminals

generate calculatorDSL "..."

Model:
	statements += Statement*;
	
Statement:
	Variable | Func | Assignment;

Variable: 'var' name=PathId; 
Func: 'def' name=ID '(' parameters += Parameter* ')'; 	 
Assignment: 'let' what = [Variable] '=' callee = [Func] '(' ')';

QualifiedName:ID ('/' ID)*;
enum PartitionType: m|c|t;
Partition: name=PartitionType ':';
Path: {Path} name=QualifiedName;
PathId: part=Partition path=Path ;

Parameter: type = ParameterType name = ID ;
ParameterType: elementType=ElementType (array ?='[' (length=INT)? ']')?; 
	
ElementType: BasicType | EntityType;
BasicType:   typeName = ('string' | 'int' | 'boolean');
EntityType:  typeName = 'composite';

In the grammar, the Assignment rule tries to use the [Variable] to get all the vars defined, so that when ctrl+space is pressed it show them up. But instead, I got an error message in the "let" statement if I try the following:

var m:this/is/a/field                
var c:this/is/a/constant 
var m:this/is/the/result

def add(
        int v1
        int v2)
// all the lines above worked as expected

let m:this/is/the/result = add()

In the above DSL code, in the let statement, an error shows "mismatched input 'm' expecting RULE_ID".

I have been trying to find solutions for hours but got nowhere. Can anyone offer some insight on how to fix it?

Thanks.

Report message to a moderator

Re: parse namespaced variable names [message #1773670 is a reply to message #1773623]

Mon, 02 October 2017 16:07

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

https://blogs.itemis.com/en/xtext-hint-identifiers-conflicting-with-keywords

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Mon, 02 October 2017 16:08]

Report message to a moderator

Re: parse namespaced variable names [message #1773675 is a reply to message #1773670]

Mon, 02 October 2017 16:50

Gang Wang

Messages: 5
Registered: October 2017

Junior Member

Chris,

Thanks for the reply, but I am not sure I fully understand your suggestion, or if the suggestion fits this use case.

First, I tested with using ^ to escape as suggested by the blog, it did not work as the blog suggests. Secondly, if there are slashes "/" in the var names, the content assistance does not pick them up.

I guess my question is more or less distilled to the following:

1. In my DSL, the variables are not just ID's, rather they are composed of two parts:
[namespace]:[path], where namespace is from an enum, and path contains "/".
2. When a prior defined var such as "m:a/b/c" is cross referenced later on in the DSL, I'd like the content assistance to bring up these vars smartly. Namely: when I type "m:" and ctrl-space, it would show "m:a/b/c" and "m:e/f/g" and so on. If I type "m:a" it would only show "m:a/b/c".

Can you share some insights on how to do this? Maybe with a minimal grammar?

Thanks in advance for your teaching. BTW, I have been reading your blogs and posts about Xtext, and learned a lot. Appreciate what you do for the community!

Report message to a moderator

Re: parse namespaced variable names [message #1773676 is a reply to message #1773675]

Mon, 02 October 2017 16:52

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

ID ('/' ID)

Use ValidID here as in the blog

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: parse namespaced variable names [message #1773687 is a reply to message #1773676]

Tue, 03 October 2017 03:08

Gang Wang

Messages: 5
Registered: October 2017

Junior Member

Chris,

Thanks again for the reply. Below is the change I made based on your suggestion, unfortunately it did not work ... I had to comment the KEYWORD part away since it does not make sense to have the keywords to be a part of the "path".

Model:
	statements += Statement*;
	
Statement:
	Variable | Func | Assignment;

Variable: 'var' name=PathId; 
Func: 'def' name=ID '(' parameters += Parameter* ')'; 	 
Assignment: 'let' what = [Variable] '=' callee = [Func] '(' ')';

//QualifiedName:ID ('/' ID)*;
QualifiedName:ValidID ('/' ValidID)*;
ValidID: ID;// | KEYWORD;
//KEYWORD: 'm:'|'t:'|'c:';

enum PartitionType: m|c|t;
Partition: name=PartitionType ':';
Path: {Path} name=QualifiedName;
PathId: part=Partition path=Path ;

Parameter: type = ParameterType name = ID ;
ParameterType: elementType=ElementType (array ?='[' (length=INT)? ']')?; 
	
ElementType: BasicType | EntityType;
BasicType:   typeName = ('string' | 'int' | 'boolean');
EntityType:  typeName = 'composite';

Report message to a moderator

Re: parse namespaced variable names [message #1773689 is a reply to message #1773687]

Tue, 03 October 2017 04:25

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

i dont get that. in your post you complain about exactly that

Model:
	statements += Statement*;
	
Statement:
	Variable | Func | Assignment;

Variable: 'var' name=PathId; 
Func: 'def' name=ID '(' parameters += Parameter* ')'; 	 
Assignment: 'let' what = [Variable|CompleteVarName] '=' callee = [Func] '(' ')';

//QualifiedName:ID ('/' ID)*;
QualifiedName:ValidID ('/' ValidID)*;
ValidID: ID | KEYWORD;
KEYWORD: 'm'|'t'|'c';

CompleteVarName:
	('m'|'t'|'c')':'ValidID ('/' ValidID)*
;

enum PartitionType: m|c|t;
Partition: name=PartitionType ':';
Path: {Path} name=QualifiedName;
PathId: part=Partition path=Path ;

Parameter: type = ParameterType name = ID ;
ParameterType: elementType=ElementType (array ?='[' (length=INT)? ']')?; 
	
ElementType: BasicType | EntityType;
BasicType:   typeName = ('string' | 'int' | 'boolean');
EntityType:  typeName = 'composite';

i ssume youalready have adapted nameprovider and or scopeprovider

e.g.

class MyDslNameProvider extends DefaultDeclarativeQualifiedNameProvider {
	
	def QualifiedName qualifiedName(Variable v) {
		QualifiedName.create(v.name.part.name.name().toLowerCase+":"+v.name.path.name)
	}
}

class MyDslNameProvider extends DefaultDeclarativeQualifiedNameProvider {
	
	def QualifiedName qualifiedName(Variable v) {
		QualifiedName.create(NodeModelUtils.findNodesForFeature(v, MyDslPackage.Literals.VARIABLE__NAME).head.text.trim)
	}
}

with

class MyDslRuntimeModule extends AbstractMyDslRuntimeModule {
	
	override bindIQualifiedNameProvider() {
		MyDslNameProvider
	}
	
}

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: parse namespaced variable names [message #1773751 is a reply to message #1773689]

Wed, 04 October 2017 06:39

Gang Wang

Messages: 5
Registered: October 2017

Junior Member

Chris,

Thanks very much for your patience, and help. I am new to the technology and your suggestions are valuable and cannot be overstated.

I was limited in my thinking to hope it can be done purely by tweaking the grammar. It hadn't occurred to me I need code up the providers. Now I kinda got it work to an extent. But I now have more questions.

1. I couldn't get the first name provider work. Particularly, I got a compiler error in the following line complaining it cannot find 'path' and 'part' for variable 'v'.

QualifiedName.create(v.name.part.name.name().toLowerCase+":"+v.name.path.name)

2. I found that QualifiedName is kinda special --- I tried to use another way such as "MyQualifiedName" in the grammar, together adjusting the code, but got problems --- "MyQualifiedName cannot be resolved to a type." Did I miss anything here?

Path: {Path} name = MyQualifiedName; //QualifiedName;
PathId: part=Partition path=Path ;

MyQualifiedName:ValidID ('/' ValidID)*;

class CalcGraphDSLNameProvider extends DefaultDeclarativeQualifiedNameProvider {
	
	def MyQualifiedName qualifiedName(Variable v) {
		MyQualifiedName.create(NodeModelUtils.findNodesForFeature(v, CalcGraphDSLPackage.Literals.VARIABLE__NAME).head.text.trim)
	}
}

3. More seriously, the hover over behavior is broken. Now if I hover over a variable, in the tool tip window, it shows just "Variable" without the variable name. If I use [Variable|STRING] rule, I can get the right behavior at hovering over i.e. "Variable m:a/b/c". of course, using STRING does not give me the right content assistance behavior as we discussed. Any insight on how I can fix it?
One thing I am guessing is that with the CompleteVarName rule, the variables in the "let" statement don't match up with the variables in the "var" statement. But why the content assistance works then? I am puzzled.

Thanks again.

Report message to a moderator

Re: parse namespaced variable names [message #1773755 is a reply to message #1773751]

Wed, 04 October 2017 07:49

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

which grammar did you use?
2 makes no sense QUalifedName from grammar and qualifiedname as xtext class have zero in common
3 dont understand might be followup of 1

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Wed, 04 October 2017 08:36]

Report message to a moderator

Re: parse namespaced variable names [message #1773784 is a reply to message #1773755]

Wed, 04 October 2017 15:10

Gang Wang

Messages: 5
Registered: October 2017

Junior Member

Thank again, Chris.

The grammar I am using is almost identical to the one you sent in your previous email. I am enclosing the code here, it is extracted and minimized to focus on the points we are on:

grammar com.intuit.tke.CalcGraphDSL with org.eclipse.xtext.common.Terminals
generate calcGraphDSL "http://www.intuit.com/tke/CalcGraphDSL"

Model:statements += Statement*;	
Statement: Variable | Func | Assignment;

Variable: 'var' name=PathId type=ParameterType;
//Variable: 'var' name=STRING type=ParameterType;
Func: 'def' name=ID '(' parameters += Parameter (',' parameters += Parameter)* ')'; 	 
//Assignment: 'let' what = [Variable|STRING] '=' callee = [Func] '(' inputs+=InputPair (',' inputs+=InputPair)* ')';
Assignment: 'let' what = [Variable|CompleteVarName] '=' callee = [Func] '(' inputs+=InputPair (',' inputs+=InputPair)* ')';

//InputPair: name=ID '=' field=[Variable|STRING];
InputPair: field=[Variable|CompleteVarName] '=>' name=ID ;

enum PartitionType: m|c|t;
Partition: name = PartitionType ':';
Path: {Path} name = QualifiedName; 
//PathId returns String: part=Partition path=Path ;
PathId: part=Partition path=Path ;

QualifiedName:ValidID ('/' ValidID)*;
ValidID: ID | KEYWORD;
KEYWORD: 'm'|'t'|'c';
CompleteVarName: ('m' |'t'|'c')':'ValidID ('/' ValidID)*;

Parameter: type = ParameterType name = ID ;
ParameterType: elementType=ElementType (array ?='[' (length=INT)? ']')?; 
	
ElementType: BasicType | EntityType;
BasicType:   typeName = ('string' | 'int' | 'boolean');
EntityType:  typeName = 'composite';

Can you elaborate on #2? I am a little lost here. Should I change it as the following? it still doesn't compile:

class CalcGraphDSLNameProvider extends DefaultDeclarative[b]My[/b]QualifiedNameProvider {
	
	def [b]My[/b]QualifiedName [b]myQualifiedName[/b](Variable v) {
		QualifiedName.create(NodeModelUtils.findNodesForFeature(v, CalcGraphDSLPackage.Literals.VARIABLE__NAME).head.text.trim)
	}
}

In other words, if I switch to use a different name than QualifiedName, how shall I fix the code?

Thanks.

Report message to a moderator

Re: parse namespaced variable names [message #1773791 is a reply to message #1773784]

Wed, 04 October 2017 16:18

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

class MyDslNameProvider extends DefaultDeclarativeQualifiedNameProvider {
	
	def QualifiedName qualifiedName(Variable v) {
		QualifiedName.create(v.name.part.name.name().toLowerCase+":"+v.name.path.name)
	}
}

=> compiles fine. no errors.
references work.

no switch you can name thing thing "Whatever" in the grammar, the nameprovider still returns QualifiedName
inherit from DefaultDeclarativeQualifiedNameProvider or IQualifiedNameProvider interface but no MyQualifiedName or Whatever.

for hover edit label provider e.g

class MyDslLabelProvider extends DefaultEObjectLabelProvider {

	@Inject
	new(AdapterFactoryLabelProvider delegate) {
		super(delegate);
	}
	
	@Inject
	IQualifiedNameProvider qnp;
	
	def text(Variable v) {
		qnp.getFullyQualifiedName(v).toString
	}

}

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Previous Topic:	Important: Xtext and Oxygen 1a
Next Topic:	How to trigger a full re-build programmatically

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Fri Apr 19 18:37:27 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter