Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Parsing block expressions vs. scope
Parsing block expressions vs. scope [message #804417] Wed, 22 February 2012 15:50 Go to next message
Alex G is currently offline Alex GFriend
Messages: 96
Registered: January 2012
Member
Hi!

I have a parsing problem for a DSL like:

Block blc1 extends blc2 {
   blc2.feat
}
Block blc2 {
   Attribute feat
}


So where's the problem? blc2 is referenced from blc1 (and defined after blc1). If I want to reference on blc2.feat and write something like

Block blc1 extends blc2 {
blc2.
}


then the next item after '.' which is parsed is '}' with error "mismatched input '}' expecting ID". If I complete the statement manually (with feat), everything is OK. But I want to have a content proposal for the attributes in blc1 and they don't show up, since scoping somehow doesn't work or the block within '{' and '}' isn't consumed as a whole expression.

If I'm defining blc2 before blc1, everything works fine!

Where do I have to hook in the XText objects in order to consume block expressions at the whole? Do I need something like a specified (infinite) lookahead?

Edit:
If I write something like
Block blc1 extends blc2 {
blc2.|
sometext
}

then scoping together with giving me the correct proposals (here the feature feat) works fine (| is the cursor position).

Edit2:
If I add to my grammar an artificial hidden() with an ANY_ELEMENT* option, then it works fine again (comapre suggestion http://www.eclipse.org/forums/index.php/mv/msg/296175/804284/#msg_804284). But it's a hack inside the grammar definition and does not reflect the language as it should be.

[Updated on: Wed, 22 February 2012 16:20]

Report message to a moderator

Re: Parsing block expressions vs. scope [message #804607 is a reply to message #804417] Wed, 22 February 2012 20:19 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik LindbergFriend
Messages: 2509
Registered: July 2009
Senior Member
Can you show the grammar?

- henrik

On 2012-22-02 16:50, Alex G wrote:
> Hi!
>
> I have a parsing problem for a DSL like:
>
>
> Block blc1 extends blc2 {
> blc2.feat
> }
> Block blc2 {
> Attribute feat
> }
>
>
> So where's the problem? blc2 is referenced from blc1 (and defined after
> blc1). If I want to reference on blc2.feat and write something like
>
>
> Block blc1 extends blc2 {
> blc2.
> }
>
>
> then the next item after '.' which is parsed is '}' with error
> "mismatched input '}' expecting ID". If I complete the statement
> manually (with feat), everything is OK. But I want to have a content
> proposal for the attributes in blc1 and they don't show up, since
> scoping somehow doesn't work or the block within '{' and '}' isn't
> consumed as a whole expression.
>
> If I'm defining blc2 before blc1, everything works fine!
>
> Where do I have to hook in the XText objects in order to consume block
> expressions at the whole? Do I need something like a specified
> (infinite) lookahead?
Re: Parsing block expressions vs. scope [message #804662 is a reply to message #804607] Wed, 22 February 2012 21:43 Go to previous messageGo to next message
Alex G is currently offline Alex GFriend
Messages: 96
Registered: January 2012
Member
I just created a small DSL example in order to reproduce the effect of my bigger DSL and couldn't create this failure.
In my bigger DSL, when I write "blc2.|" (where | is the cursor), i loose the scope on the extension (here blc2), since it gets underlined in red by the editor. So I think, I have to look at my ScopeProvider. If that doesn't work, I let here a notification.
Re: Parsing block expressions vs. scope [message #805413 is a reply to message #804662] Thu, 23 February 2012 18:38 Go to previous messageGo to next message
Alex G is currently offline Alex GFriend
Messages: 96
Registered: January 2012
Member
I found out the answer! Took me hours!!!

My part of grammar:
terminal SID : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*;
IID : SID [ INT ]
SIDone: SID;
SIDtwo: '(' SID ')';
SIDthree: SID;
IIDone : IID
IIDtwo : IID

and I had some rules which had a dependence on these data types, like
RuleOne: name = SIDone ...
RuleFoo: name = [RuleOne | SIDone] ...
RuleBar: name = SIDtwo ...
RuleFortyTwo: name = IIDtwo ...

I wanted to be flexible with these data types - now I learned my lesson ^^.

Is the only way to be flexible for later when grammer changes (like the data types for names above), to create rules and not data types like
terminal SID : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*;
IID : SID ([ INT ])?
SIDone: id=SID;
SIDtwo: id='(' SID ')';
SIDthree: id=SID;
IIDone : id=IID
IIDtwo : id=IID

and modify the QualifiedNameProbider for RuleOne, RuleFoo etc. to get the names from SIDone, SIDtwo etc? The problem in this case is, that I have then two identical simple names, what I wanted to avoid and therefore created these data types.

And what will happen with the data type "IID"? There could be collision with the terminal SID - the parser will not know, how to distinguish a SID from an IID? Can I somehow avoid it and leave IID as a data type without introducing type check in the scope or qualified name classes?
Re: Parsing block expressions vs. scope [message #806399 is a reply to message #805413] Fri, 24 February 2012 23:20 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik LindbergFriend
Messages: 2509
Registered: July 2009
Senior Member
I have a hard time following what you are trying to express.
As an example, this expression:
> IID : SID [ INT ]

reads IID is a datatype starting with a SID, followed by a reference !?!
(i.e. a pointer) to an INT using an ID to locate the INT. That just does
not make sense. typos ?

To make it more convenient - use the feature "name" as the identity of
referenceable elements, and reference them by qualified name (i.e. ID).
Then it works out of the box.

If you then need to reference things based on something else - then you
need to think about the scoping of that (same file, across files, across
projects/bundles etc.).

Are you using the default terminals? If so, there is an ID already that
overlaps with your SID, IID, etc. If you need ID to be something
different than the default, just override it (redeclare it in your grammar).

Hope that helps for starters.
- henrik


On 2012-23-02 19:38, Alex G wrote:
> I found out the answer! Took me hours!!!
>
> My part of grammar:
>
> terminal SID : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*;
> IID : SID [ INT ]
> SIDone: SID;
> SIDtwo: '(' SID ')';
> SIDthree: SID;
> IIDone : IID
> IIDtwo : IID
>
> and I had some rules which had a dependence on these data types, like
>
> RuleOne: name = SIDone ...
> RuleFoo: name = [RuleOne | SIDone] ...
> RuleBar: name = SIDtwo ...
> RuleFortyTwo: name = IIDtwo ...
>
> I wanted to be flexible with these data types - now I learned my lesson ^^.
>
> Is the only way to be flexible for later when grammer changes (like the
> data types for names above), to create rules and not data types like
>
> terminal SID : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*;
> IID : SID ([ INT ])?
> SIDone: id=SID;
> SIDtwo: id='(' SID ')';
> SIDthree: id=SID;
> IIDone : id=IID
> IIDtwo : id=IID
>
> and modify the QualifiedNameProbider for RuleOne, RuleFoo etc. to get
> the names from SIDone, SIDtwo etc? The problem in this case is, that I
> have then two identical simple names, what I wanted to avoid and
> therefore created these data types.
>
> And what will happen with the data type "IID"? There could be collision
> with the terminal SID - the parser will not know, how to distinguish a
> SID from an IID? Can I somehow avoid it and leave IID as a data type
> without introducing type check in the scope or qualified name classes?
Re: Parsing block expressions vs. scope [message #806427 is a reply to message #806399] Sat, 25 February 2012 00:21 Go to previous messageGo to next message
Alex G is currently offline Alex GFriend
Messages: 96
Registered: January 2012
Member
Hi again!

Indeed I thought that I have found out the error, but I didn't (after further hours of searching for a "bug"). I've created a small version of the problem shown in the first post. The main problem is, I think, that I reference a reference, which isn't referenced anymore while editing the model, s.t. I can't reference it while editing, but when editing is finished (and the reference written manually without content assist), the model parses well. Here the code:

Grammar:
grammar org.xtext.example.abc.ABC with org.eclipse.xtext.common.Terminals

generate aBC "http://www.xtext.org/example/abc/ABC"

ABCLanguage: (D += ItemDefinition)*;

ItemDefinition: 
	"Item" name=ID
 	(inheritance ?= ":" inherits=Inherits)?
 	body=ItemBody;
	

ItemBody:
	body?="{" 
  	(  WD += WorkingDefinition |
    	ID += ItemDefinition AI += AbstractItem
  	)*
  	"}";
  
Inherits:
	defrefs += DefinitionReference ("," defrefs+= DefinitionReference)*;
	
DefinitionReference:
	(name=ID hasName?="<-")?
	ref = [ItemDefinition | ID];

WorkingDefinition:
  	"Working" name=ID;

AbstractItem:
	"AbstractItem" id=ID "holds" ref=[ItemDefinition | ID];



What do I want to express with this grammar? Models like:

Item aaa111 : aaa222 {
	Item aaa444 : aaa222 {} AbstractItem Id holds aaa222
}

Item aaa222 {
	
}


So, Item aaa111 references to aaa222. In Item aaa111 we have Item aaa444 which wants to reference to the reference aaa222 (not directly to the Item aaa222 outside the scope(!)). After it I added a small part of grammar "AbstractItem ..." which was necessary to show up the error, that is going on.

Here my scope (XTend):

class ABCScopeProvider extends AbstractDeclarativeScopeProvider {
	
	override IScope getScope(EObject context, EReference reference) {
		
		println("")
		println("scope_" + reference.EContainingClass.name +
				"_" + reference.name + 
				"(" + context.eClass.name + ", ..)"
				)
		super.getScope(context,reference);
	}
	
	def IScope scope_DefinitionReference_ref(ItemDefinition item, EReference eRef) {
		Literals::DEFINITION_REFERENCE__REF!=null // assertion		
		
		val parent = item.eContainer
		switch (parent) {
			ItemBody : {
				val parentMod = parent.eContainer as ItemDefinition
				Scopes::scopeFor(parent.AI, scope_DefinitionReference_ref(parentMod, eRef))
			}
			ABCLanguage : {
				Scopes::scopeFor(parent.d)
			}
			default : {
				IScope::NULLSCOPE
			}
		}
	}
	
	
}


and here my qualified name:

public class ABCQualifiedNameProvider extends SimpleNameProvider  {


	// copied from DefaultDeclarativeQualifiedNameProvider and changed by
	// (1) simple names instead of fully qualified names
	// (2) scoping issues with constraining parent and child views (set parent scopes to IScope.NULLSCOPE)
	// compare getFullyQualifiedName methods
	private PolymorphicDispatcher<QualifiedName> qualifiedName = new PolymorphicDispatcher<QualifiedName>("qualifiedName",1,1,Collections.singletonList(this), PolymorphicDispatcher.NullErrorHandler.<QualifiedName>get())
			{
		@Override
		protected QualifiedName handleNoSuchMethod(Object... params) { 
			return null;
		}
	}; 
	
	private Function<EObject, String> resolver = SimpleAttributeResolver.newResolver(String.class, "name");
	
	protected Function<EObject, String> getResolver() {
		return resolver;
	}
	
	@Inject
	private IQualifiedNameConverter converter = new IQualifiedNameConverter.DefaultImpl();
	
	public QualifiedName getFullyQualifiedName(final EObject obj) {
	
		EObject temp = obj;
		QualifiedName qualifiedNameFromDispatcher = qualifiedName.invoke(temp);
		if (qualifiedNameFromDispatcher!=null)
			return qualifiedNameFromDispatcher;
		String name = getResolver().apply(temp);
		if (Strings.isEmpty(name))
			return null;
		QualifiedName qualifiedNameFromConverter = converter.toQualifiedName(name);
		return qualifiedNameFromConverter;
	}
				
	
	QualifiedName qualifiedName(DefinitionReference defref) {
		String name = null;
		QualifiedName qname = null;
		if(defref.isHasName()) {
			name = defref.getName();
			qname = converter.toQualifiedName(name);
		} else {
		// qname = getFullyQualifiedName(defref.getRef()); // can call lazy links
			List<INode> nodes = NodeModelUtils.findNodesForFeature(defref, ABCPackage.Literals.DEFINITION_REFERENCE__REF);
			INode first = null;
			if(nodes.size()>=1) {
				first = nodes.get(0);
				name = NodeModelUtils.getTokenText(first);
				qname = converter.toQualifiedName(name);
			}
		}
		return qname;
	}
}



If I define the Item aaa222 before (above) Item aaa111, everythink works fine. If I define it as in the code above (meaning after Item aaa222, then I get lost the reference to aaa222 when writing the following:

Item aaa111 : aaa222 {
	Item aaa444 : aaa222 {} AbstractItem Id holds aaa222
	Item aaa333 : | 
}

Item aaa222 {}


where | indicates the cursor. The reference aaa222 is underlined, since the Item aaa222 couldn't be parsed correctly.

I hope this can help to show up the problem and thank you for some help in advance.
Re: Parsing block expressions vs. scope [message #807274 is a reply to message #806427] Sun, 26 February 2012 06:55 Go to previous messageGo to next message
Alex G is currently offline Alex GFriend
Messages: 96
Registered: January 2012
Member
EDIT: double post, since post did not appear when submitted

[Updated on: Sun, 26 February 2012 07:06]

Report message to a moderator

Re: Parsing block expressions vs. scope [message #807279 is a reply to message #806427] Sun, 26 February 2012 07:04 Go to previous message
Alex G is currently offline Alex GFriend
Messages: 96
Registered: January 2012
Member
I think I have found what is happening here (looks like a bug):

The error could be somewhere in the AbstractDeclarativeScopeProvider or lexing/parsing:

If I remove the "scope_..." method everything works fine - meaning I can refer by clicking on F3 on the reference aaa222 to its definition.
But If I add the customized scope, the call "parent.getD()" only gives me the item aaa111, since somehow the object aaa222 isn't created (or its name)?

If I disable the "scope_..." method and add
		val parent = EcoreUtil2::getContainerOfType(context, typeof(ABCLanguage))
		println(parent.d)


int the "getScope" method, I only see the item "aaa111" to be referencable as well, although aaa222 is referencable when disabling the customized scope.

By the way, in order to reproduce this effect it is enough to change AbstractItem to:
AbstractItem: {AbstractItem}
	"AbstractItem" ;//id=ID;// "holds";// ref=[ItemDefinition | ID];

or

AbstractItem:
	"AbstractItem" id=ID;// "holds";// ref=[ItemDefinition | ID];


Can someone explain me what is happening here?

It could be also the reason that the qualified names are computed too late: I checked that sometimes EObjects do have null names but are referencable, because the customized qualified names are called after creating the EObject. Could this be the reason?
Previous Topic:language with unstructured macros
Next Topic:limit scoping of first fragment of qualified name?
Goto Forum:
  


Current Time: Tue Apr 07 08:34:57 GMT 2020

Powered by FUDForum. Page generated in 0.01913 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top