Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Block scope, validation, linking and content assist
Block scope, validation, linking and content assist [message #722326] Mon, 05 September 2011 11:42 Go to next message
eecolor is currently offline eecolorFriend
Messages: 36
Registered: September 2011
Member
First I would like to say I agree with Ed: "Xtext is the coolest of the cool projects at Eclipse today"

I am trying to model a language that has the concept of block scopes. A block is created with { and }. This means that a variable is only accessible within the current block.

I have made a simplified example with only variable declarations and variable references. The grammar (I had to remove http because it's my first post):

grammar ee.xtext.test.Test with org.eclipse.xtext.common.Terminals

generate test "www.xtext.ee/test/Test"

File:
	(expressions+=BlockExpression)+
;

Expression:
	VariableDeclarations | FeatureCall | BlockExpression
;

BlockExpression returns Expression:
	{BlockExpression}
	'{'
		(expressions+=Expression)+
	'}'
;

VariableDeclarations returns Expression:
	{VariableDeclarations}
	'var' declarations+=VariableDeclaration (',' declarations+=VariableDeclaration)*;

VariableDeclaration:
	{VariableDeclaration}
	name=ID;
	
FeatureCall returns Expression:
	{FeatureCall}
	feature=[VariableDeclaration];


An example of the resulting language in an editor:

{
	b						//should not work
	
	var a					//declare a1 (gives an error on a)
	
	
	{
		a			//refer to a1
		
		var b, c		//declare b1 and c1 (gives an error on c)
		
		c			//refer to c1
		
		{
			var c	//declare c2  (gives an error on c)
			
			a		//refer to a1
			b		//refer to b1
			c		//refer to c2
		}
		
		var a			//declare a2 (gives an error on a)
		
		a			//refer to a2
		c			//refer to c1
	}
	
	c					//should not work
}


I have added comments to each line to specify what should happen. The following problems are in play:

1. Variables can be referenced when they are not declared yet
2. Variables can be referenced outside of block
3. Variables can not be declared with the same name in a different block
4. Variables do not reference the correct declaration
5. Content assist proposes variables in a sub block and variables that are not created yet (later in the document)

After searching these forums and various posts about scoping I was able to gather the following assumptions:

1. I am not completely sure how to solve it. My guess would be to create a validator that checks if a reference references a declaration earlier in the document.
2. If I am correct, this is caused by the lack of scoping. It seems I would need a custom scope provider that creates nested scopes for each block.
3. It seems this is caused by the NamesAreUniqueValidator. From what I have gathered it does not take scoping into account. This means I would need to create a custom NamesAreUniqueValidator that does take scoping into account.
4. I don't know how to solve this
5. Part of this will be solved by scoping (see 2) as it reduces the available variables. As for referencing variables that are not declared yet, I think this can be solved with a custom ProposalProvider.

I wonder if the above assumptions are correct and if anyone can help me fill in the blanks. If I have missed available implementations I would love to hear about them.

Thank you

[Updated on: Mon, 05 September 2011 11:44]

Report message to a moderator

Re: Block scope, validation, linking and content assist [message #722331 is a reply to message #722326] Mon, 05 September 2011 12:00 Go to previous messageGo to next message
Sven Efftinge is currently offline Sven EfftingeFriend
Messages: 1823
Registered: July 2009
Senior Member
All but 3) will be solved by a proper scoping implementation.
3) can be solved by either deactivating the NamesAreUnqiue validation or by subclassing the validator and make him exclude local variables.

Xbase has a similar block expression with variables. You could have a look at how the scoping is implemented for that. http://goo.gl/TZJwG

Sven
Re: Block scope, validation, linking and content assist [message #722428 is a reply to message #722331] Mon, 05 September 2011 18:36 Go to previous messageGo to next message
eecolor is currently offline eecolorFriend
Messages: 36
Registered: September 2011
Member
Thank you for your fast reply. I will check out the Xbase scope provider.

As for 3), do you think it's doable to make it scope aware. I do want want names to be unique within a block.
Re: Block scope, validation, linking and content assist [message #722468 is a reply to message #722428] Mon, 05 September 2011 21:39 Go to previous messageGo to next message
eecolor is currently offline eecolorFriend
Messages: 36
Registered: September 2011
Member
I have tried to implement scoping like this:

public class TestScopeProvider extends AbstractDeclarativeScopeProvider {
	public IScope scope_FeatureCall_feature(BlockExpression context, EReference reference)
	{
		IScope parentScope = getParentScopes(context);
		
		// problem, no current location available
		List<VariableDeclaration> elements = getVariableDeclarations(null, context);
		
		return Scopes.scopeFor(elements, parentScope);
	}
	
	public IScope scope_FeatureCall_feature(FeatureCall context, EReference reference)
	{
		BlockExpression block = (BlockExpression) context.eContainer();
		
		IScope parentScope = getParentScopes(block);
		
		List<VariableDeclaration> elements = getVariableDeclarations(context, block);
		
		return Scopes.scopeFor(elements, parentScope);
	}

	private IScope getParentScopes(BlockExpression block) {
		
		EObject container = block.eContainer();
		
		if (container instanceof BlockExpression)
		{
			BlockExpression parentBlock = (BlockExpression) container;
			IScope parentScope = getParentScopes(parentBlock);
			
			List<VariableDeclaration> elements = getVariableDeclarations(block, parentBlock);
			
			return Scopes.scopeFor(elements, parentScope);
		} else
		{
			return IScope.NULLSCOPE;
		}
	}
	
	private List<VariableDeclaration> getVariableDeclarations(EObject context, BlockExpression block) {
		
		EList<Expression> expressions = block.getExpressions();
		int index = context == null ? expressions.size() : expressions.indexOf(context);
		List<VariableDeclaration> elements = Lists.newArrayList();
		
		for (int i = 0; i < index; i++)
		{
			Expression expression = expressions.get(i);
			
			if (expression instanceof VariableDeclarations)
			{
				elements.addAll(((VariableDeclarations) expression).getDeclarations());
			}
		}
		return elements;
	}
}


This indeed solves most of the problems. There however remains one problem, the second part of 5)

Quote:
5. Content assist proposes variables in a sub block and variables that are not created yet


The problem of variables that are not created yet is apparent in the scope_FeatureCall_feature(BlockExpression context, EReference reference) method. The current position is unknown. I have looked at the Xbase implementation and it indeed manually calls the scope provider from a custom proposal provider.

Is there currently something implemented that allows me to know at what position the scope is requested from within the scope provider? Or should I mimic the Xbase implementation and manually retrieve the scope from a custom proposal provider?
Re: Block scope, validation, linking and content assist [message #722578 is a reply to message #722468] Tue, 06 September 2011 08:57 Go to previous messageGo to next message
Sven Efftinge is currently offline Sven EfftingeFriend
Messages: 1823
Registered: July 2009
Senior Member
Yes, you need to mimic what we did in XbaseProposalProvider
Re: Block scope, validation, linking and content assist [message #722800 is a reply to message #722578] Tue, 06 September 2011 18:20 Go to previous messageGo to next message
eecolor is currently offline eecolorFriend
Messages: 36
Registered: September 2011
Member
So far so good, I modified the scope provider in order to expose a method to get the scope:

public class TestScopeProvider extends AbstractDeclarativeScopeProvider {

	public IScope getBlockExpressionScope(int index, BlockExpression context)
	{
		IScope parentScope = getParentScopes(context);
		
		List<VariableDeclaration> elements = getVariableDeclarations(index, context.getExpressions());
		
		return Scopes.scopeFor(elements, parentScope);
	}
	
	public IScope scope_FeatureCall_feature(FeatureCall context, EReference reference)
	{
		BlockExpression block = (BlockExpression) context.eContainer();
		
		IScope parentScope = getParentScopes(block);
		
		EList<Expression> expressions = block.getExpressions();
		int index = expressions.indexOf(context);
		
		List<VariableDeclaration> elements = getVariableDeclarations(index, expressions);
		
		return Scopes.scopeFor(elements, parentScope);
	}

	private IScope getParentScopes(BlockExpression block) {
		
		EObject container = block.eContainer();
		
		if (container instanceof BlockExpression)
		{
			BlockExpression parentBlock = (BlockExpression) container;
			IScope parentScope = getParentScopes(parentBlock);
			
			EList<Expression> expressions = parentBlock.getExpressions();
			int index = expressions.indexOf(block);

			List<VariableDeclaration> elements = getVariableDeclarations(index, expressions);
			
			return Scopes.scopeFor(elements, parentScope);
		} else
		{
			return IScope.NULLSCOPE;
		}
	}
	
	private List<VariableDeclaration> getVariableDeclarations(int currentIndex, EList<Expression> expressions) {
		
		List<VariableDeclaration> elements = Lists.newArrayList();
		
		for (int i = 0; i < currentIndex; i++)
		{
			Expression expression = expressions.get(i);
			
			if (expression instanceof VariableDeclarations)
			{
				elements.addAll(((VariableDeclarations) expression).getDeclarations());
			}
		}
		return elements;
	}
}


The proposal provider now looks like this:

public class TestProposalProvider extends AbstractTestProposalProvider {

	@Inject
	private ReferenceProposalCreator crossReferenceProposalCreator;
	
	@Inject
	private TestScopeProvider scopeProvider;
	
	public void completeFeatureCall_Feature(EObject model, Assignment assignment, ContentAssistContext context, ICompletionProposalAcceptor acceptor) {
		CrossReference crossReference = (CrossReference) assignment.getTerminal();
		ParserRule containingParserRule = GrammarUtil.containingParserRule(crossReference);
		
		if (!GrammarUtil.isDatatypeRule(containingParserRule)) {
			
			String ruleName = null;
			if (crossReference.getTerminal() instanceof RuleCall) {
				ruleName = ((RuleCall) crossReference.getTerminal()).getRule().getName();
			}
			
			EReference reference = GrammarUtil.getReference(crossReference);
			
			EObject previousModel = context.getPreviousModel();
			
			int index = -1;
			
			if (!previousModel.equals(model))
			{
				//find the direct child of the model
				while (!previousModel.eContainer().equals(model))
				{
					previousModel = previousModel.eContainer();
				}
				
				index = model.eContents().indexOf(previousModel);
			}
			
			IScope scope = scopeProvider.getBlockExpressionScope(index + 1, (BlockExpression) model);
			
			crossReferenceProposalCreator.lookupCrossReference(scope, model, reference, acceptor, Predicates.<IEObjectDescription> alwaysTrue(), getProposalFactory(ruleName, context));
		}
	}
}


I couldn't help noticing the difference in the naming of the 'scope' and 'complete' methods. I would expect the complete method to be named 'complete_FeatureCall_feature' or the scope method to be named 'scopeFeatureCall_Feature'.

I will now dive into the NamesAreUniqueValidator.
Re: Block scope, validation, linking and content assist [message #722830 is a reply to message #722800] Tue, 06 September 2011 20:27 Go to previous messageGo to next message
eecolor is currently offline eecolorFriend
Messages: 36
Registered: September 2011
Member
To solve the validation related problems I commented out the 'composedCheck = "org.eclipse.xtext.validation.NamesAreUniqueValidator"' entry in the workflow and added a method for validation to the custom validator:

public class TestJavaValidator extends AbstractTestJavaValidator {


	@Check
	public void checkUniqueVariableName(VariableDeclaration variableDeclaration)
	{
		Map<Object, Object> context = getContext();

		if (context != null) {
			if (context.containsKey(variableDeclaration))
				return; // variableDeclaration was already validated
			context.put(variableDeclaration, this);
		}
		
		//find block
		EObject block = variableDeclaration.eContainer();
		while (!(block instanceof BlockExpression))
		{
			block = block.eContainer();
		}

		//check if any other variable has the same name
		String name = variableDeclaration.getName();
		for (EObject object : block.eContents())
		{
			if (object instanceof VariableDeclarations)
			{
				VariableDeclarations declarations = (VariableDeclarations) object;
				
				for (VariableDeclaration foundVariableDeclaration : declarations.getDeclarations())
				{
					if (!foundVariableDeclaration.equals(variableDeclaration) && foundVariableDeclaration.getName().equals(name))
					{
						error("Duplicate variable name '"+name+"'", variableDeclaration, TestPackage.Literals.VARIABLE_DECLARATION__NAME, -1);
					}
				}
			}
		}
		
	}
}


I think I now have enough information to implement a more complex example of this simplified version in the actual language I am working on.

Thank you for the help and pointers. If anyone has suggestions to improve the listed pieces of code please post a reply!
Re: Block scope, validation, linking and content assist [message #740429 is a reply to message #722830] Tue, 18 October 2011 13:58 Go to previous messageGo to next message
Mathieu Garcia is currently offline Mathieu GarciaFriend
Messages: 14
Registered: March 2011
Location: Tououse, France
Junior Member
Hi developers,

May someone complete this post by saying if it is possible to have a different scope when validating than in a content assist context?

I'd like to have a larger scope when using content assist and reduce this scope when validating.

Thanks.
Mathieu
Re: Block scope, validation, linking and content assist [message #741294 is a reply to message #740429] Wed, 19 October 2011 11:40 Go to previous messageGo to next message
Mathieu Garcia is currently offline Mathieu GarciaFriend
Messages: 14
Registered: March 2011
Location: Tououse, France
Junior Member
Hi, answers for my questions:

I found different solutions:

1°) With two IGlobalScopeProvider (one for validation, one for content assist)
Create a second IGlobalScopeProvider that extends an arbitrary interface such as IAlternateGlobalScopeProvider in order to bind it within modules.

When computing proposals for content assists, I get and cast the scope provider and call a specific method that will get the casted delegate on which I change the globalScopeProvider.

In this solution, the alternate global scope provider used by content assist lookup into all available files near the edited file.

The "normal" global scope provider only contains URI formally declared into the edited file. So validation, just see those ones.

2°) With predicate and a large IGlobalScopeProvider
When computing proposals for content assists, I get and cast the scope provider and call a specific method that will get the casted delegate on which I turn off a predicate.
This predicate check if external EObjects refers to an import that is formally declared.
For content assist proposal, the predicate is inactive.

3°) With a large IGlobalScopeProvider and a java validation
In this case, the linking validation won't fail so I check all imports are present at java validation time.

Don't hesitate to leave comments. I hope it might help someone.
Re: Block scope, validation, linking and content assist [message #741306 is a reply to message #740429] Wed, 19 October 2011 12:02 Go to previous messageGo to next message
Jérôme Fouletier is currently offline Jérôme FouletierFriend
Messages: 39
Registered: September 2010
Location: France
Member
Mathieu Garcia wrote on Tue, 18 October 2011 15:58
Hi developers,

May someone complete this post by saying if it is possible to have a different scope when validating than in a content assist context?

I'd like to have a larger scope when using content assist and reduce this scope when validating.

Thanks.
Mathieu

Wouldn't just writing a validator be enough? You would benefit from the added advantage of providing the user with a custom error/warning message, whereas unresolved links are always errors with the same message.
Re: Block scope, validation, linking and content assist [message #741529 is a reply to message #741306] Wed, 19 October 2011 16:08 Go to previous message
Mathieu Garcia is currently offline Mathieu GarciaFriend
Messages: 14
Registered: March 2011
Location: Tououse, France
Junior Member
Yes, that's what I suggested in solution 3 and seems to be the cleaner solution. Others rather look like hacks than customizations...
Previous Topic:Mwe2 based generation not working since last Xtext update
Next Topic:Decision can match input such as "RULE_INT" using multiple alternatives: 2, 3
Goto Forum:
  


Current Time: Tue Sep 24 01:19:08 GMT 2024

Powered by FUDForum. Page generated in 0.03299 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top