Eclipse Community Forums: TMF (Xtext) » Custom scope provider

Home

Home » Modeling » TMF (Xtext) » Custom scope provider

Custom scope provider [message #1859690]

Thu, 22 June 2023 11:26

Eclipse User

Hi,

I'm trying to implement my custom scope provider for my own DSL.
Here is a part of it's definition:

VariableDeclarationNode:
	{VariableDeclarationNode} name=NodeId '=' target=VariableDeclarationTarget
;

VariableDeclarationTarget:
	ComponentNode;

RelationNode:
    {RelationNode} (source=[VariableDeclarationNode])? '->' target=[VariableDeclarationNode] (description=STRING (technology=STRING (tags+=STRING*))?)?
;

The RelationNode holds two references so in the end I would be capable of doing something like this:

component1 = component "SomeComponent"
component2 = component "Another"
component1 -> component2

It works well for very simple cases like this but the feature I'm working right now is a possilibty to "include" elements from other resources like this:

!import "classpath:/component1.xxx"
component2 = component "Another"
component1 -> component2

where component1.xxx would contain:

component1 = component "SomeComponent"

I've prepared a resolver mechanism that is capable of reading model from given URI, then I traverse trough the elements of the loaded model and if I find the VariableDeclaration then I add it to the scope for Relation.target and Relation.source.

This works well in tests, but, when I start using generated LSP it all goes crazy - when I first time import the file and write the relation immedietaly then it kinda.... works, but any additional edit or restarting the IDE instance that uses this LSP ends up with LSP unable to find component1 declaration.

What I might be doing wrong? When I'm using the default scope provider everything works pretty well but the variables are available globally and even when i dont use !include keyword.

Please help :(
I can share some more code if necessary...

Re: Custom scope provider [message #1859692 is a reply to message #1859690]

Thu, 22 June 2023 13:41

Eclipse User

Is there a reason you implement the import thing on your own.
And how do you do it?

Re: Custom scope provider [message #1859693 is a reply to message #1859692]

Thu, 22 June 2023 13:57

Eclipse User

Basically I'm rewriting this DSL: https://structurizr.com/dsl or at least trying ;)
I have a custom component I call IncludeResolver, I used it in both validator and scope provider, so in short the method that loads the included model looks like following:

public Model resolveInclude(IncludeNode include,
                                CircularIncludeGuard circularIncludeGuard,
                                ResourceSet resourceSet) throws IncludeError {
        if (circularIncludeGuard.alreadyIncluded(include)) {
            throw new IncludeError("Circular includes are forbidden!");
        }
        String importURI = include.getImportURI();
        URI uri = URI.createURI(importURI);
        try {
            Resource resource = resourceSet.getResource(uri, true);
            circularIncludeGuard.register(include);
            for (EObject importedObject : resource.getContents()) {
                if (importedObject instanceof Model includedModel) {
                    return includedModel;
                }
            }
            throw new IncludeError("Unable to find model in included resource.");
        } catch (Throwable throwable) {
            throw new IncludeError(throwable);
        }
    }

CircularIncludeGuard is just checking if there are circular includes going on...

I'm including whole resolver code here:

public class IncludeResolver {

    public Model resolveModel(Model originalModel) throws IncludeError {
        ResourceSet resourceSet = originalModel.eResource().getResourceSet();
        Model clonedModel = EcoreUtil.copy(originalModel);
        doLinkingInModel(clonedModel, new CircularIncludeGuard(), resourceSet);
        return clonedModel;
    }

    public Model resolveInclude(IncludeNode include,
                                CircularIncludeGuard circularIncludeGuard,
                                ResourceSet resourceSet) throws IncludeError {
        if (circularIncludeGuard.alreadyIncluded(include)) {
            throw new IncludeError("Circular includes are forbidden!");
        }
        String importURI = include.getImportURI();
        URI uri = URI.createURI(importURI);
        try {
            Resource resource = resourceSet.getResource(uri, true);
            circularIncludeGuard.register(include);
            for (EObject importedObject : resource.getContents()) {
                if (importedObject instanceof Model includedModel) {
                    return includedModel;
                }
            }
            throw new IncludeError("Unable to find model in included resource.");
        } catch (Throwable throwable) {
            throw new IncludeError(throwable);
        }
    }

    private void doLinkingInModel(Model model,
                                  CircularIncludeGuard circularIncludeGuard,
                                  ResourceSet resourceSet) throws IncludeError {
        ArrayList<NodeType> nodeTypes = new ArrayList<>(model.getElements());
        for (NodeType modelNode : nodeTypes) {
            linkModelNode(model, modelNode, circularIncludeGuard, resourceSet);
        }
    }

    private void linkModelNode(Model model,
                               EObject modelNode,
                               CircularIncludeGuard circularIncludeGuard,
                               ResourceSet resourceSet) throws IncludeError {
        if (modelNode instanceof VariableDeclarationNode variableDeclaration) {
            linkModelNode(model, variableDeclaration.getTarget(), circularIncludeGuard, resourceSet);
        }
        if (modelNode instanceof IncludeNode include) {
            includeResource(new ModelIncludeTarget(model), include, circularIncludeGuard, resourceSet);
        } else if (modelNode instanceof GroupNode group) {
            doLinkingInGroup(group, circularIncludeGuard, resourceSet);
        } else if (modelNode instanceof ContainerNode container) {
            doLinkingInContainer(container, circularIncludeGuard, resourceSet);
        } else if (modelNode instanceof SoftwareSystemNode softwareSystem) {
            doLinkingInSoftwareSystem(softwareSystem, circularIncludeGuard, resourceSet);
        }
    }

    private void doLinkingInGroup(GroupNode group,
                                  CircularIncludeGuard circularIncludeGuard,
                                  ResourceSet resourceSet) throws IncludeError {
        List<NodeType> groupElements = new ArrayList<>(group.getElements());
        for (NodeType groupElement : groupElements) {
            if (groupElement instanceof IncludeNode include) {
                includeResource(new GroupIncludeTarget(group), include, circularIncludeGuard, resourceSet);
            } else if (groupElement instanceof GroupNode childGroup) {
                doLinkingInGroup(childGroup, circularIncludeGuard, resourceSet);
            }
        }
    }

    private void doLinkingInContainer(ContainerNode container,
                                      CircularIncludeGuard circularIncludeGuard,
                                      ResourceSet resourceSet) throws IncludeError {
        List<NodeType> containerElements = new ArrayList<>(container.getElements());
        for (NodeType containerElement : containerElements) {
            if (containerElement instanceof IncludeNode include) {
                includeResource(new ContainerIncludeTarget(container), include, circularIncludeGuard, resourceSet);
            } else if (containerElement instanceof GroupNode childGroup) {
                doLinkingInGroup(childGroup, circularIncludeGuard, resourceSet);
            }
        }
    }

    private void doLinkingInSoftwareSystem(SoftwareSystemNode softwareSystem,
                                           CircularIncludeGuard circularIncludeGuard,
                                           ResourceSet resourceSet) throws IncludeError {
        ArrayList<NodeType> softwareSystemElements = new ArrayList<>(softwareSystem.getElements());
        for (NodeType softwareSystemElement : softwareSystemElements) {
            if (softwareSystemElement instanceof IncludeNode include) {
                includeResource(new SoftwareSystemIncludeTarget(softwareSystem), include, circularIncludeGuard, resourceSet);
            } else if (softwareSystemElement instanceof GroupNode childGroup) {
                doLinkingInGroup(childGroup, circularIncludeGuard, resourceSet);
            }
        }
    }

    private void includeResource(IncludeTarget includeTarget,
                                 IncludeNode include,
                                 CircularIncludeGuard circularIncludeGuard,
                                 ResourceSet resourceSet) throws IncludeError {
        Model includedModel = resolveInclude(include, circularIncludeGuard, resourceSet);
        includeModel(includeTarget, includedModel, include, circularIncludeGuard, resourceSet);
    }

    private void includeModel(IncludeTarget includeTarget,
                              Model includedModel,
                              IncludeNode includeNode,
                              CircularIncludeGuard circularIncludeGuard,
                              ResourceSet resourceSet) throws IncludeError {
        List<NodeType> newElements = new ArrayList<>(includeTarget.cloneElements());
        int includeNodeIndex = newElements.indexOf(includeNode);
        int includeElementIndex = includeNodeIndex;
        for (NodeType includedNode : includedModel.getElements()) {
            if (includedNode instanceof IncludeNode nestedInclude) {
                Model nestedIncludedModel = resolveInclude(nestedInclude, circularIncludeGuard, resourceSet);
                Model resolved = resolveModel(nestedIncludedModel);
                for (NodeType nestedNode : resolved.getElements()) {
                    newElements.add(++includeElementIndex, nestedNode);
                }
            } else if (!newElements.contains(includedNode)) {
                if (includeNodeIndex != -1) {
                    newElements.add(++includeElementIndex, includedNode);
                } else {
                    newElements.add(includedNode);
                }
            }
        }
        if (includeNodeIndex != -1) {
            newElements.remove(includeNodeIndex);
        }
        includeTarget.replaceElements(newElements);
    }
}

Re: Custom scope provider [message #1859694 is a reply to message #1859693]

Thu, 22 June 2023 13:58

Eclipse User

I have my code on gitlab.com so I can share it if needed (it's a private repo tho)

Re: Custom scope provider [message #1859695 is a reply to message #1859694]

Thu, 22 June 2023 14:03

Eclipse User

This looks totally unusual to me so you need to debug why it does not work on incremental builds
I assume you don't calculate a proper uri to load manually into the resource set or even just search it in the resource set where it won't be there on incremental, builds

Unfortunately I don't have time to debug big piles of code

But again I still won't understand why you do the imports yourself instead of using Xtexts ootb

I also have doubts classpath Uris work properly
In lsp

Maybe you can also write a test for incremental build in lsp to ease debugging

[Updated on: Thu, 22 June 2023 14:11] by Moderator

Re: Custom scope provider [message #1859696 is a reply to message #1859695]

Thu, 22 June 2023 14:15

Eclipse User

The classpath uris works pretty well, in fact everything works fine with unit tests (I'm just using ParseHelper), the problem starts with LSP. I'm very new in xtext and most probably I'm doing some rookie mistakes :/ Probably I'm missing some basic understanding of the process I'm trying to participate in. Nevertheless thanks for replying :)

Re: Custom scope provider [message #1859697 is a reply to message #1859696]

Thu, 22 June 2023 14:21

Eclipse User

I propose you create a unit test using lsp or the builder directly
https://github.com/eclipse/xtext/tree/main/org.eclipse.xtext.tests/src/org/eclipse/xtext/build
https://github.com/eclipse/xtext/tree/main/org.eclipse.xtext.ide.tests/src/org/eclipse/xtext/ide/tests/server

Might give some starting points.
Unfortunately there is no perfect test to start with

Maybe you can also boil this down to a mimimalst examples

Did you manage to debug lsp

[Updated on: Thu, 22 June 2023 14:22] by Moderator

Re: Custom scope provider [message #1859698 is a reply to message #1859697]

Thu, 22 June 2023 14:26

Eclipse User

As it comes to debugging LSP itself - yeah, i was trying to debug it remotely today but I was having some issues with that, will give that another try ...
Thanks for the links :) I will have a closer look.
Btw, can you share any links where I can read about incremental build?

Re: Custom scope provider [message #1859699 is a reply to message #1859698]

Thu, 22 June 2023 14:30

Eclipse User

i fear not. there is not dedicated docs on it.

Re: Custom scope provider [message #1859700 is a reply to message #1859699]

Thu, 22 June 2023 15:03

Eclipse User

did some experiments with an incremental build test

package org.xtext.example.mydsl.tests;

import java.util.ArrayList;
import java.util.List;

import org.eclipse.lsp4j.Diagnostic;
import org.eclipse.lsp4j.DidChangeTextDocumentParams;
import org.eclipse.lsp4j.DidOpenTextDocumentParams;
import org.eclipse.lsp4j.Position;
import org.eclipse.lsp4j.Range;
import org.eclipse.lsp4j.TextDocumentContentChangeEvent;
import org.eclipse.lsp4j.TextDocumentItem;
import org.eclipse.lsp4j.VersionedTextDocumentIdentifier;
import org.eclipse.xtext.ide.server.UriExtensions;
import org.eclipse.xtext.testing.AbstractLanguageServerTest;
import org.eclipse.xtext.util.Strings;
import org.eclipse.xtext.xbase.lib.Extension;
import org.eclipse.xtext.xbase.lib.IterableExtensions;
import org.eclipse.xtext.xbase.lib.ObjectExtensions;
import org.eclipse.xtext.xbase.lib.Procedures.Procedure1;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.Assertions;
import com.google.common.collect.Iterables;
import com.google.common.collect.Lists;
import com.google.inject.Inject;

public class DemoTest extends AbstractLanguageServerTest {

	public DemoTest() {
		super("mydsl");
	}

	@Inject
	@Extension
	private UriExtensions _uriExtensions;

	@Test
	public void testIncrementalBuild() {
		String _uriString = this._uriExtensions.toUriString(this.root.toURI().normalize());
		initialize();
		open(_uriString + "a.mydsl", "Hello A!");
		open(_uriString + "b.mydsl", "Hello B from A!");

		List<Diagnostic> problems = problems();
		Assertions.assertTrue(problems.size() == 0, problems.toString());
		TextDocumentContentChangeEvent c1 = new TextDocumentContentChangeEvent();
		c1.setRange(new Range(new Position(0, 0), new Position(0, 0)));
		c1.setText("Hello C from B!");
		ArrayList<TextDocumentContentChangeEvent> changes = Lists.newArrayList(c1);
		change(_uriString + "b.mydsl", 2, changes);
		problems = problems();
		Assertions.assertTrue(problems.size() == 0, problems.toString());

		TextDocumentContentChangeEvent c2 = new TextDocumentContentChangeEvent();
		c2.setRange(new Range(new Position(0, 0), new Position(0, 0)));
		c2.setText("Hello D from Does_Not_Exist!");
		changes = Lists.newArrayList(c2);
		change(_uriString + "b.mydsl", 3, changes);
		problems = problems();
		Assertions.assertTrue(problems.size() == 1, problems.toString());
		System.out.println(problems.toString());

	}

	protected void change(final String fileUri, int v, List<TextDocumentContentChangeEvent> changes) {
		DidChangeTextDocumentParams p = new DidChangeTextDocumentParams();
		VersionedTextDocumentIdentifier i = new VersionedTextDocumentIdentifier();
		i.setUri(fileUri);
		i.setVersion(v);
		p.setTextDocument(i);
		p.setContentChanges(changes);
		this.languageServer.didChange(p);
	}

	private List<Diagnostic> problems() {
		return Lists.newArrayList(IterableExtensions.flatten(getDiagnostics().values()));
	}

}

Re: Custom scope provider [message #1859717 is a reply to message #1859700]

Fri, 23 June 2023 03:34

Eclipse User

Thanks :) I will have a closer look at this.

Re: Custom scope provider [message #1859749 is a reply to message #1859717]

Sat, 24 June 2023 16:15

Eclipse User

Hi again, I spent some additional time today and I've realized I'm using Xtext completely wrong way... decide to review my DSL definition and do it correctly this time ;) Thank you again for your help.

Previous Topic:	Encountering EProxies after building projects in new workspace
Next Topic:	Xtext 2.32.0.M0 is out

Goto Forum:

-=] Back to Top [=-

Current Time: Sat Apr 19 05:41:42 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter