Eclipse Community Forums: TMF (Xtext) » Mixin Xtext with other models

Home » Modeling » TMF (Xtext) » Mixin Xtext with other models

Mixin Xtext with other models [message #1787570]

Tue, 29 May 2018 05:20

Eclipse User

Hi,

currently we are working on our first Xtext project. I've read Xtext docs, "Implementing Domain-Specific Languages with Xtext and Xtend 2ed" book and other Internet articles but I couldn't find answers to my questions.

First let me shortly describe what we would like to achieve. We are building web app (xtext+lsp+monaco) where user can upload a file (xlsx or xml) and have special editor to manipulate/calculate file contents.

For example if I upload a xlsx:

|columnX|columnY|columnZ|
|10 |20 |30 |

and if I write an expression: (row[1].columnX+row[1],columnY)/row[1].columnZ
the evaluator should return: 1; (10+20)/30

We started writing Xtext grammar for our expression language - this part is not so hard because there is a lot of examples (book & internet).

Currently we are working on content assist (if I type "row[1]." the editor should suggest: columnX, columnY, columnZ) and type checking (if I type "row[1].columnTEST" the editor should show error because our sample Excel contains only 3 columns)

Here problems starts :), below are our supposition/assumptions:
1)As far as I understand what we would like to achieve (content assist and type checking) in our case is related with cross-references
2)The expression grammar (or ecore model) should reference ecore objects representing uploaded files (excel columns or xml structure) to let content assist/type checking work properly
3)The xtext index should contain ecore objects for file(xlsx/xml) structures
4)We should provide xtext scope implementation/configuration to limit valid suggestions in specific context.

If someone could briefly describe how to achieve these goals (if they are correct) I will be grateful. Bellow are questions related to above assumptions

Ad2)Our grammar is only for typing expressions, there are no language constructs allowing to define data structures -> generated ecore model does not define classes to describe xlsx/xml file structure (we cannot reference to this object in our grammar). Do we have to add to the generated model such classes and turn of auto-generation of model from our grammar (import instead of generate) or there is another way ? (create separate model for file structure and import this model into grammar - leaving auto-generation of expression model ?)

Ad3)How to fill the Xtext index with ecore objects (EObjectDescription) when user uploads a file (xlsx/xml) describing structure of this file (existing columns in Excel or tags in XML)? This file is not our "source file" - there are no expression to parse, no AST is build, nor xtext index is filled with ecore objects. I suspect we should do this with GlobalScope and some custom ResourceDescription ? This part it the less known for us.

Sorry if this questions are a little chaotic but we don't know Xtext and EMF internals very well and we fell like we are walking around in a fog ;)

Regards
Daniel

Re: Mixin Xtext with other models [message #1789897 is a reply to message #1787570]

Thu, 31 May 2018 10:54

Eclipse User

the term to research on beeing able to reference non xtext resources is IResourceServiceProviders. you can finds blog posts on that topic e.g. here https://typefox.io/linking-xtext-models-with-other-emf-models and here https://www.dietrich-it.de/xtext/uml/iresourceserviceprovider/2011/07/17/xtext-2-0-and-uml.html

you need basically to provide a emf metamodel for e.g. xls files and a resource service provider so that the files are indexed and referrable from xtext

Re: Mixin Xtext with other models [message #1790322 is a reply to message #1789897]

Fri, 08 June 2018 07:34

Eclipse User

Hi,

thanks for replay, it helped a lot. We have one more issue/question - how can we add to XText LSP Server "static" (library) resource ?

In stand-alone (main) mode we can retrieve ResourceSet and add this resource to the set. However with LSP things are a little complicated and we don't know how to get correct resource set ? Should we get resource set from the injector or we should do this with custom ProjectManager like here: https://github.com/eclipse/xtext-core/issues/589

Daniel

Re: Mixin Xtext with other models [message #1790323 is a reply to message #1790322]

Fri, 08 June 2018 07:37

Eclipse User

i think the project manager is the way i would go

Re: Mixin Xtext with other models [message #1790327 is a reply to message #1790323]

Fri, 08 June 2018 07:46

Eclipse User

Thank you.

Do you know were we can find some examples and documentation about XText LSP integration ?

For example ProjectManager was only mentioned in this git hub issue.

I didn't find any information about Xtext LSP in Xtext book or official documentation.

Daniel

Re: Mixin Xtext with other models [message #1790328 is a reply to message #1790327]

Fri, 08 June 2018 07:47

Eclipse User

no i dont know. lsp usage is not really wide spread so far

Re: Mixin Xtext with other models [message #1790346 is a reply to message #1790328]

Fri, 08 June 2018 09:15

Eclipse User

One more question, I dont understand how to use ProjectManager to load "virtual" resources. By virtual I mean, not stored as physical files on disk but rather generated on the fly (ie. from DB).

Our resources have URI like file://resource1.test and we add this URI in ProjectManager subclass (like in this example https://github.com/eclipse/xtext-core/issues/589).

And thats all :(, file is not loaded -> cross references not work in the editor. I suppose that we should provide(how?) InputStream to the resource.load(..) method, but I don't know how to do this ?

Do we have to use some custom URI protocol like indb://resource1.test (instead of file://.. )and configure ?some? provider? to translate URI -> InputStream ?

Do you have any suggestion were should we look for solution/examples ? Maybe I should start another thread on this/other forum ?

Regards
Daniel

Re: Mixin Xtext with other models [message #1790354 is a reply to message #1790346]

Fri, 08 June 2018 09:52

Eclipse User

i still dont understand which files are physical and which are not.
in the example from the issue it is about physical files.

the magic is done here:

override Result doInitialBuild(CancelIndicator cancelIndicator) {
		val uris = newArrayList
		projectConfig.sourceFolders.forEach [
			fileSystemScanner.scan(path)[uris += it]
		]
		val r = MyDslLib.classLoader.getResource(MyDslLib.LIB)
		uris+=URI.createURI(r.toString)
		return doBuild(uris, emptyList, emptyList, cancelIndicator)
	}

it loads the files from the classpath.

unfortualtely there is no easy shiny way to do anything here.
you would have to teach the doBuild to be able to work with your other uris

=> i assume you need a custom uri scheme and converter as you would do it outside of lsp (e.g. with InMemoryURIHandler)
=> you then would have to make sure this handler is used with the resourceset org.eclipse.xtext.ide.server.ProjectManager.createNewResourceSet(ResourceDescriptionsData)
=> or you adopt the resource loading org.eclipse.xtext.build.ClusteringStorageAwareResourceLoader.executeClustered(Iterable<URI>, (Resource)=>T)

Re: Mixin Xtext with other models [message #1790588 is a reply to message #1790354]

Wed, 13 June 2018 05:51

Eclipse User

Thanks, it works. One more question.

What is the most simple way of retriving resource by its URI ?
My resources represents tree structures. In my ScopeProvider I want to add root of this structure (Descrption of EObject) to the scope knowing only the resource URI.

Re: Mixin Xtext with other models [message #1790592 is a reply to message #1790588]

Wed, 13 June 2018 06:28

Eclipse User

hi,

i dont get this question. can you be more precise on that?

Re: Mixin Xtext with other models [message #1790596 is a reply to message #1790592]

Wed, 13 June 2018 07:23

Eclipse User

Sure,

We have two kind (types) of EMF Resources:
1)XText Resource with our expression DSL - this resources are editable via Monaco and managed by XText language server (users are writing simple scripts via http form)
2)Our "custom", in-memory resources with data structures- we wrote IResourceServiceProvider so we can cross-reference to objects from these resources from our grammars. Also these objects are added to Xtext Index so validation and ContentAssist works.

Now we want to adjust XText behaviour to our needs.
Our Expression DSL allow to write simple path expression (ie "/a/b/c"). We want to write our ScopeProvider to make content assist and validation works correctly.

So, if I have Resource with such EObject structure:
-aNode
--bNode (child of aNode)
----cNode (child of bNode)

and user enters "/" and press content assist hot key (ctrl+space) I want content assist to suggest only "aNode". Currently (with default ScopeProvider) all three objects are suggested (aNode, bNode and cNode). I don't know name of this "root" object. I know only that this object is root object in resource with URI "xyz:/dataStructure1.ds" (URI is static and not changing).

Finally, in our ScopeProvider I would like to write:

Resource resource = ...............
Node root = ((Node) resource.getAllContents().next());
List<Node> elements = new ArrayList<Node>();
elements.add(root.getChilds().get(0));
return Scopes.scopeFor(elements);

but I don't know how to acces resource by URI (URI of resource is known). I suspect that it is trivial operation, but I couldn't find any working example.

I tried with injecting ResourceSet and getting resource from this set, but I suspect that resourceset is managed by different Inecjtor(LSP?) then used by ScopeProvider ->injected resource set was empty :(

Re: Mixin Xtext with other models [message #1790608 is a reply to message #1790596]

Wed, 13 June 2018 09:57

Eclipse User

- ask the context for resource and that for resourceset
- ask the resourceset for resource with uri with trimmed fragment

but: if the referenced resource is in the index anyway. why no filtering global scope
=> i am not sure if what you try to do is a bad idea

Re: Mixin Xtext with other models [message #1790659 is a reply to message #1790608]

Thu, 14 June 2018 03:01

Eclipse User

Thanks, it works, but for me it is a little bit strange, that the simplest way of accessing resourceset is by navigating from another resource ;) (I expected some form of injection/provider/factory/registry).

I thought about filtering global index but I don't have enough information in the index about my nodes to do this efficient (without navigating from object descrition to eobject). As I said my file is tree-like structure with a lot of nodes. All this nodes are in the index (to allow crossreference from expression dsl). When user types "/" content assist should suggest only root of this tree. I don't know how to check in the index if a node is root node or child ? (all nodes, including root have the same Ecore type and I don't know name of the root). That's why we are creating scope for "/" by getting root object from resource.

Re: Mixin Xtext with other models [message #1790662 is a reply to message #1790659]

Thu, 14 June 2018 03:08

Eclipse User

you can adapt the indexing and add additional information to the index entries userdata. in this case you would store the information if a node i a root.
then you would simply filter for that flag

Re: Mixin Xtext with other models [message #1790731 is a reply to message #1790662]

Fri, 15 June 2018 03:34

Eclipse User

OK, I didn't know that I can add extra info to the index. I will try.

Thanks once again for all your help and quick responses.

Previous Topic:	Caches in Code Generator
Next Topic:	Compilation error inside the grammar access class when upgrading to Xtext 2.14

Goto Forum:

-=] Back to Top [=-

Current Time: Tue Jul 15 00:55:52 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter