Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Retrieve index without having to load resources in standalone context?
Retrieve index without having to load resources in standalone context? [message #1732478] Tue, 17 May 2016 14:53 Go to next message
Niels Brouwers is currently offline Niels BrouwersFriend
Messages: 80
Registered: July 2009
Member
Hi,

Is there a way to be able to retrieve the index on a XtextResourceSet index in Java stand-alone configuration, without having to explicitly load the resources when adding them to the XtextResourceSet instance?

Example code:

ResourceSet rs = new XtextResourceSet();
Resource r = rs.getResource(ROOT_MODELS_URI.appendSegment("a.hddd"), false);
IResourceDescriptions index = rdp.getResourceDescriptions(rs);
Iterable<IEObjectDescription> exported = index.getExportedObjects();


In my case, the index is an object of type ResourceSetBasedResourceDescriptions. The index object doesn't contain any data. When loading the models when adding them to the resourceset, the index contains the values as expected.

What do I need to do to trigger building the index?

Thanks!


Kind regards,
Niels Brouwers.
Re: Retrieve index without having to load resources in standalone context? [message #1732488 is a reply to message #1732478] Tue, 17 May 2016 15:27 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13887
Registered: July 2009
Senior Member
in a standalone app it is your JOB to add all resources to the resourceset.
so please explain what behaviour are you expecting?


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Retrieve index without having to load resources in standalone context? [message #1732580 is a reply to message #1732488] Wed, 18 May 2016 11:05 Go to previous messageGo to next message
Jan Koehnlein is currently offline Jan KoehnleinFriend
Messages: 760
Registered: July 2009
Location: Hamburg
Senior Member
In Eclipse, the workspace is the scope for your index. In standalone, you have to define that scope yourself.
Where do you expect the index data to come from?


---
Get professional support from the Xtext committers at www.typefox.io
Re: Retrieve index without having to load resources in standalone context? [message #1732641 is a reply to message #1732580] Wed, 18 May 2016 18:25 Go to previous messageGo to next message
Niels Brouwers is currently offline Niels BrouwersFriend
Messages: 80
Registered: July 2009
Member
Thanks for your replies.

In the context of Eclipse, and please correct me if I am wrong, IResourceDescription objects are created by the Xtext builder. It loads and traverses the models it can find on it's class path to create these objects. In this scenario, the actual loading (and unloading?) of the resources is therefore performed by the builder.

I was wondering if there is an additional helper class that I could invoke that creates the IResourceDescription objects for all the resouces in the XtextResourceSet. If there was, I would expect such helper class to load the resources to create the objects, and unloads the resources when it's done. In this case, I could load only the modes in the XtextResourceSet that are of particular interest, and perform additional queries on the index for performance reasons, i.e. to find all the objects referencing elements within the loaded resources.

If such a additional helper class does not exist, I can only load all the models I can find in my 'scope' in the XtextResourceSet to populate the full index. This would potentially result in lower performance and higher memory usage, as all models need to remain loaded, right?

I realize that you might need some more explanation on the use case I aim to implement. Use case:

  • I have two sets of models; a set of 'essential' models and a set of 'decorator' models. The latter category of models contain elements that 'decorate' an element from an essential model. This is implemented by the decorating element to have a reference to the decorated element. The decorating element contains additional meta-data useful for a particular context.
  • In the entire scope, e.g. the Eclipse workspace, I could have multiple coherent set of essential models. Correspondingly, I could have a coherent set of decorator models for each coherent set of essential models.
  • When processing one coherent set of essential models, I would like to discover the corresponding coherent set of decorator models. These are identified by traversing the reference for each decorating element. If it targets an element from the coherent set of essential models, the decorator model is added to the corresponding coherent set of decorator models.
  • For performance reasons, I would like to prevent loading all the decorator models that can be found in the entire scope to be loaded in a single XtextResourceSet and that they keep being loaded. Instead, I would like these generator models to be loaded and traversed only once to build up the complete index. After building up the index, the models can be unloaded and the set of coherent generator models can be found leveraging the performance optimized Xtext index.


Can you please comment on this use case and provide some guidance how to implement this in stand-alone mode with Xtext?

Thanks!


Kind regards,
Niels Brouwers.
Re: Retrieve index without having to load resources in standalone context? [message #1836700 is a reply to message #1732641] Tue, 12 January 2021 13:09 Go to previous messageGo to next message
Manisha Ghule is currently offline Manisha GhuleFriend
Messages: 20
Registered: June 2015
Junior Member
I know this is very old thread, but I could not find any useful posts for my problem and this question is exactly what I need, so posting here.

Hi Neils,

Did you find any solution to your problem?

I want to achieve the same i.e. retrieve index without loading resource. I have multiple types of DSLs and based its references I need to generate some other DSLs. I have almost 20K model files and loading them all into a resourceSet takes hours.

I am using Xtext 2.21

Any help is really appreciated.

Thanks!

[Updated on: Tue, 12 January 2021 13:09]

Report message to a moderator

Re: Retrieve index without having to load resources in standalone context? [message #1836752 is a reply to message #1836700] Wed, 13 January 2021 10:37 Go to previous messageGo to next message
Niels Brouwers is currently offline Niels BrouwersFriend
Messages: 80
Registered: July 2009
Member
Hi Manisha,

Sorry, this is too long ago for me to remember...


Kind regards,
Niels Brouwers.
Re: Retrieve index without having to load resources in standalone context? [message #1836754 is a reply to message #1836752] Wed, 13 January 2021 10:52 Go to previous messageGo to next message
Manisha Ghule is currently offline Manisha GhuleFriend
Messages: 20
Registered: June 2015
Junior Member
Thanks for the reply.

Can anyone from the community provide any pointers/suggestions on this?
Or any pointer to make loading of large number of resources faster?
Re: Retrieve index without having to load resources in standalone context? [message #1836760 is a reply to message #1836754] Wed, 13 January 2021 12:20 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13887
Registered: July 2009
Senior Member
can you give more context where you try to do what

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Retrieve index without having to load resources in standalone context? [message #1836762 is a reply to message #1836760] Wed, 13 January 2021 12:41 Go to previous messageGo to next message
Manisha Ghule is currently offline Manisha GhuleFriend
Messages: 20
Registered: June 2015
Junior Member
Hi Christian,

I have around 15 different DSLs say DSL1..DSL15. And now introduced two more DSL16 and DSL17
I have my custom ANT task where I load all these resources into a ResourceSet. Then I use ResourceDescriptions to find references of DSL16 in DSL1..DSL15 to generate models of type DSL17

Following is the snippet from code to load resources:

resourceSet = injector.getInstance(XtextResourceSet.class);
resourceSet.setClasspathURIContext(Thread.currentThread().getContextClassLoader());
resourceSet.getLoadOptions().put(XtextResource.OPTION_RESOLVE_ALL, Boolean.TRUE);
ResourceDescriptionsProvider resourceDescriptionsProvider = injector.getInstance(ResourceDescriptionsProvider.class);
resourceDescriptions = resourceDescriptionsProvider.getResourceDescriptions(resourceSet);

// load resources from model directories
for (File file : modelDir.listFiles()) {
 Resource resource = resourceSet.getResource(URI.createFileURI(file.getAbsolutePath()), true);
}

// code to find cross refs and generation etc...


Re: Retrieve index without having to load resources in standalone context? [message #1836763 is a reply to message #1836762] Wed, 13 January 2021 12:47 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13887
Registered: July 2009
Senior Member
if you dont save the index to disk, then there wont be any chance to speed that up

tipp: dont use XtextResource.OPTION_RESOLVE_ALL
call resolve on the resourceset once all resources are added


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Retrieve index without having to load resources in standalone context? [message #1836769 is a reply to message #1836763] Wed, 13 January 2021 14:42 Go to previous messageGo to next message
Manisha Ghule is currently offline Manisha GhuleFriend
Messages: 20
Registered: June 2015
Junior Member
Thanks Christian for the tip.
Do you mean calling
EcoreUtil2.resolveAll(resourceSet);
?
But won't it take the same time to resolve all or did you mean something else?
I'll give it a try and let you know.

About saving index:
I cannot save index to disk because model files are modified across the SVN branches and are merged into main branch. Mentioned ANT task gets called from Jenkins builds which are frequently triggered after merging SVN branches.

Another strange thing I noticed:
The DSL17 model generation code (which uses ResourceDescriptions) is also called from UI (toolbar action). Generation from UI takes hardly 5-6min but the same code from ANT task takes more than an hour. What could be the possible reason?

Thanks in advance :)
Re: Retrieve index without having to load resources in standalone context? [message #1836773 is a reply to message #1836769] Wed, 13 January 2021 15:44 Go to previous message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13887
Registered: July 2009
Senior Member
there might be optimizations that dont fit in standalone as you use it
you may also check out
org.eclipse.xtext.build.IncrementalBuilder
and
org.eclipse.xtext.builder.standalone.StandaloneBuilder

which make both use of ResourceDescriptionsData


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Wed, 13 January 2021 15:51]

Report message to a moderator

Previous Topic:Beginner Grammar Question
Next Topic:Duplicate elements in ELists
Goto Forum:
  


Current Time: Tue Apr 20 08:18:22 GMT 2021

Powered by FUDForum. Page generated in 0.02015 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top