Eclipse Community Forums: TMF (Xtext) » Reduce resource consumption in project indexing

Home » Modeling » TMF (Xtext) » Reduce resource consumption in project indexing(Scope, Indexing)

Reduce resource consumption in project indexing [message #1705950]

Thu, 20 August 2015 07:29

Eclipse User

In a project with approximately 700 source files of my DSL, indexing takes a lot of resources. When the memory is exhausted, eclipse is almost idle (10% CPU) and I assume it is waiting for the garbage collector to release whatever memory it can. Until eclipse uses the available memory CPU usage is quite high. In the end, the index is never built completely. In projects with fewer files the index is built in reasonable time.

I provided a specific IDefaultResourceDescriptionStrategy implementation that indexes only objects that might be referenced by other elements of the grammar. This helped a bit but not much. I think the problem is that although my DSL uses includes and *not* namespaces, all files are indexed regardless if another file includes them.

How could I prevent the indexing of files not included by any other file? Is this the responsibility of the Scope? My Scope implementation extends AbstractDeclarativeScopeProvider. Should I extend some other class instead?

Thank you in advance for your help.

Re: Reduce resource consumption in project indexing [message #1705951 is a reply to message #1705950]

Thu, 20 August 2015 07:42

Eclipse User

Hi,

this is not possible. in the indexing Phase the files no not know what is left or right of them.
what you may do is to enable clustinging when indexing (DynamicResourceClusteringPolicy as IResourceClusteringPolicy )

Re: Reduce resource consumption in project indexing [message #1705956 is a reply to message #1705951]

Thu, 20 August 2015 08:18

Eclipse User

Hey Anakreontas,

first of all: Why don't you use namespaces? They are both more efficient and more user friendly than uri imports.

700 files doesn't sound like that much. Have you actually profiled to see where the time is spent?

How many elements are in the index? How much memory do you give to the workbench?

Also note that the index only needs to contain elements that are *globally* referencable. For example: For Java you don't need to index fields and methods, because they are always referenced in the context of a type. So only top-level types need to be indexed.

Cheers,
Stefan

Re: Reduce resource consumption in project indexing [message #1705979 is a reply to message #1705956]

Thu, 20 August 2015 12:03

Eclipse User

Hello Stefan.

The DSL is quite old and I can't change the design.

Could you point to documentation about querying the index for the number of elements it contains?
I launch the workbench with these parameters "-Xms40m -Xmx4048m -XX:MaxPermSize=1024m".

Re: Reduce resource consumption in project indexing [message #1705981 is a reply to message #1705979]

Thu, 20 August 2015 12:15

Eclipse User

You can use the open model element dialog for that

Re: Reduce resource consumption in project indexing [message #1705985 is a reply to message #1705981]

Thu, 20 August 2015 12:48

Eclipse User

It reports 11.010 objects. The index building was not complete but I opened the dialog multiple times and was reporting the same number.

I am trying to profile eclipse running the plugin. Got a snapshot of the memory with jmap but jhat run out of memory when processing the file (1.3 GB). jmap -histo:live reported that most of the memory was allocated for LeafNode and CompositeNode instances. I am running jhat again with more memory to get better results on memory usage,

Re: Reduce resource consumption in project indexing [message #1705987 is a reply to message #1705985]

Thu, 20 August 2015 13:23

Eclipse User

did you try to use

package org.xtext.example.mydsl.ui;

import org.eclipse.xtext.resource.clustering.DynamicResourceClusteringPolicy;
import org.eclipse.xtext.resource.clustering.IResourceClusteringPolicy;

import com.google.inject.Binder;
import com.google.inject.Module;

@SuppressWarnings("restriction")
public class OverridingGuiceModule implements Module {

	@Override
	public void configure(Binder binder) {
		binder.bind(IResourceClusteringPolicy.class).to(DynamicResourceClusteringPolicy.class);

	}

}

 <extension
        point="org.eclipse.xtext.ui.shared.overridingGuiceModule">
     <module
           class="org.xtext.example.mydsl.ui.OverridingGuiceModule">
     </module>
  </extension>

package org.xtext.example.mydsl.ui;

import org.eclipse.ui.plugin.AbstractUIPlugin;
import org.eclipse.xtext.resource.clustering.DynamicResourceClusteringPolicy;
import org.eclipse.xtext.resource.clustering.IResourceClusteringPolicy;

@SuppressWarnings("restriction")
public class MyDslUiModule extends org.xtext.example.mydsl.ui.AbstractMyDslUiModule {
	public MyDslUiModule(AbstractUIPlugin plugin) {
		super(plugin);
	}
	
	public Class<? extends IResourceClusteringPolicy> bindIResourceClusteringPolicy() {
		return DynamicResourceClusteringPolicy.class;
	}
}

as i proposed

Re: Reduce resource consumption in project indexing [message #1706060 is a reply to message #1705987]

Fri, 21 August 2015 09:26

Eclipse User

Hello Chrisitan.

I tried that but still the indexing would not complete. The DSL uses file extensions as a convention for files that are meant to be included. Myy implementation of DefaultResourceDescriptionStrategy returns false in the createEObjectDescriptions and createReferenceDescriptions methods for files that have a different extension. Since file extensions are just a convention I did not enforce this rule before. Now indexing is quite faster because it has to process fewer files. However, whenever the user edits a file and saves it, the index start building again and the user is interrupted from continuing to edit the file because eclipse either becomes unresponsive or very sluggish for 1-2 seconds.

Would you suggest any resources about profiling xtext languages or eclipse plugins? I need to investigate which parts of the code consumes most resources.

Re: Reduce resource consumption in project indexing [message #1706081 is a reply to message #1706060]

Fri, 21 August 2015 18:06

Eclipse User

Perhaps I have misunderstood the role of the Scope.

My scope implementation derives from AbstractDeclarativeScopeProvider. I replaced it with an empty descendant of AbstractDeclarativeScopeProvider and surprisingly almost everything worked fine. I could for example jump to the implementation of function or the declaration of variables, a service I thought was possible because of my Scope implementation. Moreover, memory consumption is fairly reasonable and Eclipse is responsive.

Perhaps a specialized scope implementation is needed for namespace based resolution of methods and variables.

Re: Reduce resource consumption in project indexing [message #1706084 is a reply to message #1706081]

Fri, 21 August 2015 18:33

Eclipse User

Xtext has quite nice out of the box behavior for scoping. So it's normal that many things work even with no special scope implementation =)

It seems your scope provider contains some very inefficient code. For more suggestions we'd need to see the full Grammar and ScopeProvider.

Re: Reduce resource consumption in project indexing [message #1818260 is a reply to message #1706084]

Wed, 11 December 2019 01:51

Eclipse User

Hello Anakreontas ,
I am also facing same kind of memory issue. Can you tell me more about empty descendant of AbstractDeclarativeScopeProvider?

Re: Reduce resource consumption in project indexing [message #1818336 is a reply to message #1818260]

Thu, 12 December 2019 12:28

Eclipse User

Simply have a look at the scope provider you have and the one the new xtext project wizard creates

Are you sure you have a subclass of abstractdeclarativescopeprovider

If not you won't profit

If you might change it the way the wizard creates it for new projects

(You have to customize getScope and ifelse
Instead of using scope_ methods

Please also measure where your memory is actually spent

Previous Topic:	Xtext / Xtend 2.20 release
Next Topic:	XbaseCompiler does not generate lambdas when running from xtext-maven-plugin

Goto Forum:

-=] Back to Top [=-

Current Time: Sat Nov 08 11:05:35 EST 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter