Eclipse Community Forums: TMF (Xtext) » Xtext Build and Indexing taking more time

Home » Modeling » TMF (Xtext) » Xtext Build and Indexing taking more time(Xtext index performance issue )

Xtext Build and Indexing taking more time [message #1832350]

Thu, 17 September 2020 01:08

Eclipse User

Hello All,

When project build happen, indexing taking lot of time. build time increased, not allowed to work on the DSL until build completes.

The grammar size little big[1100 lines] and contains some cross reference rules.

Project also contains more number of DSLs[more than 3000 dsl files], so when build happen on this project it takes around 18-20 min for complete build
including [parsing+linking+validation+generation]. if i exclude generation it reduces for 2 min.

Need some ideas/best methodologies to improvise the build and index performance.

Thank you

Regards
Shashi

Re: Xtext Build and Indexing taking more time [message #1832351 is a reply to message #1832350]

Thu, 17 September 2020 01:11

Eclipse User

hi,

you should first start profiling to see "where" the performance is lost.
is it calculation or is it writing file

(e.g. writing thousands of files to a network drive might become terrible slow and there is nothing to optimize there)

same for reading files

=> measure where the time is spent

[Updated on: Thu, 17 September 2020 01:11] by Moderator

Re: Xtext Build and Indexing taking more time [message #1832353 is a reply to message #1832351]

Thu, 17 September 2020 01:44

Eclipse User

Hello Christain,

Thanks for your suggestion.

No not writing any files to network drive and not reading any resources from network drive.

Did a profiling and identified that the doUpdate method of ClusteringBuilderState is taking more time.

Thank you

Re: Xtext Build and Indexing taking more time [message #1832357 is a reply to message #1832353]

Thu, 17 September 2020 02:06

Eclipse User

doUpdate is all
- parsing
- linking
-validation

we need to know the subtaks like

- writeNewResourceDescriotion
- updateMarkers
- resolveLazyCrossReferences

Re: Xtext Build and Indexing taking more time [message #1832361 is a reply to message #1832357]

Thu, 17 September 2020 02:58

Eclipse User

Hello Christain,

Thank you very much.

Number of DSL files in the current project = 1850
The writeNewResourceDescriotion method is taking around 32 seconds.

Aafter this there is a execution step for

- resolveLazyCrossReferences
- updateMarkers

// Step 6: Iteratively got through the queue. For each resource, create a new resource description and queue all depending
// resources that are not yet in the delta. Validate resources. Do this in chunks.

Both the methods are not taking much time to process the individual resource.

But there is a while loop to process all the resources for crossReferences resolution and marker update, so this loop actually taking very long time.

The below loop taking more time, it takes more than 10 min.
Loop :
Queue<URI> queue = buildData.getURIQueue();
while (!queue.isEmpty()) {
- resolveLazyCrossReferences
- updateMarkers
}

Thank you

Re: Xtext Build and Indexing taking more time [message #1832362 is a reply to message #1832361]

Thu, 17 September 2020 03:04

Eclipse User

Hi

I find that the Xtext indexing is very enthusiastic about indexing anything that might just possibly be a model, regardless of whether there is any active client that could use it.

I therefore minimize the number of projects that have an Xtext nature by migrating all the Xtext mwe2/build functionality into a separate ...build plugin. This also has the benefit of reducing the footprint of the delivered Xtext editor.

But don't let me discourage your profiling. It seems that you may be close to a repro for index/update performance issues that have been outstanding for far too long.

Regards

Ed Willink

Re: Xtext Build and Indexing taking more time [message #1832364 is a reply to message #1832362]

Thu, 17 September 2020 03:15

Eclipse User

Hello Ed,

Thank you,

What does it mean. could you please elaborate more on your below statement.

Minimize the number of projects that have an Xtext nature by migrating all the Xtext mwe2/build functionality into a separate ...build plugin.

How that actually helps for performance improvement? any examples will definitely help for better understanding.

Thank you

Re: Xtext Build and Indexing taking more time [message #1832394 is a reply to message #1832364]

Thu, 17 September 2020 07:55

Eclipse User

Hi,

Any inputs/suggestions are highly appreciated.

Thank you

Re: Xtext Build and Indexing taking more time [message #1832396 is a reply to message #1832394]

Thu, 17 September 2020 08:02

Eclipse User

again: can you provide actual measured data on what on the build takes time.
there can be

- parallel parsing
- reduce number of index object / tree traversing on indexing
- fix bad O(nsquared) validation algos
- fix bad scoping
etc

Re: Xtext Build and Indexing taking more time [message #1832475 is a reply to message #1832396]

Fri, 18 September 2020 09:55

Eclipse User

Hi

See https://bugs.eclipse.org/bugs/show_bug.cgi?id=420710#c2 and https://bugs.eclipse.org/bugs/show_bug.cgi?id=486083 and its see alsoes for some history on this problem. The problem seems so old that a number of Bugzillas have been auto-CLOSED WONTFIX.

My attempts to debug the build recursions failed because I couldn't understand the code, or install a debuggable version to instrument usefully or find the developer's debug ".options".

I therefore gave up and adjusted my working practices to minimize the pain caused by the defective index, that I never use, and/or defective auto-builder that should only build once. I also minimize my use of Xtend to just M2T StringTemplates..

You can see my restructuring in the org.eclipse.ocl.examples.build plugin that houses the MWE2 build procedures and dependencies for the five/six org.eclipse.ocl.xtext.... editors. The editor plugins do NOT have an Xtext nature. Test plugins do NOT have an Xtext nature to prevent *.uml files burning time even though I know of no Xtext-based UML tooling that requires an index. Only org.eclipse.ocl.examples.build has an Xtext nature which much reduces the opportunities for Xtext build loops. Beware of checking out org.eclipse.emf.core since that now has an Xtext nature.

My observations suggested that the dreadful performance was intermittent, perhaps caused by accidental GIT checkout timing, perhaps caused by indeterminate Set ordering in worklists. If you have a solid repro please put it forward for investigation.

The solution might be as trivial as adding a validation warning to alert the user to erroneous Xtext/Java build nature ordering. Since Xtext depends on the Index which depends on Java, I never know what the correct order is or even if there is a correct order. Perhaps the bug is differently a failure of a builder to detect nothing was actually built and so that no timestamps are to be changed and so provoke a loop.

Regards

Ed Willink

Previous Topic:	Xtend template expressions are not resolved using Xtend Maven plugin
Next Topic:	Model-Transformation: How to resolve custom generated Proxy-Objects.

Goto Forum:

-=] Back to Top [=-

Current Time: Mon Jul 21 15:55:48 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter