Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » BuilderDeltaConverter leaks JDT JavaModelCache.childrenCache
BuilderDeltaConverter leaks JDT JavaModelCache.childrenCache [message #1802955] Tue, 19 February 2019 15:46 Go to next message
Sergio Otero is currently offline Sergio OteroFriend
Messages: 37
Registered: June 2012
Member
I've been making performance optimizations on a pretty big project with almost 10.000 source files and after solving XText related problems (for example limiting scope), i'm stuck with an OutOfMemory from JDT itself.

I've been narrowing down the problem and i've found 2 issues:

* OutOfMemory when the files appear due to an interaction with XText BuilderDeltaConverter
* OutOfMemory when doing a full build because of JDT batch compilation of 2.000 files: this can be solved with -DmaxCompiledUnitsAtOnce=200

The first one only happens with XText plugins installed, even when the project is a simple java project with no xtext

Steps to reproduce:

* Open an Eclipse 2018-02 without XText Plugins. Use "-DmaxCompiledUnitsAtOnce=200" in eclipse.ini to avoid the other JDT problem
* Create a simple Java Project and paste the class GenCode below
* Execute GenCode to generate 10.000 files with 2.000 variables each
* Connect JVisualVM to Eclipse with VisualGC
* Refresh the project to let eclipse index the new files
* Look at OldGen occupation growing to 800Mb but going down to 400Mb after finishing (it may need a "perform gc" from JVisualVM)
* Then repeat the same experiment but with XText Plugins installed in Eclipse (happens at least in Eclipse Neon and 2018-12)
* In this case, memory goes up to several Gb and if it has enough to finish, it does not go down less than 2.4Gb. It happens event wild autobuild off
* Doing a heapdump shows that almost all memory is due to org.eclipse.jdt.internal.core.SourceFieldElementInfo and it's root path points to JavaModelManager->JavaModelCache->childrenCache (see attached jdt-dump.png)
* With the help of some breakpoints, i've seen that XText BuilderDeltaConverter somehow triggers that cache been filled when it's designed to hold only information about opened files. See the stacktrace at the bottom

My DSL does not need to use java types like xtend. Is there any way to disable BuilderDeltaConverter of Java code?
I've seen that the binding is done in "org.eclipse.xtext.common.types.shared -> CommonTypesContribution", but don't know how to disable in my DSL ui project

Apart from disabling it, i think solving the root cause could benefit other projects as well.

Thanks,

Sergio

JavaModelCache

	/**
	 * Cache of open children of openable Java Model Java elements
	 */
	protected Map childrenCache;


package test;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;

public class GenCode {
	private static final int NUM_FILES = 10000;
	private static final int NUM_VARS = 2000;
	
	public static void main(String[] args) throws IOException {
		Path dir = Paths.get("src").resolve("gen");
		
		Files.createDirectories(dir);
		
		String pre = "package gen;\n\n";
	
		StringBuffer sb = new StringBuffer();
		   
		for (int i = 0; i < NUM_FILES; i++) {
			// Class
			sb.append(pre);
			sb.append("public class TEST_").append(i).append(" {\n\n");

			// Vars					
			for (int j = 0; j < NUM_VARS; j++) {
				sb.append("	private String var_").append(j).append(";\n");
			}
			
			// Method to avoid unused variable warning
			sb.append("	public void test() {\n");
			for (int j = 0; j < NUM_VARS; j++) {
				sb.append("		System.out.println(var_").append(j).append(");\n");
			}
			sb.append(" }\n");
			sb.append("}");
			
			//Write class
			Files.write(dir.resolve("TEST_"+i+".java"), sb.toString().getBytes(), StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING);
			
			sb.setLength(0);
			
			if (i % 1000 == 0) {
				System.out.println(i);
			}
		}
	}
}


Thread [Worker-17: refresh] (Suspended (breakpoint at line 252 in JavaModelCache))	
	owns: JavaModelManager  (id=65)	
	JavaModelCache.putInfo(IJavaElement, Object) line: 252	
	JavaModelManager.putInfos(IJavaElement, Object, boolean, Map<IJavaElement,Object>) line: 4116	
	CompilationUnit(JavaElement).openWhenClosed(Object, boolean, IProgressMonitor) line: 610	
	CompilationUnit(JavaElement).getElementInfo(IProgressMonitor) line: 326	
	CompilationUnit(JavaElement).getElementInfo() line: 312	
	CompilationUnit(JavaElement).getChildren() line: 267	
	CompilationUnit(JavaElement).getChildrenOfType(int) line: 281	
	CompilationUnit.getTypes() line: 972	
	BuilderDeltaConverter(DeltaConverter).convertNewTypes(ICompilationUnit, List<Delta>) line: 257	
	BuilderDeltaConverter(DeltaConverter).convertAddedPackageFragment(IJavaElementDelta, List<Delta>) line: 497	
	BuilderDeltaConverter(DeltaConverter).convertPackageFragment(IJavaElementDelta, List<Delta>) line: 484	
	BuilderDeltaConverter(DeltaConverter).convertCompilationUnits(IJavaElementDelta, List<Delta>) line: 474	
	BuilderDeltaConverter(DeltaConverter).convertCompilationUnits(IJavaElementDelta, List<Delta>) line: 471	
	BuilderDeltaConverter(DeltaConverter).convertCompilationUnits(IJavaElementDelta, List<Delta>) line: 471	
	BuilderDeltaConverter(DeltaConverter).convertCompilationUnits(IJavaElementDelta, List<Delta>) line: 471	
	BuilderDeltaConverter(DeltaConverter).convert(IJavaElementDelta) line: 148	
	JavaChangeQueueFiller.elementChanged(ElementChangedEvent) line: 38	
	DeltaProcessor$3.run() line: 1739	
	SafeRunner.run(ISafeRunnable) line: 45	
	DeltaProcessor.notifyListeners(IJavaElementDelta, int, IElementChangedListener[], int[], int) line: 1727	
	DeltaProcessor.firePostChangeDelta(IJavaElementDelta, IElementChangedListener[], int[], int) line: 1560	
	DeltaProcessor.fire(IJavaElementDelta, int) line: 1536	
	DeltaProcessor.notifyAndFire(IJavaElementDelta) line: 2257	
	DeltaProcessor.resourceChanged(IResourceChangeEvent) line: 2147	
	DeltaProcessingState.resourceChanged(IResourceChangeEvent) line: 477	
	NotificationManager$1.run() line: 300	
	SafeRunner.run(ISafeRunnable) line: 45	
	NotificationManager.notify(ResourceChangeListenerList$ListenerEntry[], ResourceChangeEvent, boolean) line: 290	
	NotificationManager.broadcastChanges(ElementTree, ResourceChangeEvent, boolean) line: 153	
	Workspace.broadcastPostChange() line: 379	
	Workspace.endOperation(ISchedulingRule, boolean) line: 1502	
	ResourceMgmtActionProvider$1$1(InternalWorkspaceJob).run(IProgressMonitor) line: 49	
	Worker.run() line: 63
  • Attachment: jdt-dump.png
    (Size: 49.61KB, Downloaded 8 times)

[Updated on: Tue, 19 February 2019 15:46]

Report message to a moderator

Re: BuilderDeltaConverter leaks JDT JavaModelCache.childrenCache [message #1802978 is a reply to message #1802955] Wed, 20 February 2019 04:26 Go to previous messageGo to next message
Karsten Thoms is currently offline Karsten ThomsFriend
Messages: 687
Registered: July 2009
Location: Dortmund, Germany
Senior Member

If you think that this is an Xtext problem to investigate then please open an issue here: https://github.com/eclipse/xtext/issues
For JDT, report to https://bugs.eclipse.org/bugs/enter_bug.cgi?product=JDT

The forum is the wrong place for that, this would get lost. However, it is unlikely that we can take action for your corner case immediately. All that I could say is that we generally are working to improve performance and memory consumption constantly for each release.

We experienced an increased memory consumption from newer JDT versions, but as you say it is also on Neon then the case we observed might be a different one.

Quote:

My DSL does not need to use java types like xtend. Is there any way to disable BuilderDeltaConverter of Java code?
I've seen that the binding is done in "org.eclipse.xtext.common.types.shared -> CommonTypesContribution", but don't know how to disable in my DSL ui project


Not really. This is a contribution to the shared state. It is not something you could disable per language. When you potentially have multiple languages and some of them use Java types (e.g. Xtend), then this is needed. But you could create a target platform without the common.types bundle and everything related to Xbase. In principle it is possible to use Xtext languages on a target without JDT.
Re: BuilderDeltaConverter leaks JDT JavaModelCache.childrenCache [message #1802981 is a reply to message #1802978] Wed, 20 February 2019 06:48 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13064
Registered: July 2009
Senior Member
See https://github.com/eclipse/xtext-eclipse/issues/70 too

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/xtext/
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: BuilderDeltaConverter leaks JDT JavaModelCache.childrenCache [message #1803003 is a reply to message #1802981] Wed, 20 February 2019 14:40 Go to previous message
Sergio Otero is currently offline Sergio OteroFriend
Messages: 37
Registered: June 2012
Member
https://github.com/eclipse/xtext-eclipse/issues/986

I have added a proposed fix in org.eclipse.xtext.common.types.ui.notification.DeltaConverter for convertNewTypes and convertChangedCompilationUnit
Previous Topic:Xtext maven plugin language order
Next Topic:How to get Absolute Path of the imported Resource
Goto Forum:
  


Current Time: Tue Aug 20 05:55:43 GMT 2019

Powered by FUDForum. Page generated in 0.02679 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top