Home » Modeling » TMF (Xtext) » BuilderDeltaConverter leaks JDT JavaModelCache.childrenCache
| BuilderDeltaConverter leaks JDT JavaModelCache.childrenCache [message #1802955] |
Tue, 19 February 2019 15:46  |
 Sergio Otero
Messages: 39
Registered: June 2012 |
Member |
|
|
I've been making performance optimizations on a pretty big project with almost 10.000 source files and after solving XText related problems (for example limiting scope), i'm stuck with an OutOfMemory from JDT itself.
I've been narrowing down the problem and i've found 2 issues:
* OutOfMemory when the files appear due to an interaction with XText BuilderDeltaConverter
* OutOfMemory when doing a full build because of JDT batch compilation of 2.000 files: this can be solved with -DmaxCompiledUnitsAtOnce=200
The first one only happens with XText plugins installed, even when the project is a simple java project with no xtext
Steps to reproduce:
* Open an Eclipse 2018-02 without XText Plugins. Use "-DmaxCompiledUnitsAtOnce=200" in eclipse.ini to avoid the other JDT problem
* Create a simple Java Project and paste the class GenCode below
* Execute GenCode to generate 10.000 files with 2.000 variables each
* Connect JVisualVM to Eclipse with VisualGC
* Refresh the project to let eclipse index the new files
* Look at OldGen occupation growing to 800Mb but going down to 400Mb after finishing (it may need a "perform gc" from JVisualVM)
* Then repeat the same experiment but with XText Plugins installed in Eclipse (happens at least in Eclipse Neon and 2018-12)
* In this case, memory goes up to several Gb and if it has enough to finish, it does not go down less than 2.4Gb. It happens event wild autobuild off
* Doing a heapdump shows that almost all memory is due to org.eclipse.jdt.internal.core.SourceFieldElementInfo and it's root path points to JavaModelManager->JavaModelCache->childrenCache (see attached jdt-dump.png)
* With the help of some breakpoints, i've seen that XText BuilderDeltaConverter somehow triggers that cache been filled when it's designed to hold only information about opened files. See the stacktrace at the bottom
My DSL does not need to use java types like xtend. Is there any way to disable BuilderDeltaConverter of Java code?
I've seen that the binding is done in "org.eclipse.xtext.common.types.shared -> CommonTypesContribution", but don't know how to disable in my DSL ui project
Apart from disabling it, i think solving the root cause could benefit other projects as well.
Thanks,
Sergio
JavaModelCache
/**
* Cache of open children of openable Java Model Java elements
*/
protected Map childrenCache;
package test;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;
public class GenCode {
private static final int NUM_FILES = 10000;
private static final int NUM_VARS = 2000;
public static void main(String[] args) throws IOException {
Path dir = Paths.get("src").resolve("gen");
Files.createDirectories(dir);
String pre = "package gen;\n\n";
StringBuffer sb = new StringBuffer();
for (int i = 0; i < NUM_FILES; i++) {
// Class
sb.append(pre);
sb.append("public class TEST_").append(i).append(" {\n\n");
// Vars
for (int j = 0; j < NUM_VARS; j++) {
sb.append(" private String var_").append(j).append(";\n");
}
// Method to avoid unused variable warning
sb.append(" public void test() {\n");
for (int j = 0; j < NUM_VARS; j++) {
sb.append(" System.out.println(var_").append(j).append(");\n");
}
sb.append(" }\n");
sb.append("}");
//Write class
Files.write(dir.resolve("TEST_"+i+".java"), sb.toString().getBytes(), StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING);
sb.setLength(0);
if (i % 1000 == 0) {
System.out.println(i);
}
}
}
}
Thread [Worker-17: refresh] (Suspended (breakpoint at line 252 in JavaModelCache))
owns: JavaModelManager (id=65)
JavaModelCache.putInfo(IJavaElement, Object) line: 252
JavaModelManager.putInfos(IJavaElement, Object, boolean, Map<IJavaElement,Object>) line: 4116
CompilationUnit(JavaElement).openWhenClosed(Object, boolean, IProgressMonitor) line: 610
CompilationUnit(JavaElement).getElementInfo(IProgressMonitor) line: 326
CompilationUnit(JavaElement).getElementInfo() line: 312
CompilationUnit(JavaElement).getChildren() line: 267
CompilationUnit(JavaElement).getChildrenOfType(int) line: 281
CompilationUnit.getTypes() line: 972
BuilderDeltaConverter(DeltaConverter).convertNewTypes(ICompilationUnit, List<Delta>) line: 257
BuilderDeltaConverter(DeltaConverter).convertAddedPackageFragment(IJavaElementDelta, List<Delta>) line: 497
BuilderDeltaConverter(DeltaConverter).convertPackageFragment(IJavaElementDelta, List<Delta>) line: 484
BuilderDeltaConverter(DeltaConverter).convertCompilationUnits(IJavaElementDelta, List<Delta>) line: 474
BuilderDeltaConverter(DeltaConverter).convertCompilationUnits(IJavaElementDelta, List<Delta>) line: 471
BuilderDeltaConverter(DeltaConverter).convertCompilationUnits(IJavaElementDelta, List<Delta>) line: 471
BuilderDeltaConverter(DeltaConverter).convertCompilationUnits(IJavaElementDelta, List<Delta>) line: 471
BuilderDeltaConverter(DeltaConverter).convert(IJavaElementDelta) line: 148
JavaChangeQueueFiller.elementChanged(ElementChangedEvent) line: 38
DeltaProcessor$3.run() line: 1739
SafeRunner.run(ISafeRunnable) line: 45
DeltaProcessor.notifyListeners(IJavaElementDelta, int, IElementChangedListener[], int[], int) line: 1727
DeltaProcessor.firePostChangeDelta(IJavaElementDelta, IElementChangedListener[], int[], int) line: 1560
DeltaProcessor.fire(IJavaElementDelta, int) line: 1536
DeltaProcessor.notifyAndFire(IJavaElementDelta) line: 2257
DeltaProcessor.resourceChanged(IResourceChangeEvent) line: 2147
DeltaProcessingState.resourceChanged(IResourceChangeEvent) line: 477
NotificationManager$1.run() line: 300
SafeRunner.run(ISafeRunnable) line: 45
NotificationManager.notify(ResourceChangeListenerList$ListenerEntry[], ResourceChangeEvent, boolean) line: 290
NotificationManager.broadcastChanges(ElementTree, ResourceChangeEvent, boolean) line: 153
Workspace.broadcastPostChange() line: 379
Workspace.endOperation(ISchedulingRule, boolean) line: 1502
ResourceMgmtActionProvider$1$1(InternalWorkspaceJob).run(IProgressMonitor) line: 49
Worker.run() line: 63
Attachment: jdt-dump.png
(Size: 49.61KB, Downloaded 116 times)
[Updated on: Tue, 19 February 2019 15:46] Report message to a moderator |
|
|
| Re: BuilderDeltaConverter leaks JDT JavaModelCache.childrenCache [message #1802978 is a reply to message #1802955] |
Wed, 20 February 2019 04:26  |
|
If you think that this is an Xtext problem to investigate then please open an issue here: https://github.com/eclipse/xtext/issues
For JDT, report to https://bugs.eclipse.org/bugs/enter_bug.cgi?product=JDT
The forum is the wrong place for that, this would get lost. However, it is unlikely that we can take action for your corner case immediately. All that I could say is that we generally are working to improve performance and memory consumption constantly for each release.
We experienced an increased memory consumption from newer JDT versions, but as you say it is also on Neon then the case we observed might be a different one.
Quote:
My DSL does not need to use java types like xtend. Is there any way to disable BuilderDeltaConverter of Java code?
I've seen that the binding is done in "org.eclipse.xtext.common.types.shared -> CommonTypesContribution", but don't know how to disable in my DSL ui project
Not really. This is a contribution to the shared state. It is not something you could disable per language. When you potentially have multiple languages and some of them use Java types (e.g. Xtend), then this is needed. But you could create a target platform without the common.types bundle and everything related to Xbase. In principle it is possible to use Xtext languages on a target without JDT.
|
|
| | |
Goto Forum:
Current Time: Sat Apr 27 01:06:01 GMT 2024
Powered by FUDForum. Page generated in 0.03260 seconds |
] 
Search
Help
Register
Login
Home
