| Using NeoEMF with Xtext [message #1725599] |
Fri, 04 March 2016 12:16  |
Eclipse User |
|
|
|
Dear all,
sorry for the long description, I just wanted to explain my current situation.
I am currently struggling with the integration of NeoEMF (https://github.com/atlanmod/NeoEMF, former name Neo4EMF) with Xtext. I try to develop a MoDisco discoverer https://eclipse.org/MoDisco/ which scans directories and parses the individual source files with Xtext (which works quite well). The EMF root objects (e.g., with the name Foo) which are returned by Xtext are then stored as elements in an EList of the main EMF model. The problem is that the default XMI persistence layer is at some point not able to handle the amount of data any more and raises an OutOfMemory runtime exception. Now I am trying to migrate to NeoEMF which can use the NoSQL Neo4j graph database as a back-end. This also works quite well for my JUnit plug-in test cases. But when running on real code it also throws an OutOfMemory exception which may be preventable.
In my code I basically do the following (error handling and other stuff omitted):
private Foo parseFooUnitFor(final IFile iFile) {
IResourceFactory resourceFactory = this.injector.getInstance(IResourceFactory.class);
Resource.Factory.Registry.INSTANCE.getExtensionToFactoryMap().put(
".foo", resourceFactory);
final IProject iProject = iFile.getProject();
final XtextResourceSetProvider provider = this.injector
.getInstance(XtextResourceSetProvider.class);
final XtextResourceSet resourceSet = (XtextResourceSet) provider
.get(iProject);
resourceSet.addLoadOption(XtextResource.OPTION_RESOLVE_ALL,
Boolean.TRUE);
final URI uri = URI.createURI(iFile.getLocationURI().toString());
final Resource resource = resourceSet.getResource(uri, true);
final Foo result = (Foo)resource.getContents().get(0);
return result;
}
private void addFooUnitTo(final Model model, final IFile iFile) {
final TranslationUnit unit = FooFactory.eINSTANCE
.createTranslationUnit();
unit.setPath(iFile.getLocationURI().toString());
model.add(unit);
Foo foo = parseFooUnitFor(iFile);
MyLog.debug(XtextHandler.class, "foo='" + foo + "'");
MyLog.debug(XtextHandler.class, "foo-res='" + foo.eResource()
+ "'");
MyLog.debug(XtextHandler.class, "foo-resSet='"
+ foo.eResource().getResourceSet() + "'");
MyLog.debug(XtextHandler.class, "unit='" + unit + "'");
MyLog.debug(XtextHandler.class, "unit-res='" + unit.eResource() + "'");
MyLog.debug(XtextHandler.class, "unit-resSet='"
+ unit.eResource().getResourceSet() + "'");
// the next line is where the OutOfMemory exception is thrown
unit.setFoo(foo);
MyLog.debug(XtextHandler.class, "");
MyLog.debug(XtextHandler.class, "foo='" + foo + "'");
MyLog.debug(XtextHandler.class, "foo-res='" + foo.eResource()
+ "'");
MyLog.debug(XtextHandler.class, "foo-resSet='"
+ foo.eResource().getResourceSet() + "'");
MyLog.debug(XtextHandler.class, "unit='" + unit + "'");
MyLog.debug(XtextHandler.class, "unit-res='" + unit.eResource() + "'");
MyLog.debug(XtextHandler.class, "unit-resSet='"
+ unit.eResource().getResourceSet() + "'");
}
I am getting the following output (when no OutOfMemory Exception is thrown:
foo='foo.foo.impl.FooImpl@8a490b9b (eClass: org.eclipse.emf.ecore.impl.EClassImpl@7c2f2675 (name: Foo) (instanceClassName: null) (abstract: false, interface: false))'
foo-res='org.eclipse.xtext.linking.lazy.LazyLinkingResource@9196714 uri='file:/home/harry/workspace/runtime-EclipseApplication/Test/tmp-discover/home/harry/workspace/runtime-EclipseApplication/Test/src/b.foo''
foo-resSet='org.eclipse.xtext.resource.SynchronizedXtextResourceSet@3e2ef95f resources=[org.eclipse.xtext.linking.lazy.LazyLinkingResource@9196714 uri='file:/home/harry/workspace/runtime-EclipseApplication/Test/tmp-discover/home/harry/workspace/runtime-EclipseApplication/Test/src/b.foo']'
unit='foo.foo.impl.TranslationUnitImpl@ddb48f0d (eClass: org.eclipse.emf.ecore.impl.EClassImpl@396b060e (name: TranslationUnit) (instanceClassName: null) (abstract: false, interface: false))'
unit-res='fr.inria.atlanmod.neoemf.resources.impl.PersistentResourceImpl@183c95d2 uri='neo-blueprints:/home/harry/workspace/runtime-EclipseApplication/Test/tmp-discover/discover-2016-03-03-175607_c.neoemf''
unit-resSet='org.eclipse.emf.ecore.resource.impl.ResourceSetImpl@7cea2c4e resources=[fr.inria.atlanmod.neoemf.resources.impl.PersistentResourceImpl@183c95d2 uri='neo-blueprints:/home/harry/workspace/runtime-EclipseApplication/Test/tmp-discover/discover-2016-03-03-175607_c.neoemf']'
foo='foo.foo.impl.FooImpl@8a490b9b (eClass: org.eclipse.emf.ecore.impl.EClassImpl@7c2f2675 (name: Foo) (instanceClassName: null) (abstract: false, interface: false))'
foo-res='fr.inria.atlanmod.neoemf.resources.impl.PersistentResourceImpl@183c95d2 uri='neo-blueprints:/home/harry/workspace/runtime-EclipseApplication/Test/tmp-discover/discover-2016-03-03-175607_c.neoemf''
foo-resSet='org.eclipse.emf.ecore.resource.impl.ResourceSetImpl@7cea2c4e resources=[fr.inria.atlanmod.neoemf.resources.impl.PersistentResourceImpl@183c95d2 uri='neo-blueprints:/home/harry/workspace/runtime-EclipseApplication/Test/tmp-discover/discover-2016-03-03-175607_c.neoemf']'
unit='foo.foo.impl.TranslationUnitImpl@ddb48f0d (eClass: org.eclipse.emf.ecore.impl.EClassImpl@396b060e (name: TranslationUnit) (instanceClassName: null) (abstract: false, interface: false))'
unit-res='fr.inria.atlanmod.neoemf.resources.impl.PersistentResourceImpl@183c95d2 uri='neo-blueprints:/home/harry/workspace/runtime-EclipseApplication/Test/tmp-discover/discover-2016-03-03-175607_c.neoemf''
unit-resSet='org.eclipse.emf.ecore.resource.impl.ResourceSetImpl@7cea2c4e resources=[fr.inria.atlanmod.neoemf.resources.impl.PersistentResourceImpl@183c95d2 uri='neo-blueprints:/home/harry/workspace/runtime-EclipseApplication/Test/tmp-discover/discover-2016-03-03-175607_c.neoemf']'
So I believe that the problem is that somehow the Foo EObject element is copied over from the LazyLinkingResource to the PersistentResourceImpl which results in a Java heap overflow for very large Foo objects.
So my question is: would it be possible to directly create the resource of Foo as a PersistentResourceImpl resource while still being able to parse EMF models from text files?
|
|
|
|
|
|
|
| Re: Using NeoEMF with Xtext [message #1726065 is a reply to message #1725718] |
Wed, 09 March 2016 05:57  |
| Eclipse User |
|
|
|
Well, I made a work-around for this problem by doing the following (as my Foo EObject mainly consists of a list of Bar objects:
Foo secondFoo = FooFactory.eINSTANCE.createFoo();
final EList<Bar> list = foo.getBar();
final Iterator<Bar> it = list.getIterator();
for (int i = 0; it.hasNext(); i++) {
final Bar bar = it.next();
it.remove();
secondFoo.getBar().add(bar);
if ((i % 1000) == 0) {
secondFoo.eResource().save(new HashMap<>());
}
}
This solves the OutOfMemory exception problems for me.
I do not know how it would work with sub-classing XtextResource/XtextResource.Factory/XtextResourceSet, but maybe applying the adapter pattern might help... Just some thoughts. Anyway, I think that it would be better to have the persistence layer separated from the Xtext resource handling.
|
|
|
Powered by
FUDForum. Page generated in 0.04644 seconds
Home