Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » OOM, ResourceSet, unload(), reloads (Xtext), IReferableElementsUnloader..
OOM, ResourceSet, unload(), reloads (Xtext), IReferableElementsUnloader.. [message #1234979] Thu, 23 January 2014 09:17 Go to next message
Michael Vorburger is currently offline Michael VorburgerFriend
Messages: 103
Registered: July 2009
Senior Member
Hello,

couple of questions re. memory usage; not theoretical, but based on big problems with OOMs in in-house product... Wink BTW, we're Xtext users, but I think these Qs are generic:

First of all: Is my basic understanding correct that in EMF, out of the box, a given ResourceSet just keeps growing, until you eventually OOM (if you have loads of models), unless you manually unload() Resources?

So, in a "bulk" use case, say Validating or Code Generating say 1000x (Xtext) Resources, you just.. iterate over your stuff, getResource(uri, true) from "the" ResourceSet, do your.. thing, and then simply unload() it again? (Or would you delete() it??)

But, in Xtext, during typical Code Gen or Validation, when you follow an EReference to EObjects, often others Resources will get loaded on demand (via the Index and all that fantastic magic). When you unload that primary resource you just validated / generated, as above, will those secondary on-demand loaded Resources also get unloaded? If not, then when will they get unloaded, by who?

And isn't this very inefficient.. suppose a model with a resource B which references to stuff in resource A (not the other way around, suppose). You you bulk code gen/validate "in order", so first A loaded... do your thing.. unload - it's gone from the ResourceSet. Now B... loading it... some validation/code gen code traveres an EReference into A.. now loading A -- again; we just we had it in mem before!- Done, unloading B. Uops, A is still in the RS. Sad

Now, I do understand that I'm essentially asking to "have cake and eat it too" (first expecting a Resource to stick around in the RS, and the complaining that it has..), but - wouldn't it be feasible to, somehow, build some.. thing (e.g. ResourceSet wrapper?) for a kind of automagic Resource unload strategy? Say on reaching a max number of entries in the ResourceSet, or when GC needs heap, if the Resources are not referenced anywhere else (WeakReference..), do unload(). Are there any other projects I should be aware of which have successfully done this before on top of EMF? Or is this a completely stupid idea? Would further public investigations around this be of general interest to others?

BTW: What exactly is the org.eclipse.xtext.parser.antlr.IReferableElementsUnloader thing, what's it's point, why needed, how are you supposed to use it, and does it relate to the above at all? (There doesn't seem to be any documentation about this, anywhere I looked at least.)

Sorry, too many Qs.. do enlighten me! Maybe I'm just totally not getting how this is supposed to work / be used. Thanks a lot.
Re: OOM, ResourceSet, unload(), reloads (Xtext), IReferableElementsUnloader.. [message #1234999 is a reply to message #1234979] Thu, 23 January 2014 10:03 Go to previous messageGo to next message
Felix Dorner is currently offline Felix DornerFriend
Messages: 295
Registered: March 2012
Senior Member
On 23/01/2014 10:17, Michael Vorburger wrote:

> First of all: Is my basic understanding correct that in EMF, out of the
> box, a given ResourceSet just keeps growing, until you eventually OOM
> (if you have loads of models), unless you manually unload() Resources?

Even unloading might increase heap usage, see Eikes blog post:
http://thegordian.blogspot.it/2008/11/how-scalable-are-my-models.html
Re: OOM, ResourceSet, unload(), reloads (Xtext), IReferableElementsUnloader.. [message #1235039 is a reply to message #1234979] Thu, 23 January 2014 12:15 Go to previous messageGo to next message
Sven Efftinge is currently offline Sven EfftingeFriend
Messages: 1823
Registered: July 2009
Senior Member
A ResourceSet holds strong references to all contained resources (through the contents list).
In Xtext a fresh resource set is created for every editor and for every project build.

In the editor it contains all resources which are somehow referenced from the edited resource.
In the builder it contains all resources currently build (all contained in the current project on clean builds) plus referenced resources. There is some fancy clustering mechanism in Xtext, but it is deactived by default, so all the resources will be contained during build.

And yes, unloading them is potentially expensive as you'd have to reload resources again. OTOH we had customers where using clustering imrpoved performance dramatically as the GC wasn't triggered so often Smile

It all depends on you system and available memory. The possibly simplest solution is to increase the heap size via -Xmx.

Cheers,
Sven
Re: OOM, ResourceSet, unload(), reloads (Xtext), IReferableElementsUnloader.. [message #1237863 is a reply to message #1234979] Thu, 30 January 2014 20:11 Go to previous message
Michael Vorburger is currently offline Michael VorburgerFriend
Messages: 103
Registered: July 2009
Senior Member
Tx for responses.. just FYI, I've just blogged something vaguely related on http://blog2.vorburger.ch/2014/01/eclipse-emf-xtext-memory-consumption-in.html (although that, re. removing of node models, is obviouly Xtext specific and not EMF generic).
Previous Topic:EMF Eclipse Forms Editor
Next Topic:Re: [JFace Databinding] Bind EMap value to Text
Goto Forum:
  


Current Time: Wed Aug 21 03:37:54 GMT 2019

Powered by FUDForum. Page generated in 0.02032 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top