Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » EclipseLink » QueryValueHolder Strategy for Better Memory Management
QueryValueHolder Strategy for Better Memory Management [message #388460] Fri, 29 May 2009 20:20 Go to next message
NateB is currently offline NateB
Messages: 4
Registered: July 2009
Junior Member
We've been using TopLink (EclipseLink now) for 6 years and have a very
large application with hundreds of tables mapped.

In addition to the large number of objects, our persistence framework has
a lot of interlinking (many 1-M and 1-1 back reference mappings throughout
the application). The object graph that is loaded into memory is extremely
intertwined.

In instances of our application where datasets are very large we are
running into the situation where the GC cannot free memory because active
threads retain hard references to objects, which, because of
QueryValueHolder, retains hard references to related objects throughout
our object model. This is true regardless of the setting in the
IdentityMap.

For several reasons not worth exploring on this thread, we cannot remove
these ValueHolder references.

An interesting idea came out of our internal discussions and I wanted to
post it to this forum to learn if this approach had any merit.

We would like to create an implementation of ValueHolder (we would assume
QueryValueHolder) with the following properties:

1) It would have a "reinitialize()" method that would release it's hold on
any loaded objects and be treated as if it had never loaded anything from
the DB.

2) Have the ValueHolder retain the reference to these objects via a
WeakReference so that these relationships could be GC'd (and reloaded, if
necessary).

Would this approach be worth exploring? What would break?
Re: QueryValueHolder Strategy for Better Memory Management [message #388467 is a reply to message #388460] Sat, 30 May 2009 05:26 Go to previous messageGo to next message
Tom Eugelink is currently offline Tom Eugelink
Messages: 807
Registered: July 2009
Senior Member
> 2) Have the ValueHolder retain the reference to these objects via a
> WeakReference so that these relationships could be GC'd (and reloaded,
> if necessary).


I had troubles with data not being GC-ed as well, but as of Eclipselink 1.0.2 there is a weak-reference option that I could set.

<property name="eclipselink.persistence-context.reference-mode" value="WEAK"/>

Have you tried this?

Tom
Re: QueryValueHolder Strategy for Better Memory Management [message #388476 is a reply to message #388460] Sun, 31 May 2009 16:07 Go to previous messageGo to next message
Doug Clarke is currently offline Doug Clarke
Messages: 155
Registered: July 2009
Senior Member
Nate,

I have investigated this in the past and believe it is definitely
possible. The challenge is rebuilding the necessary row required for the
query. I will take a look to see if I can find my previous example.

The tougher question will be around the application of this. In a large
model I could see applying this effectively to be much more work.

Doug
Re: QueryValueHolder Strategy for Better Memory Management [message #388486 is a reply to message #388467] Mon, 01 June 2009 12:08 Go to previous messageGo to next message
NateB is currently offline NateB
Messages: 4
Registered: July 2009
Junior Member
I haven't. We're in the process of migrating from the latest Oracle
Toplink to EclipseLink.

We use Java mappings (as opposed to XML configs), so I'm not sure where
your example fits in the Java APIs. Does the WeakReference option you are
referring to affect the ValueHolder collections off of persistent objects
or is this an IdentityMap setting?
Re: QueryValueHolder Strategy for Better Memory Management [message #388487 is a reply to message #388476] Mon, 01 June 2009 12:12 Go to previous messageGo to next message
NateB is currently offline NateB
Messages: 4
Registered: July 2009
Junior Member
Our idea was to make the ValueHolder reference itself a WeakReference
rather that use a WeakReference collection that could have individual
objects that could be GC'd.

The idea here would be if the GC harvested the QueryValueHolder itself (or
the reference to the collection) this new ValueHolder would interpret this
as having never been initialized.

Maybe you already understood this, but this sounded more simple to
implement (obviously we don't understand all the issues here, hence this
post).

It seems as if we had a way to reinit a ValueHolder that would be the 1st
step.

Where should we start? Or, would you steer us in a different direction?
Re: QueryValueHolder Strategy for Better Memory Management [message #388489 is a reply to message #388487] Mon, 01 June 2009 12:31 Go to previous messageGo to next message
Doug Clarke is currently offline Doug Clarke
Messages: 155
Registered: July 2009
Senior Member
Nate,

I believe we need to work through the requirements and usage scenarios. If
you have not already please log an enhancement request against EclipseLink
for this and we can use a wiki page to capture the requirements. Then we
can have a more complete discussion about solution options.

Doug
Re: QueryValueHolder Strategy for Better Memory Management [message #388490 is a reply to message #388486] Mon, 01 June 2009 13:09 Go to previous messageGo to next message
Tom Eugelink is currently offline Tom Eugelink
Messages: 807
Registered: July 2009
Senior Member
> We use Java mappings (as opposed to XML configs), so I'm not sure where
> your example fits in the Java APIs. Does the WeakReference option you
> are referring to affect the ValueHolder collections off of persistent
> objects or is this an IdentityMap setting?

I use Java annotations as well, this is part of the persistence.xml configuration.

Ehm. Well. Ahhhhh. I think the latter. Not sure here though. I was trying to get a finger behind several memory leaks and this was one of the suggestions, and it seemed to work.
Re: QueryValueHolder Strategy for Better Memory Management [message #388492 is a reply to message #388489] Mon, 01 June 2009 14:02 Go to previous message
NateB is currently offline NateB
Messages: 4
Registered: July 2009
Junior Member
Doug, I have logged the request.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=278650

We will be assigning some engineering resources to look at this in our
next sprint (the next couple of days). We'd be interested in any way we
could be involved to contribute, test, etc.
Previous Topic:EclipseLink does not store in database
Next Topic:Moving from ToplinkLink Essentials (SessionCustomizer problem)
Goto Forum:
  


Current Time: Fri Aug 29 14:24:57 EDT 2014

Powered by FUDForum. Page generated in 0.04577 seconds