Eclipse Community Forums: EclipseLink » QueryValueHolder Strategy for Better Memory Management

Help

Home

Home » Eclipse Projects » EclipseLink » QueryValueHolder Strategy for Better Memory Management

Show: Today's Messages :: Show Polls :: Message Navigator

QueryValueHolder Strategy for Better Memory Management [message #388460]

Sat, 30 May 2009 00:20

NateB

Messages: 4
Registered: July 2009

Junior Member

We've been using TopLink (EclipseLink now) for 6 years and have a very
large application with hundreds of tables mapped.

In addition to the large number of objects, our persistence framework has
a lot of interlinking (many 1-M and 1-1 back reference mappings throughout
the application). The object graph that is loaded into memory is extremely
intertwined.

In instances of our application where datasets are very large we are
running into the situation where the GC cannot free memory because active
threads retain hard references to objects, which, because of
QueryValueHolder, retains hard references to related objects throughout
our object model. This is true regardless of the setting in the
IdentityMap.

For several reasons not worth exploring on this thread, we cannot remove
these ValueHolder references.

An interesting idea came out of our internal discussions and I wanted to
post it to this forum to learn if this approach had any merit.

We would like to create an implementation of ValueHolder (we would assume
QueryValueHolder) with the following properties:

1) It would have a "reinitialize()" method that would release it's hold on
any loaded objects and be treated as if it had never loaded anything from
the DB.

2) Have the ValueHolder retain the reference to these objects via a
WeakReference so that these relationships could be GC'd (and reloaded, if
necessary).

Would this approach be worth exploring? What would break?

Report message to a moderator

Re: QueryValueHolder Strategy for Better Memory Management [message #388467 is a reply to message #388460]

Sat, 30 May 2009 09:26

Tom Eugelink

Messages: 817
Registered: July 2009

Senior Member

> 2) Have the ValueHolder retain the reference to these objects via a
> WeakReference so that these relationships could be GC'd (and reloaded,
> if necessary).

I had troubles with data not being GC-ed as well, but as of Eclipselink 1.0.2 there is a weak-reference option that I could set.

<property name="eclipselink.persistence-context.reference-mode" value="WEAK"/>

Have you tried this?

Tom

Report message to a moderator

Re: QueryValueHolder Strategy for Better Memory Management [message #388476 is a reply to message #388460]

Sun, 31 May 2009 20:07

Doug Clarke

Messages: 155
Registered: July 2009

Senior Member

Nate,

I have investigated this in the past and believe it is definitely
possible. The challenge is rebuilding the necessary row required for the
query. I will take a look to see if I can find my previous example.

The tougher question will be around the application of this. In a large
model I could see applying this effectively to be much more work.

Doug

Report message to a moderator

Re: QueryValueHolder Strategy for Better Memory Management [message #388486 is a reply to message #388467]

Mon, 01 June 2009 16:08

NateB

Messages: 4
Registered: July 2009

Junior Member

I haven't. We're in the process of migrating from the latest Oracle
Toplink to EclipseLink.

We use Java mappings (as opposed to XML configs), so I'm not sure where
your example fits in the Java APIs. Does the WeakReference option you are
referring to affect the ValueHolder collections off of persistent objects
or is this an IdentityMap setting?

Report message to a moderator

Re: QueryValueHolder Strategy for Better Memory Management [message #388487 is a reply to message #388476]

Mon, 01 June 2009 16:12

NateB

Messages: 4
Registered: July 2009

Junior Member

Our idea was to make the ValueHolder reference itself a WeakReference
rather that use a WeakReference collection that could have individual
objects that could be GC'd.

The idea here would be if the GC harvested the QueryValueHolder itself (or
the reference to the collection) this new ValueHolder would interpret this
as having never been initialized.

Maybe you already understood this, but this sounded more simple to
implement (obviously we don't understand all the issues here, hence this
post).

It seems as if we had a way to reinit a ValueHolder that would be the 1st
step.

Where should we start? Or, would you steer us in a different direction?

Report message to a moderator

Re: QueryValueHolder Strategy for Better Memory Management [message #388489 is a reply to message #388487]

Mon, 01 June 2009 16:31

Doug Clarke

Messages: 155
Registered: July 2009

Senior Member

Nate,

I believe we need to work through the requirements and usage scenarios. If
you have not already please log an enhancement request against EclipseLink
for this and we can use a wiki page to capture the requirements. Then we
can have a more complete discussion about solution options.

Doug

Report message to a moderator

Re: QueryValueHolder Strategy for Better Memory Management [message #388490 is a reply to message #388486]

Mon, 01 June 2009 17:09

Tom Eugelink

Messages: 817
Registered: July 2009

Senior Member

> We use Java mappings (as opposed to XML configs), so I'm not sure where
> your example fits in the Java APIs. Does the WeakReference option you
> are referring to affect the ValueHolder collections off of persistent
> objects or is this an IdentityMap setting?

I use Java annotations as well, this is part of the persistence.xml configuration.

Ehm. Well. Ahhhhh. I think the latter. Not sure here though. I was trying to get a finger behind several memory leaks and this was one of the suggestions, and it seemed to work.

Report message to a moderator

Re: QueryValueHolder Strategy for Better Memory Management [message #388492 is a reply to message #388489]

Mon, 01 June 2009 18:02

NateB

Messages: 4
Registered: July 2009

Junior Member

Doug, I have logged the request.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=278650

We will be assigning some engineering resources to look at this in our
next sprint (the next couple of days). We'd be interested in any way we
could be involved to contribute, test, etc.

Report message to a moderator

Previous Topic:	EclipseLink does not store in database
Next Topic:	Moving from ToplinkLink Essentials (SessionCustomizer problem)

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Fri Apr 19 00:26:14 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter