Re: [eclipselink-users] Object retention issues.

["Nathan Bryant" <nathan.bryant@xxxxxxxxxxxxx>]

Tim-

Thanks for the suggestion.

I ended up solving the problem a couple different ways:
 (1) Use NamedNativeQueries for the two largest tables, one of which is read-only and the other of which is insert-once-and-forget for the purpose of this algorithm.
 (2) After implementing (1), the heap histograms had a new biggest component: Huge object arrays, which led me to problems with CursoredStream. If using a CursoredStream, you must call releasePrevious(). It's certainly not sufficient to just use it as an Iterator. (I ended up wrapping an Iterator adaptor around it.)

I was also noticing some issues with EclipseLink seeming to spend a lot of CPU time in change tracking - that problem now seems fixed.

-----Original Message-----
From: eclipselink-users-bounces@xxxxxxxxxxx [mailto:eclipselink-users-bounces@xxxxxxxxxxx] On Behalf Of Tim Hollosy
Sent: Monday, November 08, 2010 11:09 AM
To: EclipseLink User Discussions
Subject: Re: [eclipselink-users] Object retention issues.

I think I've heard it said before on the list when batch performance
issues came up, it was best to grab new entity managers every so often
to avoid memory issues.

./tch



On Wed, Nov 3, 2010 at 2:18 PM, Nathan Bryant
<nathan.bryant@xxxxxxxxxxxxx> wrote:
> Hi,
>
>
>
> I'm seeing some object retention issues. This is in 2.1.0RC4
>
>
>
> I have a large batch task that includes an aggregation task (pseudocode)
>
>
>
> for (x : some CursoredStream) {
>
>                 Map<Key,Y> m
>
>                 Commit every 10K {
>
>                                 K k = new Key(x.someColumns)
>
>                                 Z z = new Z(x); persist(z); // z has a
> @ManyToOne field referencing X
>
>                                 Y y = m.get(k) or new Y if null
>
>                                 y.amt += x.amt
>
>                 }
>
> }
>
>
>
> The problem seems to be that multiple units of work arise from the "commit
> every 10K" logic, and can retain references to many clones of Xs; when in
> reality the Xs are not being modified and shouldn't need to remain
> referenced. I am working to remove container-managed references from X to
> other objects that may have been created in other units of work, because it
> seems like this may be enough to cause the other UnitOfWork's to fail to be
> gc'd. Is  there anything else I need to do to avoid this entirely? Should I
> consider writing native SQL to load the Xs?
>
>
>
> Note that all objects are retrieved from the same EntityManager.
>
>
>
> This is also somewhat unpredictable: most of the time, the memory usage does
> not grow too much, but sometimes it's enough to cause problems.
>
> _______________________________________________
> eclipselink-users mailing list
> eclipselink-users@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
>
>
_______________________________________________
eclipselink-users mailing list
eclipselink-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/eclipselink-users