Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » EclipseLink » Eclipselink is spending an hour releasing write locks(The process I am troubleshooting has acquired a lot of write locks. I need help identifying why that would be. I also have a potential Eclipselink performance improvement/patch.)
Eclipselink is spending an hour releasing write locks [message #499452] Sun, 22 November 2009 17:45 Go to next message
David Mulligan is currently offline David Mulligan
Messages: 6
Registered: November 2009
Junior Member
I am working on a large application that is, for the most part, working quite well in production. However one of our clients have reported a process that is taking 2 hours to complete. For this process I would expect that half of that time, 1 hour, would be reasonable and one quarter of it, 30 minutes, would be optimal. We are dealing with a lot of data and processing. After profiling the process I found that it is taking 60 minutes to clear write locks.

Specifically it is spending the 60 minutes in WriteLockManager.releaseAllAcquiredLocks at the end of the for loop on the the line that reads locks.remove(). A little more digging I found that there are about 1.5 million write locks to be removed at this point. The rest of the application is not running into this so I am looking at something different about this process or the mappings that it uses. The number of objects being changed is not unique though. Any ideas about why so many write locks are being acquired?

We are running this on a very fast box (16 Xeon cores with 128MB ram) running RHES running against Oracle 10G also on a fast box. We are running EclipseLink 1.1.3 on Java 1.6.0_4.

I have been able to make the symptom go away with a small change to Eclipselink. I removed the locks.remove() line from WriteLockManager.releaseAllAcquiredLocks and put mergeManager.getAcquiredLocks().clear() outside of the loop. The performance problem here is that we are calling ArrayList.iterator.remove() at the beginning of a list which in turn calls arraycopy to shift the contents of the arraylist over one element down. It is far more efficient to empty the whole list after we are done iterating over it to perform the cleanup on the individual locks. As I mentioned above my application was spending 60 minutes removing individual elements from the list. After this change releaseAllAcquiredLocks is practically instant.

Thank you,
David

[Updated on: Sun, 22 November 2009 18:12]

Report message to a moderator

Re: Eclipselink is spending an hour releasing write locks [message #499676 is a reply to message #499452] Mon, 23 November 2009 16:51 Go to previous messageGo to next message
James Sutherland is currently offline James Sutherland
Messages: 1939
Registered: July 2009
Location: Ottawa, Canada
Senior Member

This issue and your patch seems valid, please log a bug and attach your patch.

However, in general committing 1.5 million objects in a single EntityManager transaction may not be a good idea.

Normally when dealing with a large set of objects, it is best to split the load into batches. Ideally for each batch (1,000-10,000 objects) create a new EntityManager and transaction. If you need them all to be the same transaction, you can use flush() then clear() after each batch.

Also, if you are mass inserting these objects, it would be best to disable the cache, as it does not help inserts and does add a small (normally) overhead. This can be done setting "eclipselink.cache.shared.default"="false" in your persistence.xml or properties. Also try enabling batch writing, and ensure you are using sequence pre-allocation.


James : Wiki : Book : Blog : Twitter
Re: Eclipselink is spending an hour releasing write locks [message #499751 is a reply to message #499676] Mon, 23 November 2009 22:51 Go to previous messageGo to next message
David Mulligan is currently offline David Mulligan
Messages: 6
Registered: November 2009
Junior Member
Thank you James. I have submitted a bug along with a patch.

I have done a little more digging. It turns out that I do indeed have 1.5 million objects being updated. I thought that I had a lot less than that and was wondering what could cause there to be so many write locks required with so few objects.

Just to make sure that I understand though; every object being updated, deleted, or inserted will normally require it's own write lock?

Thanks,
David
Re: Eclipselink is spending an hour releasing write locks [message #499967 is a reply to message #499452] Tue, 24 November 2009 15:38 Go to previous message
James Sutherland is currently offline James Sutherland
Messages: 1939
Registered: July 2009
Location: Ottawa, Canada
Senior Member

Yes, if you have the shared cache enabled, each object that was inserted, updated, or deleted, during the transaction will acquire a cache write-lock during the shared cache merge, after the database flush.



James : Wiki : Book : Blog : Twitter
Previous Topic:Caching a Multi-tenant applicantion
Next Topic:spring IndexOutOfBoundsException
Goto Forum:
  


Current Time: Sun Oct 26 05:14:42 GMT 2014

Powered by FUDForum. Page generated in 0.02264 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software