Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » EclipseLink » Hung threads
Hung threads [message #1129321] Tue, 08 October 2013 14:13 Go to next message
Amita Mehta is currently offline Amita Mehta
Messages: 8
Registered: May 2013
Junior Member
We are using EclipseLink 2.5.1.v20130918-f2b9fc5 with WAS8

We are getting hung threads and a lot of time is being spent in the getChildObject call. Sometimes it happens for a one to one relationship and sometimes for a one to many.

We have removed all the joins.
We were having this problem initially with TopLink and so upgraded to EclipseLink but get the same same problem.
Any suggestions and solutions are appreciated.
Thanks.

Log

at org/eclipse/persistence/internal/helper/ConcurrencyManager.acquire(ConcurrencyManager.java:80(Compiled Code))
at org/eclipse/persistence/internal/identitymaps/CacheKey.acquire(CacheKey.java:132(Compiled Code))
at org/eclipse/persistence/internal/identitymaps/AbstractIdentityMap.acquireLock(AbstractIdentityMap.java:108(Compiled Code))
at org/eclipse/persistence/internal/identitymaps/IdentityMapManager.acquireLock(IdentityMapManager.java:157(Compiled Code))
at org/eclipse/persistence/internal/sessions/IdentityMapAccessor.acquireLock(IdentityMapAccessor.java:99(Compiled Code))
at org/eclipse/persistence/internal/sessions/IdentityMapAccessor.acquireLock(IdentityMapAccessor.java:90(Compiled Code))
at org/eclipse/persistence/internal/sessions/AbstractSession.retrieveCacheKey(AbstractSession.java:5143(Compiled Code))
at org/eclipse/persistence/internal/descriptors/ObjectBuilder.buildObject(ObjectBuilder.java:964(Compiled Code))
at org/eclipse/persistence/internal/descriptors/ObjectBuilder.buildObject(ObjectBuilder.java:736(Compiled Code))
at org/eclipse/persistence/internal/descriptors/ObjectBuilder.buildObject(ObjectBuilder.java:688(Compiled Code))
at org/eclipse/persistence/queries/ObjectLevelReadQuery.buildObject(ObjectLevelReadQuery.java:795(Compiled Code))
at org/eclipse/persistence/queries/ReadObjectQuery.executeObjectLevelReadQuery(ReadObjectQuery.java:554(Compiled Code))
at org/eclipse/persistence/queries/ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1168(Compiled Code))
at org/eclipse/persistence/queries/DatabaseQuery.execute(DatabaseQuery.java:899(Compiled Code))
at org/eclipse/persistence/queries/ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1127(Compiled Code))
at org/eclipse/persistence/queries/ReadObjectQuery.execute(ReadObjectQuery.java:431(Compiled Code))
at org/eclipse/persistence/internal/sessions/AbstractSession.internalExecuteQuery(AbstractSession.java:3203(Compiled Code))
at org/eclipse/persistence/internal/sessions/AbstractSession.executeQuery(AbstractSession.java:1793(Compiled Code))
at org/eclipse/persistence/internal/sessions/AbstractSession.executeQuery(AbstractSession.java:1775(Compiled Code))
at org/eclipse/persistence/internal/indirection/NoIndirectionPolicy.valueFromQuery(NoIndirectionPolicy.java:326(Compiled Code))
at org/eclipse/persistence/mappings/ForeignReferenceMapping.valueFromRowInternal(ForeignReferenceMapping.java:2294(Compiled Code))
at org/eclipse/persistence/mappings/OneToOneMapping.valueFromRowInternal(OneToOneMapping.java:1795(Compiled Code))
at org/eclipse/persistence/mappings/ForeignReferenceMapping.valueFromRow(ForeignReferenceMapping.java:2144(Compiled Code))
at org/eclipse/persistence/mappings/ForeignReferenceMapping.readFromRowIntoObject(ForeignReferenceMapping.java:1471(Compiled Code))
at org/eclipse/persistence/internal/descriptors/ObjectBuilder.buildAttributesIntoObject(ObjectBuilder.java:461(Compiled Code))
at org/eclipse/persistence/internal/descriptors/ObjectBuilder.refreshObjectIfRequired(ObjectBuilder.java:4300(Compiled Code))
at org/eclipse/persistence/internal/descriptors/ObjectBuilder.buildObject(ObjectBuilder.java:1039(Compiled Code))
at org/eclipse/persistence/internal/descriptors/ObjectBuilder.buildObject(ObjectBuilder.java:736(Compiled Code))
at org/eclipse/persistence/internal/descriptors/ObjectBuilder.buildObject(ObjectBuilder.java:688(Compiled Code))
at org/eclipse/persistence/queries/ObjectLevelReadQuery.buildObject(ObjectLevelReadQuery.java:795(Compiled Code))
at org/eclipse/persistence/queries/ReadObjectQuery.executeObjectLevelReadQuery(ReadObjectQuery.java:554(Compiled Code))
at org/eclipse/persistence/queries/ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1168(Compiled Code))
at org/eclipse/persistence/queries/DatabaseQuery.execute(DatabaseQuery.java:899(Compiled Code))
at org/eclipse/persistence/queries/ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1127(Compiled Code))
at org/eclipse/persistence/queries/ReadObjectQuery.execute(ReadObjectQuery.java:431(Compiled Code))
at org/eclipse/persistence/internal/sessions/AbstractSession.internalExecuteQuery(AbstractSession.java:3203(Compiled Code))
at org/eclipse/persistence/internal/sessions/AbstractSession.executeQuery(AbstractSession.java:1793(Compiled Code))
at org/eclipse/persistence/internal/sessions/AbstractSession.executeQuery(AbstractSession.java:1775(Compiled Code))
at org/eclipse/persistence/internal/indirection/QueryBasedValueHolder.instantiate(QueryBasedValueHolder.java:129(Compiled Code))
at org/eclipse/persistence/internal/indirection/QueryBasedValueHolder.instantiate(QueryBasedValueHolder.java:116(Compiled
Code))


Re: Hung threads [message #1129432 is a reply to message #1129321] Tue, 08 October 2013 16:16 Go to previous messageGo to next message
Chris Delahunt is currently offline Chris Delahunt
Messages: 1034
Registered: July 2009
Senior Member
You will need to obtain the full thread dump to track down which threads are involved in issue, and also call session.getIdentityMapAccessor().printIdentityMapLocks() to see which threads own which locks for the objects involved. See http://wiki.eclipse.org/EclipseLink/FAQ/JPA#How_to_diagnose_and_resolve_hangs_and_deadlocks.3F for potential workarounds, and contact support if you are able to. The stack trace shown does not use indirection on the OneToOne mapping on the object being read in, so this might be a good place to start if you are able to make changes. It helps to isolate the object involved and see if this is a true deadlock or just a severe bottleneck caused by thread contention on a single object.

Best Regards,
Chris
Re: Hung threads [message #1130768 is a reply to message #1129432] Wed, 09 October 2013 22:58 Go to previous messageGo to next message
Amita Mehta is currently offline Amita Mehta
Messages: 8
Registered: May 2013
Junior Member
On further analysis I see that I am fetching the 1-m child records by an expression query. The problems lies with setting refreshIdentityMapResult on the query.
If I comment out the line - query.refreshIdentityMapResult();
I am not seeing hung threads so far atleast in the dozen times I ran my unit test. I still have to test it more since with TopLink the hung thread was very sporadic.

What does refreshIdentityMapResult do to lock multiple threads?
What are the possible problems I need to look out if I do not call refreshIdentityMapResult?

Thanks, Amita
Re: Hung threads [message #1131359 is a reply to message #1130768] Thu, 10 October 2013 12:56 Go to previous messageGo to next message
Chris Delahunt is currently offline Chris Delahunt
Messages: 1034
Registered: July 2009
Senior Member
refreshIdentityMapResult does just what it says - it refreshes the objects being read in with the data from the database. It must lock the objects though so that any other threads that might be trying to access those objects (such as for other reads or writes) wait until it is done updating them, otherwise the data would be inconsistent and cause concurrency issues.

Care must always be taken with the refresh settings, but they are commonly used to avoid stale data. Problems that might occur from removing refreshing could be a greater occurrence of optimistic lock exceptions.
Re: Hung threads [message #1131474 is a reply to message #1131359] Thu, 10 October 2013 15:05 Go to previous messageGo to next message
Amita Mehta is currently offline Amita Mehta
Messages: 8
Registered: May 2013
Junior Member
Thanks Chris for explanation.

The problem is that the locks never get released. I have let the process run over 2 days and the locks are still not released.
Another curious thing is that I have named queries that have this setting set in the workbench but I haven't seen the named queries hang so far. Is the locking different for the different types of queries?
We migrated from TopLink to EclipseLink for this hung thread problems and are not using JPA.
Re: Hung threads [message #1131830 is a reply to message #1129432] Thu, 10 October 2013 20:47 Go to previous message
Amita Mehta is currently offline Amita Mehta
Messages: 8
Registered: May 2013
Junior Member
How does EclipseLink support work? Do we use our existing Oracle support?
Thanks
Previous Topic:JPA: subquery SQL generation bug
Next Topic:Utilizing value in joined table (ExpressionBuilder)
Goto Forum:
  


Current Time: Thu Oct 23 20:43:44 GMT 2014

Powered by FUDForum. Page generated in 0.02504 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software