| Stuck thread when EJBQueryImpl.getSingleResult [message #1787500] |
Sun, 27 May 2018 07:49  |
 Wei Xiang Tam
Messages: 1
Registered: May 2018 |
Junior Member |
|
|
Hi all,
I was facing a stuck thread issue when trying to retrieve particular record from DB table. the stuck thread happen whenever the query involve the same DB record; and more and more stuck thread accumulate under the weblogic server and it has impacted to the request from the client
Based on input from DBA, there was a DB contention happening when the stuck thread happening (DB contention has last for 15 mins) and there is row-lock (that is transaction lock) for the DB record during the incident happen (one thread trying to update that record), but there is no symptoms for lock not getting released
can i know even if the DB contention over, why the retrieval of that particular DB record is not responding and causing stuck thread accumulating?
The framework i am using
Eclipselink - v2.3.3
Weblogic - v12.1.1
JDK - 1.7.045
<May 5, 2018 8:00:38 AM SGT> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '335' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "607" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 607447 ms
", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:503)
org.eclipse.persistence.internal.helper.ConcurrencyManager.acquire(ConcurrencyManager.java:94)
org.eclipse.persistence.internal.identitymaps.CacheKey.acquire(CacheKey.java:126)
org.eclipse.persistence.internal.identitymaps.AbstractIdentityMap.acquireLock(AbstractIdentityMap.java:122)
org.eclipse.persistence.internal.identitymaps.IdentityMapManager.acquireLock(IdentityMapManager.java:144)
org.eclipse.persistence.internal.sessions.IdentityMapAccessor.acquireLock(IdentityMapAccessor.java:92)
org.eclipse.persistence.internal.sessions.IdentityMapAccessor.acquireLock(IdentityMapAccessor.java:83)
org.eclipse.persistence.internal.sessions.AbstractSession.retrieveCacheKey(AbstractSession.java:4688)
org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:772)
org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildWorkingCopyCloneNormally(ObjectBuilder.java:715)
org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObjectInUnitOfWork(ObjectBuilder.java:668)
org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:601)
org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:560)
org.eclipse.persistence.queries.ObjectLevelReadQuery.buildObject(ObjectLevelReadQuery.java:717)
org.eclipse.persistence.queries.ReadAllQuery.registerResultInUnitOfWork(ReadAllQuery.java:769)
org.eclipse.persistence.queries.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:433)
org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1081)
org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:844)
org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1040)
org.eclipse.persistence.queries.ReadAllQuery.execute(ReadAllQuery.java:392)
org.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:1128)
org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2871)
org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1516)
org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1498)
org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1463)
org.eclipse.persistence.internal.jpa.EJBQueryImpl.executeReadQuery(EJBQueryImpl.java:485)
org.eclipse.persistence.internal.jpa.EJBQueryImpl.getSingleResult(EJBQueryImpl.java:773)
|
|
|
| Re: Stuck thread when EJBQueryImpl.getSingleResult [message #1787604 is a reply to message #1787500] |
Tue, 29 May 2018 22:04  |
Chris Delahunt
Messages: 1389
Registered: July 2009 |
Senior Member |
|
|
One thread is not enough to go on. The thread you posted is just a reading an object and trying to obtain a lock on the object in the shared cache so it can clone it to track changes. It gets stuck waiting for some other thread to release its lock before it can obtain its own. You would need to look at other threads to see why they might have a lock and not completing on the object this thread is trying to return. If all threads are just like this one, you would need to take thread dump at intervals to see if they are all locked, or if threads are progressing slowly.
The situation you described would form a bottleneck on the objects involved, and just like an accident on a particular highway off ramp, once it backs up, takes time to clear after the initial accident itself is cleared. You'll have to check to see if you have more threads coming in and accessing those object then can be handled, and maybe check that you don't have some processes retrying slow transactions repeatedly, contributing to the problem under load.
[Updated on: Tue, 29 May 2018 22:09] Report message to a moderator |
|
|
Powered by
FUDForum. Page generated in 0.06349 seconds
] 
Search
Help
Register
Login
Home
