Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » EclipseLink » application freeze
application freeze [message #819006] Mon, 12 March 2012 07:15 Go to next message
Tom Eugelink is currently offline Tom Eugelink
Messages: 807
Registered: July 2009
Senior Member
My application freezes up fully when doing a quick fetch for a article description in the EDT. As the thread dump below shows, the AWT-EventQueue-0 is stuck in the acquiring of a connection from the pool.
I understand from the documentation that there is a default maximum of 32 connections, however my application's debug code indicate that only 4 EM's are open.
- Is there a way to log the pool information?

I hoped to unfreeze the app by setting a timeout on the EMF, but that does not seem to help.

lOptions.put(PersistenceUnitProperties.JDBC_CONNECTIONS_WAIT, "10000");

Eclipselink 2.3.0, Java 1.6.0_17 against Informix.

Tom






Full thread dump Java HotSpot(TM) Client VM (21.0-b17 mixed mode):

"Foxtrot Multi Worker Thread Runner #3" daemon prio=6 tid=0x3e740000 nid=0x13c i
n Object.wait()
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:503)
at foxtrot.workers.SingleWorkerThread.takeTask(SingleWorkerThread.java:1
32)
- locked <0x16ffb7f0> (a foxtrot.workers.MultiWorkerThread)
at foxtrot.workers.SingleWorkerThread.run(SingleWorkerThread.java:180)
at java.lang.Thread.run(Thread.java:722)

"Foxtrot Multi Worker Thread Runner #1" daemon prio=6 tid=0x3e8ad400 nid=0xb2c i
n Object.wait()
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:503)
at foxtrot.workers.SingleWorkerThread.takeTask(SingleWorkerThread.java:1
32)
- locked <0x16f7e400> (a foxtrot.workers.MultiWorkerThread)
at foxtrot.workers.SingleWorkerThread.run(SingleWorkerThread.java:180)
at java.lang.Thread.run(Thread.java:722)

"Thread-4 StateChecker" prio=6 tid=0x3e0d0c00 nid=0x884 waiting on condition
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at nl.reinders.Reinders$6.run(Reinders.java:677)
at java.lang.Thread.run(Thread.java:722)

"SwingWorker-pool-2-thread-1" daemon prio=6 tid=0x3d472800 nid=0x6d8 waiting on
condition
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1302eba8> (a java.util.concurrent.locks.Abstra
ctQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject
..await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.jav
a:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.ja
va:1043)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1103)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
..java:603)
at java.lang.Thread.run(Thread.java:722)

"D3D Screen Updater" daemon prio=8 tid=0x00903400 nid=0x70c in Object.wait()
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at sun.java2d.d3d.D3DScreenUpdateManager.run(D3DScreenUpdateManager.java
:436)
- locked <0x1301bbb0> (a java.lang.Object)
at java.lang.Thread.run(Thread.java:722)

"TimerQueue" daemon prio=6 tid=0x3d5c3000 nid=0x4ac waiting on condition
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x12d16950> (a java.util.concurrent.locks.Abstra
ctQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject
..await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:209)
at javax.swing.TimerQueue.run(TimerQueue.java:171)
at java.lang.Thread.run(Thread.java:722)

"DestroyJavaVM" prio=6 tid=0x00916400 nid=0x300 waiting on condition
java.lang.Thread.State: RUNNABLE

"AWT-EventQueue-0" prio=6 tid=0x3d538c00 nid=0x298 in Object.wait()

java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.eclipse.persistence.sessions.server.ConnectionPool.acquireConnect
ion(ConnectionPool.java:102)
- locked <0x13f45cf0> (a org.eclipse.persistence.sessions.server.Connect
ionPool)
at org.eclipse.persistence.sessions.server.ServerSession.allocateReadCon
nection(ServerSession.java:477)
at org.eclipse.persistence.sessions.server.ServerSession.executeCall(Ser
verSession.java:525)
at org.eclipse.persistence.internal.sessions.IsolatedClientSession.execu
teCall(IsolatedClientSession.java:133)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism
..executeCall(DatasourceCallQueryMechanism.java:206)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism
..executeCall(DatasourceCallQueryMechanism.java:192)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism
..selectOneRow(DatasourceCallQueryMechanism.java:639)
at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.sel
ectOneRowFromTable(ExpressionQueryMechanism.java:2582)
at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.sel
ectOneRow(ExpressionQueryMechanism.java:2553)
at org.eclipse.persistence.queries.ReadObjectQuery.executeObjectLevelRea
dQuery(ReadObjectQuery.java:431)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQ
uery(ObjectLevelReadQuery.java:997)
at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.j
ava:675)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLe
velReadQuery.java:958)
at org.eclipse.persistence.queries.ReadObjectQuery.execute(ReadObjectQue
ry.java:399)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfW
ork(ObjectLevelReadQuery.java:1021)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExec
uteQuery(UnitOfWorkImpl.java:2898)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuer
y(AbstractSession.java:1225)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuer
y(AbstractSession.java:1207)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuer
y(AbstractSession.java:1167)
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.executeQuery(E
ntityManagerImpl.java:733)
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.findInternal(E
ntityManagerImpl.java:677)
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityMan
agerImpl.java:581)
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityMan
agerImpl.java:460)
at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at nl.knowledgeplaza.util.jpa.EntityManagerExtender.invoke(EntityManager
Extender.java:545)
at $Proxy4.find(Unknown Source)
at nl.reinders.bm.generated.Sellorder.findOptionallyLockByPK(Sellorder.j
ava:2910)
at nl.reinders.bm.generated.Sellorder.findByPK(Sellorder.java:2920)
at nl.reinders.bm.generated.Sellorder.findByPK(Sellorder.java:2946)
at nl.reinders.RND_SellOrderGUIWindow$1.store(RND_SellOrderGUIWindow.jav
a:105)
at com.ovsoftware.oid.OVS_OID_Handler_1.actionPerformed(OVS_OID_Handler_
1.java:454)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:20
18)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.jav
a:2341)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel
..java:402)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259
)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonL
istener.java:252)
at java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:2
89)
at java.awt.Component.processMouseEvent(Component.java:6504)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3321)
at java.awt.Component.processEvent(Component.java:6269)
at java.awt.Container.processEvent(Container.java:2229)
at java.awt.Component.dispatchEventImpl(Component.java:4860)
at java.awt.Container.dispatchEventImpl(Container.java:2287)
at java.awt.Component.dispatchEvent(Component.java:4686)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4832
)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4492)

at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4422)
at java.awt.Container.dispatchEventImpl(Container.java:2273)
at java.awt.Window.dispatchEventImpl(Window.java:2713)
at java.awt.Component.dispatchEvent(Component.java:4686)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:707)
at java.awt.EventQueue.access$000(EventQueue.java:101)
at java.awt.EventQueue$3.run(EventQueue.java:666)
at java.awt.EventQueue$3.run(EventQueue.java:664)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDo
main.java:76)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDo
main.java:87)
at java.awt.EventQueue$4.run(EventQueue.java:680)
at java.awt.EventQueue$4.run(EventQueue.java:678)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDo
main.java:76)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:677)
at org.tbee.swing.EventQueue.dispatchEvent(EventQueue.java:83)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThre
ad.java:211)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.
java:128)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThre
ad.java:117)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:113)

at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:105)

at java.awt.EventDispatchThread.run(EventDispatchThread.java:90)

"AWT-Windows" daemon prio=6 tid=0x3d521000 nid=0x6ec runnable
java.lang.Thread.State: RUNNABLE
at sun.awt.windows.WToolkit.eventLoop(Native Method)
at sun.awt.windows.WToolkit.run(WToolkit.java:299)
at java.lang.Thread.run(Thread.java:722)

"AWT-Shutdown" prio=6 tid=0x3d51f800 nid=0x6f0 in Object.wait()
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:503)
at sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:287)
- locked <0x12d21f00> (a java.lang.Object)
at java.lang.Thread.run(Thread.java:722)

"Java2D Disposer" daemon prio=10 tid=0x3d51e800 nid=0x6b8 in Object.wait()
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
- locked <0x12d21f90> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
at sun.java2d.Disposer.run(Disposer.java:145)
at java.lang.Thread.run(Thread.java:722)

"Service Thread" daemon prio=6 tid=0x3cf61000 nid=0x7b0 runnable
java.lang.Thread.State: RUNNABLE

"C1 CompilerThread0" daemon prio=10 tid=0x3cf5dc00 nid=0x7b4 waiting on conditio
n
java.lang.Thread.State: RUNNABLE

"Attach Listener" daemon prio=10 tid=0x3cf59800 nid=0x77c runnable
java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x3cf58400 nid=0x7a8 waiting on condition

java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=8 tid=0x009fc000 nid=0x740 in Object.wait()

java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
- locked <0x12d221c0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)

"Reference Handler" daemon prio=10 tid=0x009f7000 nid=0x2a8 in Object.wait() [0x
3cecf000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:503)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
- locked <0x12d16d20> (a java.lang.ref.Reference$Lock)

"VM Thread" prio=10 tid=0x009f1400 nid=0x3f0 runnable

"VM Periodic Task Thread" prio=10 tid=0x3cf74c00 nid=0x73c waiting on condition


JNI global references: 4950

Heap
def new generation total 78720K, used 52102K [0x02d00000, 0x08260000, 0x12d00
000)
eden space 70016K, 74% used [0x02d00000, 0x05fe1838, 0x07160000)
from space 8704K, 0% used [0x079e0000, 0x079e0000, 0x08260000)
to space 8704K, 0% used [0x07160000, 0x07160000, 0x079e0000)
tenured generation total 174784K, used 84593K [0x12d00000, 0x1d7b0000, 0x32d0
0000)
the space 174784K, 48% used [0x12d00000, 0x17f9c750, 0x17f9c800, 0x1d7b0000)

compacting perm gen total 37376K, used 37330K [0x32d00000, 0x35180000, 0x3ad00
000)
the space 37376K, 99% used [0x32d00000, 0x35174b38, 0x35174c00, 0x35180000)
No shared spaces configured.
Re: application freeze [message #821342 is a reply to message #819006] Thu, 15 March 2012 04:16 Go to previous messageGo to next message
Tom Eugelink is currently offline Tom Eugelink
Messages: 807
Registered: July 2009
Senior Member
I've been breaking my head over this, but I figure this may be a likely location of the problem.

Currently I'm migrating an older piece of code over to Eclipselink, but partially the old direct JDBC code is still used. In order to make sure both use the same connection, I get the JDBC connection that EL uses to write through.

entityManager.getTransaction().begin();
// Get JDBC connection. You should be in a JPA transaction to access the Connection. Otherwise, you will be responsible for releasing the connection.
UnitOfWorkImpl lUnitOfWork = (UnitOfWorkImpl)EclipselinkUtil.castToJpaEntityManager(entityManager).getActiveSession();
lUnitOfWork.beginEarlyTransaction();
Accessor lAccessor = lUnitOfWork.getAccessor();
lAccessor.incrementCallCount(lUnitOfWork.getParent());
lAccessor.decrementCallCount();
Connection lConnection = lAccessor.getConnection();

If the connection I obtained is NOT released by EL, then I can imagine that the pool will slowly run dry.

Will the connection be release back into the pool or not?
Can someone please explain in detail what the code above does (I copied it from the internet)?

Tom
Re: application freeze (found the cause, now need the solution) [message #821615 is a reply to message #821342] Thu, 15 March 2012 11:57 Go to previous messageGo to next message
Tom Eugelink is currently offline Tom Eugelink
Messages: 807
Registered: July 2009
Senior Member
Ok, I've hooked into the eclipselink.jar using aspectj and analyzed when connections where obtained and returned. Certain connections were indeed never returned, like this one:

2012-03-15 16:47:03,716 INFO EclipseLinkPoolUsage.ajc$afterReturning$EclipseLinkPoolUsage$1$aa7e6855(pool.aj:9) !!! acquireConnection: execution(Accessor org.eclipse.persistence.sessions.server.ConnectionPool.acquireConnection())=27915053
java.lang.Throwable
at EclipseLinkPoolUsage.ajc$afterReturning$EclipseLinkPoolUsage$1$aa7e6855(pool.aj:8)
at org.eclipse.persistence.sessions.server.ConnectionPool.acquireConnection(ConnectionPool.java:145)
at org.eclipse.persistence.sessions.server.ServerSession.acquireClientConnection(ServerSession.java:241)
at org.eclipse.persistence.sessions.server.ClientSession.basicBeginTransaction(ClientSession.java:130)
at org.eclipse.persistence.internal.sessions.AbstractSession.beginTransaction(AbstractSession.java:580)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.beginTransaction(UnitOfWorkImpl.java:554)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.beginEarlyTransaction(UnitOfWorkImpl.java:544)
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.getActivePersistenceContext(EntityManagerImpl.java:1542)
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.getActiveSession(EntityManagerImpl.java:1042)
at nl.knowledgeplaza.util.jpa.EclipselinkUtil.getJdbcConnectionForWrite(EclipselinkUtil.java:135)

So it indeed is this line in the code from the previous post that allocates a connection, but never returns it:

UnitOfWorkImpl lUnitOfWork = (UnitOfWorkImpl)EclipselinkUtil.castToJpaEntityManager(entityManager).getActiveSession();

Now I need to know how to obtain the database connection Eclipselink uses for updates without bleeding the pool dry, or how to correctly release it again.

Tom
Re: application freeze (found the cause, now need the solution) [message #824075 is a reply to message #821615] Mon, 19 March 2012 04:20 Go to previous messageGo to next message
Tom Eugelink is currently offline Tom Eugelink
Messages: 807
Registered: July 2009
Senior Member
> So it indeed is this line in the code from the previous post that allocates a connection, but never returns it:

I found the code I use in the Eclipselink wiki:

http://wiki.eclipse.org/EclipseLink/Examples/JPA/EMAPI#Getting_a_JDBC_Connection_from_an_EntityManager

As said, the second approach does not release the connection back into the pool.
Re: application freeze (found the cause, now need the solution) [message #824111 is a reply to message #824075] Mon, 19 March 2012 05:16 Go to previous messageGo to next message
Tom Eugelink is currently offline Tom Eugelink
Messages: 807
Registered: July 2009
Senior Member
I'm getting the eerie feeling that my "missing commit" issue (discussed in another thread) is related to this issue; not committing, not returning the connection to the pool...

There is no commercial support available on Eclipselink?
Re: application freeze (found the cause, now need the solution) [message #824350 is a reply to message #824111] Mon, 19 March 2012 11:10 Go to previous messageGo to next message
Tom Eugelink is currently offline Tom Eugelink
Messages: 807
Registered: July 2009
Senior Member
> I'm getting the eerie feeling that my "missing commit" issue (discussed in another thread) is related to this issue; not committing, not returning the connection to the pool...

I've upgraded to EL 2.3.3-M1 losing the ability to weave using -javaagent because of the class loader issues.

The upgrade allowed me to use the JPA 2.0 way of obtaining a connection (unwrap) and that seems to work. Testing...
Re: application freeze (found the cause, now need the solution) [message #824379 is a reply to message #824111] Mon, 19 March 2012 11:59 Go to previous messageGo to next message
Chris Delahunt is currently offline Chris Delahunt
Messages: 1017
Registered: July 2009
Senior Member
Oracle provides commercial support for EclipseLink:
http://www.oracle.com/us/corporate/press/017498_EN

Can you provide the settings you are using to log in with? See
http://wiki.eclipse.org/Using_EclipseLink_JPA_Extensions_(ELUG)#How_to_Use_EclipseLink_JPA_Extensions_for_JDBC_Connection_Pooling
for setting up the internal EclipseLink connection pools - I do not believe it defaults to 32 connections, and in looking at the log from your other post, I can see only 3 connections being initialized at startup.
Are their any other settings you have configured for your persistence unit such as exclusive connections etc?

I just now found https://bugs.eclipse.org/bugs/show_bug.cgi?id=276505 which might be related. Please vote for it if it is related - can you try testing the app on a different database to see if you get the same behavior?
Re: application freeze (found the cause, now need the solution) [message #824394 is a reply to message #824379] Mon, 19 March 2012 12:15 Go to previous messageGo to next message
Chris Delahunt is currently offline Chris Delahunt
Messages: 1017
Registered: July 2009
Senior Member
You might also want to try the EclipseLink 2.4 nightly build to rule out the possiblity of hitting bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=370855.

Best Regards,
Chris
Re: application freeze (found the cause, now need the solution) [message #824467 is a reply to message #824394] Mon, 19 March 2012 13:55 Go to previous message
Tom Eugelink is currently offline Tom Eugelink
Messages: 807
Registered: July 2009
Senior Member
Is that available in Maven?


On 2012-03-19 17:15, Chris Delahunt wrote:
> You might also want to try the EclipseLink 2.4 nightly build to rule out the possiblity of hitting bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=370855.
>
> Best Regards,
> Chris
Previous Topic:missing commit
Next Topic:@Version and DescriptorException
Goto Forum:
  


Current Time: Fri Aug 22 23:37:07 EDT 2014

Powered by FUDForum. Page generated in 0.02204 seconds