| Re: [eclipselink-users] blocked threads, oracle11g and tomcat |
Tom, Upgrading is not an option for us. This would involve a long integration and test traject. The kind of issue your link points to is not the issue we are facing. See attached threaddump. Thread t@185 holds a lock on a physical connection. A lot of other threads are blocked on acquiring this lock. Those threads are all trying to close a statement which holds a reference to a connection which is now being used by another thread namely t@185. Basic question is: why is Eclipselink trying to access a connection which has already been closed (and retrieved from the pool by another thread). Greetings, Huub -----Original Message----- From: eclipselink-users-bounces@xxxxxxxxxxx [mailto:eclipselink-users-bounces@xxxxxxxxxxx] On Behalf Of Tom Ware Sent: Tuesday, June 04, 2013 2:29 PM To: eclipselink-users@xxxxxxxxxxx Subject: Re: [eclipselink-users] blocked threads, oracle11g and tomcat 'any chance of trying a newer version of EclipseLink. We have fixed quite a number of concurrency issues since 1.1.4. A quick search yielded this as a candidate: https://bugs.eclipse.org/bugs/show_bug.cgi?id=333902 On 03/06/2013 4:17 PM, Huub Sepers wrote: > Hi, > > Setup of our application: > > eclipselink 1.1.4 > > tomcat-7.0.29 (using tomcat jdbc pool or dbcp pool) > > Oracle 11G. We use latest oracle thin 11G jdbc driver. > > All our connections are obtained through JNDI from an external connection pool > (tomcat-jdbc or dbcp) > > We are experiencing blocked threads while running our application. > > The scenario is roughly as follows: > > Thread-A has send a batch of insert statements to the database and is in a wait > state. It holds a lock > > on a T4CConnection object. > > Thread-B also tries to do a batch of inserts. However it tries to obtain a lock > on the same T4CConnection object > > as Thread-A. > > Thread-B's stack includes a call to DatabaseAccessor.reconnect() which calls > DatabaseAccessor.clearStatementCache() > > which tries to close a statement in the DatabaseAccessor statement cache. > > So it seems that Thread-B has a stament in the cache which has a connection that > had already been closed and > > returned to the pool and that connection is now used by Thread-A (!?). > > Note: when running our application on oc4j in combination with it's connection > pool and Oracle 10G we never experienced these locks. > > Question: > > - How is the DatabaseAccessor statement cache supposed to work with a connection > pool. Trying to close > > statements in the cache will lead to accessing connections that have been closed? > > "http-bio-8280-exec-8" - Thread t@106 > > java.lang.Thread.State: RUNNABLE > > at java.net.SocketInputStream.socketRead0(Native Method) > > at java.net.SocketInputStream.read(SocketInputStream.java:129) > > at oracle.net.ns.Packet.receive(Packet.java:300) > > at oracle.net.ns.DataPacket.receive(DataPacket.java:106) > > at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:315) > > at oracle.net.ns.NetInputStream.read(NetInputStream.java:260) > > at oracle.net.ns.NetInputStream.read(NetInputStream.java:185) > > at oracle.net.ns.NetInputStream.read(NetInputStream.java:102) > > at > oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketI nputStreamWrapper.java:124) > > at > oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStream Wrapper.java:80) > > at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1137) > > at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:290) > > at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192) > > at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531) > > at > oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.jav a:207) > > at > oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatem ent.java:1044) > > at > oracle.jdbc.driver.OraclePreparedStatement.executeForRowsWithTimeout(Ora clePreparedStatement.java:10143) > > at > oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedSt atement.java:10249) > > - locked <19740c7> (a oracle.jdbc.driver.T4CConnection) > > at > oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWr apper.java:230) > > at > org.apache.tomcat.dbcp.dbcp.DelegatingStatement.executeBatch(DelegatingS tatement.java:297) > > at > org.apache.tomcat.dbcp.dbcp.DelegatingStatement.executeBatch(DelegatingS tatement.java:297) > > at > org.eclipse.persistence.platform.database.oracle.Oracle10Platform.execut eBatch(Oracle10Platform.java:53) > > at > org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.execute JDK12BatchStatement(DatabaseAccessor.java:801) > > at > org.eclipse.persistence.internal.databaseaccess.ParameterizedSQLBatchWri tingMechanism.executeBatchedStatements(ParameterizedSQLBatchWritingMecha nism.java:138) > > at > org.eclipse.persistence.internal.databaseaccess.ParameterizedSQLBatchWri tingMechanism.appendCall(ParameterizedSQLBatchWritingMechanism.java:82) > > at > org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicEx ecuteCall(DatabaseAccessor.java:546) > > at > org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.execute Call(DatabaseAccessor.java:512) > > at > org.eclipse.persistence.internal.sessions.AbstractSession.executeCall(Ab stractSession.java:872) > > at > org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.ex ecuteCall(DatasourceCallQueryMechanism.java:205) > > at > org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.ex ecuteCall(DatasourceCallQueryMechanism.java:191) > > at > org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.ex ecuteNoSelectCall(DatasourceCallQueryMechanism.java:234) > > at > org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.ex ecuteNoSelect(DatasourceCallQueryMechanism.java:214) > > at > org.eclipse.persistence.internal.queries.StatementQueryMechanism.execute NoSelect(StatementQueryMechanism.java:115) > > at > org.eclipse.persistence.queries.DataModifyQuery.executeDatabaseQuery(Dat aModifyQuery.java:85) > > at > org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java :664) > > at > org.eclipse.persistence.queries.DatabaseQuery.executeInUnitOfWork(Databa seQuery.java:583) > > at > org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecute Query(UnitOfWorkImpl.java:2750) > > at > org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(A bstractSession.java:1181) > > at > org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(A bstractSession.java:1165) > > at > org.eclipse.persistence.mappings.ManyToManyMapping.insertIntoRelationTab le(ManyToManyMapping.java:852) > > at > org.eclipse.persistence.mappings.ManyToManyMapping.performDataModificati onEvent(ManyToManyMapping.java:1010) > > at > org.eclipse.persistence.internal.sessions.CommitManager.commitAllObjects WithChangeSet(CommitManager.java:130) > > at > org.eclipse.persistence.internal.sessions.AbstractSession.writeAllObject sWithChangeSet(AbstractSession.java:3176) > > at > org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabas e(UnitOfWorkImpl.java:1299) > > at > org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabas eWithChangeSet(UnitOfWorkImpl.java:1399) > > at > org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitRootUnitO fWork(UnitOfWorkImpl.java:1230) > > at > org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commit(UnitOfWo rkImpl.java:1008) > > at > nl.portinfolink.platform.domain.handler.PilDataObjectHandlerImpl.commitU nitOfWork(PilDataObjectHandlerImpl.java:329) > > at > nl.portinfolink.platform.domain.handler.PilDataObjectHandlerImpl.commitC hanges(PilDataObjectHandlerImpl.java:174) > > at > nl.portinfolink.platform.util.ToplinkUtils.commitChanges(ToplinkUtils.ja va:414) > > at > nl.portinfolink.platform.facade.BaseWebFacade.commitWebTransaction(BaseW ebFacade.java:66) > > at > nl.portinfolink.platform.facade.BaseWebFacade.handleWebTransaction(BaseW ebFacade.java:92) > > at > nl.portbase.road.facade.RoadPlanningWebFacade.save(RoadPlanningWebFacade .java:153) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav a:39) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor Impl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > nl.portinfolink.portal.actions.ServiceProcessWebFacadeCaller.invokeWebFa cadeMethod(ServiceProcessWebFacadeCaller.java:314) > > at > nl.portinfolink.portal.actions.ServiceProcessWebFacadeCaller.execute(Ser viceProcessWebFacadeCaller.java:268) > > "http-bio-8280-exec-53" - Thread t@166 > > java.lang.Thread.State: BLOCKED > > at oracle.jdbc.driver.OracleStatement.close(OracleStatement.java:1559) > > - waiting to lock <19740c7> (a oracle.jdbc.driver.T4CConnection) > owned by "http-bio-8280-exec-8" t@106 > > at > oracle.jdbc.driver.OracleStatementWrapper.close(OracleStatementWrapper.j ava:94) > > at > oracle.jdbc.driver.OraclePreparedStatementWrapper.close(OraclePreparedSt atementWrapper.java:80) > > at > org.apache.tomcat.dbcp.dbcp.DelegatingStatement.close(DelegatingStatemen t.java:168) > > at > org.apache.tomcat.dbcp.dbcp.DelegatingStatement.close(DelegatingStatemen t.java:168) > > at > org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.clearSt atementCache(DatabaseAccessor.java:308) > > at > org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.reconne ct(DatabaseAccessor.java:1399) > > at > org.eclipse.persistence.internal.databaseaccess.DatasourceAccessor.incre mentCallCount(DatasourceAccessor.java:300) > > - locked <5f3edd> (a > org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor) > > at > org.eclipse.persistence.internal.databaseaccess.DatasourceAccessor.begin Transaction(DatasourceAccessor.java:233) > > at > org.eclipse.persistence.internal.sessions.AbstractSession.basicBeginTran saction(AbstractSession.java:398) > > at > org.eclipse.persistence.internal.sessions.AbstractSession.basicBeginTran saction(AbstractSession.java:387) > > at > org.eclipse.persistence.sessions.server.ClientSession.basicBeginTransact ion(ClientSession.java:127) > > at > org.eclipse.persistence.internal.sessions.AbstractSession.beginTransacti on(AbstractSession.java:552) > > at > org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.beginTransactio n(UnitOfWorkImpl.java:509) > > at > org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabas e(UnitOfWorkImpl.java:1263) > > at > org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabas eWithChangeSet(UnitOfWorkImpl.java:1399) > > at > org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitRootUnitO fWork(UnitOfWorkImpl.java:1230) > > at > org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commit(UnitOfWo rkImpl.java:1008) > > at > nl.portinfolink.platform.domain.handler.PilDataObjectHandlerImpl.commitU nitOfWork(PilDataObjectHandlerImpl.java:329) > > ... > > Greetings, > > Huub > > > > _______________________________________________ > eclipselink-users mailing list > eclipselink-users@xxxxxxxxxxx > https://dev.eclipse.org/mailman/listinfo/eclipselink-users > _______________________________________________ eclipselink-users mailing list eclipselink-users@xxxxxxxxxxx https://dev.eclipse.org/mailman/listinfo/eclipselink-users
Attachment:
threaddump-1370016129244.tdump
Description: threaddump-1370016129244.tdump