[CDO] scalability test problem [message #500491] |
Thu, 26 November 2009 11:13 |
Lothar Werzinger Messages: 153 Registered: July 2009 Location: Bay Area |
Senior Member |
|
|
--nextPart21350895.kcvHbq28VH
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8Bit
Hi,
I am trying to do a scalability test where I try to add 500000 objects to a
CDO repository. The model I use is proprietary, but similar to the Library
example in the fact that the library has a feature containing all the books
in the library and my model has a similar feature where I try to add those
500000 objects.
The code that I use to add the objects opens a new session and transaction
for every 10000 objects and closes both transaction and session after the
commit. I did that to minimize the resource needs in both server and
client.
I also use a CollectionLoadingPolicy to avoid loading all the objects into
memory:
> CDOCollectionLoadingPolicy policy =
> CDOUtil.createCollectionLoadingPolicy(0, 100);
> session.options().setCollectionLoadingPolicy(policy);
However after ca. 200000 objects I get exceptions. The server and
application log is attached. I also attached the cdo server config file and
a snippet of the code where I create the objects.
I would appreciate any help to create a configuration that works with 500000
or more (millions) of objects.
P.S.
After I can successfully create such a large repository I am planing to
access it with queries to test the scalability accessing/modifying such a
large repository.
Lothar
--nextPart21350895.kcvHbq28VH
Content-Type: text/plain; name="addobjects.txt"
Content-Transfer-Encoding: 8Bit
Content-Disposition: attachment; filename="addobjects.txt"
try
{
// Prepare container
IManagedContainer container = ContainerUtil.createContainer();
Net4jUtil.prepareContainer(container); // Register Net4j factories
TCPUtil.prepareContainer(container); // Register TCP factories
CDONet4jUtil.prepareContainer(container); // Register CDO factories
container.activate();
// Create connector
IConnector connector = TCPUtil.getConnector(container, "localhost:2036"); //$NON-NLS-1$
CDOSessionConfiguration configuration = CDONet4jUtil.createSessionConfiguration();
configuration.setConnector(connector);
configuration.setRepositoryName("testapp");
long numiterations = 50;
for(long j = 1; j <= numiterations; ++j)
{
System.out.println("" + j + " of " + numiterations + " creating ...");
// Open session
CDOSession session = configuration.openSession();
//chunk load large ELists
CDOCollectionLoadingPolicy policy = CDOUtil.createCollectionLoadingPolicy(0, 100);
session.options().setCollectionLoadingPolicy(policy);
CDOTransaction transaction = session.openTransaction();
// add 10000 objects
System.out.println("" + j + " of " + numiterations + " committing ...");
transaction.commit();
transaction.close();
session.close();
System.out.println("" + j + " of " + numiterations + " committing done");
}
}
catch(Throwable t)
{
t.printStackTrace();
}
--nextPart21350895.kcvHbq28VH
Content-Type: text/plain; name="app.log"
Content-Transfer-Encoding: 8Bit
Content-Disposition: attachment; filename="app.log"
calcName(209997): vqyl
calcName(209998): wqyl
calcName(209999): xqyl
j=20of 50 committing ...
[ERROR] EOFException
java.io.EOFException
at java.io.DataInputStream.readBoolean(DataInputStream.java:227 )
at org.eclipse.net4j.util.io.ExtendedDataInput$Delegating.readB oolean(ExtendedDataInput.java:55)
at org.eclipse.emf.internal.cdo.net4j.protocol.CommitTransactio nRequest.confirmingCheckError(CommitTransactionRequest.java: 229)
at org.eclipse.emf.internal.cdo.net4j.protocol.CommitTransactio nRequest.confirming(CommitTransactionRequest.java:162)
at org.eclipse.emf.internal.cdo.net4j.protocol.CommitTransactio nRequest.confirming(CommitTransactionRequest.java:120)
at org.eclipse.emf.internal.cdo.net4j.protocol.CommitTransactio nRequest.confirming(CommitTransactionRequest.java:1)
at org.eclipse.net4j.signal.RequestWithMonitoring.confirming(Re questWithMonitoring.java:169)
at org.eclipse.net4j.signal.RequestWithConfirmation.doExtendedI nput(RequestWithConfirmation.java:123)
at org.eclipse.net4j.signal.Signal.doInput(Signal.java:312)
at org.eclipse.net4j.signal.RequestWithConfirmation.doExecute(R equestWithConfirmation.java:103)
at org.eclipse.net4j.signal.RequestWithMonitoring.doExecute(Req uestWithMonitoring.java:233)
at org.eclipse.net4j.signal.SignalActor.execute(SignalActor.jav a:66)
at org.eclipse.net4j.signal.Signal.runSync(Signal.java:239)
at org.eclipse.net4j.signal.SignalProtocol.startSignal(SignalPr otocol.java:423)
at org.eclipse.net4j.signal.RequestWithConfirmation.doSend(Requ estWithConfirmation.java:87)
at org.eclipse.net4j.signal.RequestWithConfirmation.send(Reques tWithConfirmation.java:73)
at org.eclipse.net4j.signal.RequestWithMonitoring.send(RequestW ithMonitoring.java:108)
at org.eclipse.emf.internal.cdo.net4j.protocol.CDOClientProtoco l.send(CDOClientProtocol.java:302)
at org.eclipse.emf.internal.cdo.net4j.protocol.CDOClientProtoco l.commitTransaction(CDOClientProtocol.java:220)
at org.eclipse.emf.internal.cdo.transaction.CDOSingleTransactio nStrategyImpl.commit(CDOSingleTransactionStrategyImpl.java:5 8)
at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl. commit(CDOTransactionImpl.java:598)
at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl. commit(CDOTransactionImpl.java:612)
at testapp.perspective.handlers.CreateEntitiesHandler.execute(C reateEntitiesHandler.java:138)
at org.eclipse.ui.internal.handlers.HandlerProxy.execute(Handle rProxy.java:294)
at org.eclipse.core.commands.Command.executeWithChecks(Command. java:476)
at org.eclipse.core.commands.ParameterizedCommand.executeWithCh ecks(ParameterizedCommand.java:508)
at org.eclipse.ui.internal.handlers.HandlerService.executeComma nd(HandlerService.java:169)
at org.eclipse.ui.internal.handlers.SlaveHandlerService.execute Command(SlaveHandlerService.java:241)
at org.eclipse.ui.menus.CommandContributionItem.handleWidgetSel ection(CommandContributionItem.java:770)
at org.eclipse.ui.menus.CommandContributionItem.access$10(Comma ndContributionItem.java:756)
at org.eclipse.ui.menus.CommandContributionItem$5.handleEvent(C ommandContributionItem.java:746)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java :84)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1176)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.ja va:3493)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java :3112)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.jav a:2405)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2369)
at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:22 21)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:500)
at org.eclipse.core.databinding.observable.Realm.runWithDefault (Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Work bench.java:493)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.j ava:149)
at testapp.Application.start(Application.java:20)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(Eclips eAppHandle.java:194)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher .runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher .start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseS tarter.java:368)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseS tarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce ssorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe thodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java: 559)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:514)
at org.eclipse.equinox.launcher.Main.run(Main.java:1311)
at org.eclipse.equinox.launcher.Main.main(Main.java:1287)
!SESSION 2009-11-26 01:42:17.864 -----------------------------------------------
eclipse.buildId=unknown
java.version=1.6.0_01
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US
Framework arguments: -application TestApp.common.application
Command-line arguments: -application TestApp.common.application -data /workspaces/rcp-3.5/../runtime-TestApp.common.application -dev file:/workspaces/rcp-3.5/.metadata/.plugins/org.eclipse.pde. core/TestApp.rcp/dev.properties -os linux -ws gtk -arch x86_64 -consolelog -console
!ENTRY org.eclipse.net4j 4 0 2009-11-26 02:30:19.963
!MESSAGE EOFException
!STACK 0
java.io.EOFException
at java.io.DataInputStream.readBoolean(DataInputStream.java:227 )
at org.eclipse.net4j.util.io.ExtendedDataInput$Delegating.readB oolean(ExtendedDataInput.java:55)
at org.eclipse.emf.internal.cdo.net4j.protocol.CommitTransactio nRequest.confirmingCheckError(CommitTransactionRequest.java: 229)
at org.eclipse.emf.internal.cdo.net4j.protocol.CommitTransactio nRequest.confirming(CommitTransactionRequest.java:162)
at org.eclipse.emf.internal.cdo.net4j.protocol.CommitTransactio nRequest.confirming(CommitTransactionRequest.java:120)
at org.eclipse.emf.internal.cdo.net4j.protocol.CommitTransactio nRequest.confirming(CommitTransactionRequest.java:1)
at org.eclipse.net4j.signal.RequestWithMonitoring.confirming(Re questWithMonitoring.java:169)
at org.eclipse.net4j.signal.RequestWithConfirmation.doExtendedI nput(RequestWithConfirmation.java:123)
at org.eclipse.net4j.signal.Signal.doInput(Signal.java:312)
at org.eclipse.net4j.signal.RequestWithConfirmation.doExecute(R equestWithConfirmation.java:103)
at org.eclipse.net4j.signal.RequestWithMonitoring.doExecute(Req uestWithMonitoring.java:233)
at org.eclipse.net4j.signal.SignalActor.execute(SignalActor.jav a:66)
at org.eclipse.net4j.signal.Signal.runSync(Signal.java:239)
at org.eclipse.net4j.signal.SignalProtocol.startSignal(SignalPr otocol.java:423)
at org.eclipse.net4j.signal.RequestWithConfirmation.doSend(Requ estWithConfirmation.java:87)
at org.eclipse.net4j.signal.RequestWithConfirmation.send(Reques tWithConfirmation.java:73)
at org.eclipse.net4j.signal.RequestWithMonitoring.send(RequestW ithMonitoring.java:108)
at org.eclipse.emf.internal.cdo.net4j.protocol.CDOClientProtoco l.send(CDOClientProtocol.java:302)
at org.eclipse.emf.internal.cdo.net4j.protocol.CDOClientProtoco l.commitTransaction(CDOClientProtocol.java:220)
at org.eclipse.emf.internal.cdo.transaction.CDOSingleTransactio nStrategyImpl.commit(CDOSingleTransactionStrategyImpl.java:5 8)
at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl. commit(CDOTransactionImpl.java:598)
at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl. commit(CDOTransactionImpl.java:612)
at testapp.perspective.handlers.CreateEntitiesHandler.execute(C reateEntitiesHandler.java:138)
at org.eclipse.ui.internal.handlers.HandlerProxy.execute(Handle rProxy.java:294)
at org.eclipse.core.commands.Command.executeWithChecks(Command. java:476)
at org.eclipse.core.commands.ParameterizedCommand.executeWithCh ecks(ParameterizedCommand.java:508)
at org.eclipse.ui.internal.handlers.HandlerService.executeComma nd(HandlerService.java:169)
at org.eclipse.ui.internal.handlers.SlaveHandlerService.execute Command(SlaveHandlerService.java:241)
at org.eclipse.ui.menus.CommandContributionItem.handleWidgetSel ection(CommandContributionItem.java:770)
at org.eclipse.ui.menus.CommandContributionItem.access$10(Comma ndContributionItem.java:756)
at org.eclipse.ui.menus.CommandContributionItem$5.handleEvent(C ommandContributionItem.java:746)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java :84)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1176)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.ja va:3493)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java :3112)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.jav a:2405)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2369)
at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:22 21)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:500)
at org.eclipse.core.databinding.observable.Realm.runWithDefault (Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Work bench.java:493)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.j ava:149)
at testapp.Application.start(Application.java:20)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(Eclips eAppHandle.java:194)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher .runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher .start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseS tarter.java:368)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseS tarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce ssorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe thodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java: 559)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:514)
at org.eclipse.equinox.launcher.Main.run(Main.java:1311)
at org.eclipse.equinox.launcher.Main.main(Main.java:1287)
org.eclipse.net4j.util.transaction.TransactionException: org.eclipse.emf.cdo.common.util.TransportException: java.util.concurrent.TimeoutException: Timeout
at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl. commit(CDOTransactionImpl.java:606)
at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl. commit(CDOTransactionImpl.java:612)
at testapp.perspective.handlers.CreateEntitiesHandler.execute(C reateEntitiesHandler.java:138)
at org.eclipse.ui.internal.handlers.HandlerProxy.execute(Handle rProxy.java:294)
at org.eclipse.core.commands.Command.executeWithChecks(Command. java:476)
at org.eclipse.core.commands.ParameterizedCommand.executeWithCh ecks(ParameterizedCommand.java:508)
at org.eclipse.ui.internal.handlers.HandlerService.executeComma nd(HandlerService.java:169)
at org.eclipse.ui.internal.handlers.SlaveHandlerService.execute Command(SlaveHandlerService.java:241)
at org.eclipse.ui.menus.CommandContributionItem.handleWidgetSel ection(CommandContributionItem.java:770)
at org.eclipse.ui.menus.CommandContributionItem.access$10(Comma ndContributionItem.java:756)
at org.eclipse.ui.menus.CommandContributionItem$5.handleEvent(C ommandContributionItem.java:746)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java :84)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1176)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.ja va:3493)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java :3112)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.jav a:2405)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2369)
at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:22 21)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:500)
at org.eclipse.core.databinding.observable.Realm.runWithDefault (Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Work bench.java:493)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.j ava:149)
at testapp.Application.start(Application.java:20)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(Eclips eAppHandle.java:194)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher .runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher .start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseS tarter.java:368)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseS tarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce ssorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe thodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java: 559)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:514)
at org.eclipse.equinox.launcher.Main.run(Main.java:1311)
at org.eclipse.equinox.launcher.Main.main(Main.java:1287)
Caused by: org.eclipse.emf.cdo.common.util.TransportException: java.util.concurrent.TimeoutException: Timeout
at org.eclipse.emf.internal.cdo.net4j.protocol.CDOClientProtoco l.send(CDOClientProtocol.java:310)
at org.eclipse.emf.internal.cdo.net4j.protocol.CDOClientProtoco l.commitTransaction(CDOClientProtocol.java:220)
at org.eclipse.emf.internal.cdo.transaction.CDOSingleTransactio nStrategyImpl.commit(CDOSingleTransactionStrategyImpl.java:5 8)
at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl. commit(CDOTransactionImpl.java:598)
... 35 more
Caused by: java.util.concurrent.TimeoutException: Timeout
at org.eclipse.net4j.signal.Signal.runSync(Signal.java:243)
at org.eclipse.net4j.signal.SignalProtocol.startSignal(SignalPr otocol.java:423)
at org.eclipse.net4j.signal.RequestWithConfirmation.doSend(Requ estWithConfirmation.java:87)
at org.eclipse.net4j.signal.RequestWithConfirmation.send(Reques tWithConfirmation.java:73)
at org.eclipse.net4j.signal.RequestWithMonitoring.send(RequestW ithMonitoring.java:108)
at org.eclipse.emf.internal.cdo.net4j.protocol.CDOClientProtoco l.send(CDOClientProtocol.java:302)
... 38 more
[ERROR] [Ljava.lang.StackTraceElement;
java.lang.ClassNotFoundException: [Ljava.lang.StackTraceElement;
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInter nal(BundleLoader.java:494)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(Bund leLoader.java:410)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(Bund leLoader.java:398)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loa dClass(DefaultClassLoader.java:105)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at org.eclipse.net4j.util.io.ExtendedIOUtil$ClassLoaderClassRes olver.resolveClass(ExtendedIOUtil.java:231)
at org.eclipse.net4j.util.io.ExtendedIOUtil$3.resolveClass(Exte ndedIOUtil.java:163)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream .java:1575)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.ja va:1496)
at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1 624)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java :1323)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStrea m.java:1945)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.j ava:1869)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStre am.java:1753)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java :1329)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java: 351)
at org.eclipse.net4j.util.io.ExtendedIOUtil.readObject(Extended IOUtil.java:177)
at org.eclipse.net4j.util.io.ExtendedIOUtil.readObject(Extended IOUtil.java:131)
at org.eclipse.net4j.util.io.ExtendedDataInputStream.readObject (ExtendedDataInputStream.java:46)
at org.eclipse.net4j.signal.RemoteExceptionIndication.indicatin g(RemoteExceptionIndication.java:46)
at org.eclipse.net4j.signal.Indication.doExtendedInput(Indicati on.java:55)
at org.eclipse.net4j.signal.Signal.doInput(Signal.java:312)
at org.eclipse.net4j.signal.Indication.execute(Indication.java: 49)
at org.eclipse.net4j.signal.Signal.runSync(Signal.java:239)
at org.eclipse.net4j.signal.Signal.run(Signal.java:147)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Threa dPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo lExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
!ENTRY org.eclipse.net4j.util 4 0 2009-11-26 02:35:22.578
!MESSAGE [Ljava.lang.StackTraceElement;
!STACK 0
java.lang.ClassNotFoundException: [Ljava.lang.StackTraceElement;
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInter nal(BundleLoader.java:494)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(Bund leLoader.java:410)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(Bund leLoader.java:398)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loa dClass(DefaultClassLoader.java:105)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at org.eclipse.net4j.util.io.ExtendedIOUtil$ClassLoaderClassRes olver.resolveClass(ExtendedIOUtil.java:231)
at org.eclipse.net4j.util.io.ExtendedIOUtil$3.resolveClass(Exte ndedIOUtil.java:163)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream .java:1575)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.ja va:1496)
at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1 624)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java :1323)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStrea m.java:1945)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.j ava:1869)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStre am.java:1753)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java :1329)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java: 351)
at org.eclipse.net4j.util.io.ExtendedIOUtil.readObject(Extended IOUtil.java:177)
at org.eclipse.net4j.util.io.ExtendedIOUtil.readObject(Extended IOUtil.java:131)
at org.eclipse.net4j.util.io.ExtendedDataInputStream.readObject (ExtendedDataInputStream.java:46)
at org.eclipse.net4j.signal.RemoteExceptionIndication.indicatin g(RemoteExceptionIndication.java:46)
at org.eclipse.net4j.signal.Indication.doExtendedInput(Indicati on.java:55)
at org.eclipse.net4j.signal.Signal.doInput(Signal.java:312)
at org.eclipse.net4j.signal.Indication.execute(Indication.java: 49)
at org.eclipse.net4j.signal.Signal.runSync(Signal.java:239)
at org.eclipse.net4j.signal.Signal.run(Signal.java:147)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Threa dPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo lExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
--nextPart21350895.kcvHbq28VH
Content-Type: text/plain; name="cdo-server.xml"
Content-Transfer-Encoding: 8Bit
Content-Disposition: attachment; filename="cdo-server.xml"
<?xml version="1.0" encoding="UTF-8"?>
<cdoServer>
<acceptor type="tcp" listenAddr="0.0.0.0" port="2036">
<!--
<negotiator type="challenge" description="/temp/users.db"/>
-->
</acceptor>
<repository name="testapp">
<property name="overrideUUID" value="1ff5d226-b1f0-40fb-aba2-0c31b38c764f"/>
<property name="supportingAudits" value="true"/>
<property name="supportingRevisionDeltas" value="true"/>
<property name="verifyingRevisions" value="false"/>
<!--
<property name="currentLRUCapacity" value="10000"/>
<property name="revisedLRUCapacity" value="100"/>
-->
<property name="currentLRUCapacity" value="0"/>
<property name="revisedLRUCapacity" value="0"/>
<store type="db">
<!-- type: horizontal | vertical | <any user-contributed type>-->
<mappingStrategy type="horizontal">
<!-- ONE_TABLE_PER_REFERENCE | ONE_TABLE_PER_CLASS | ONE_TABLE_PER_PACKAGE | ONE_TABLE_PER_REPOSITORY | LIKE_ATTRIBUTES -->
<property name="toManyReferences" value="ONE_TABLE_PER_REFERENCE"/>
<!-- LIKE_ATTRIBUTES | LIKE_TO_MANY_REFERENCES-->
<property name="toOneReferences" value="LIKE_ATTRIBUTES"/>
<!-- MODEL | STRATEGY-->
<property name="mappingPrecedence" value="MODEL"/>
</mappingStrategy>
<dbAdapter name="mysql"/>
<dataSource class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource"
url="jdbc:mysql://localhost/testapp"
user="cdo"
password="cdo"/>
<!--<dbAdapter name="derby-embedded"/>
<dataSource class="org.apache.derby.jdbc.EmbeddedDataSource"
databaseName="/temp/cdodb1"
createDatabase="create"/>-->
<!--<dbAdapter name="hsqldb"/>
<dataSource class="org.eclipse.net4j.db.hsqldb.HSQLDBDataSource"
database="jdbc:hsqldb:mem:cdodb1"
user="sa"/>-->
<!--<dbAdapter name="mysql"/>
<dataSource class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource"
url="jdbc:mysql://localhost/cdodb1"
user="sa"/>-->
<!--<dbAdapter name="postgresql"/>
<dataSource class="org.postgresql.ds.PGSimpleDataSource"
url="jdbc:postgresql://localhost:5432/cdo"
databaseName="cdo"
user="cdo"
password="cdo"/>-->
</store>
</repository>
</cdoServer>
--nextPart21350895.kcvHbq28VH
Content-Type: text/plain; name="server.log"
Content-Transfer-Encoding: 8Bit
Content-Disposition: attachment; filename="server.log"
osgi> [INFO] CDO Server starting
!SESSION 2009-11-26 01:42:10.896 -----------------------------------------------
eclipse.buildId=unknown
java.version=1.6.0_01
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US
Framework arguments: -product org.eclipse.emf.cdo.server.product
Command-line arguments: -product org.eclipse.emf.cdo.server.product -data /workspaces/rap-rcp-common-3.5/cdoserver -dev file:/workspaces/rcp-3.5/.metadata/.plugins/org.eclipse.pde. core/cdoserver/dev.properties -os linux -ws gtk -arch x86_64 -console -consolelog
!ENTRY org.eclipse.emf.cdo.server 1 0 2009-11-26 01:42:11.814
!MESSAGE CDO Server starting
[INFO] First start: Nov 26, 2009 1:42:12 AM
!ENTRY org.eclipse.emf.cdo.server.db 1 0 2009-11-26 01:42:12.295
!MESSAGE First start: Nov 26, 2009 1:42:12 AM
[INFO] No user manager configured. Users will not be authenticated
!ENTRY org.eclipse.emf.cdo.server 1 0 2009-11-26 01:42:12.297
!MESSAGE No user manager configured. Users will not be authenticated
[INFO] CDO Server started
!ENTRY org.eclipse.emf.cdo.server 1 0 2009-11-26 01:42:12.724
!MESSAGE CDO Server started
[ERROR] FATAL: Memory infrastructure corrupted after successful commit operation of the store
[ERROR] Java heap space
java.lang.OutOfMemoryError: Java heap space
at java.io.BufferedWriter.<init>(BufferedWriter.java:87)
at java.io.BufferedWriter.<init>(BufferedWriter.java:70)
at org.eclipse.core.runtime.adaptor.EclipseLog.logForStream(Ecl ipseLog.java:463)
at org.eclipse.core.runtime.adaptor.EclipseLog.openFile(Eclipse Log.java:245)
at org.eclipse.core.runtime.adaptor.EclipseLog.log(EclipseLog.j ava:301)
at org.eclipse.core.internal.runtime.PlatformLogWriter.logging( PlatformLogWriter.java:35)
at org.eclipse.core.internal.runtime.RuntimeLog.log(RuntimeLog. java:95)
at org.eclipse.core.internal.runtime.InternalPlatform.log(Inter nalPlatform.java:711)
at org.eclipse.core.internal.runtime.Log.log(Log.java:56)
at org.eclipse.net4j.util.om.log.EclipseLoggingBridge.logged(Ec lipseLoggingBridge.java:41)
at org.eclipse.net4j.internal.util.bundle.AbstractPlatform.log( AbstractPlatform.java:222)
at org.eclipse.net4j.util.om.log.Logger.log(Logger.java:35)
at org.eclipse.net4j.util.om.log.Logger.log(Logger.java:60)
at org.eclipse.net4j.util.om.log.Logger.error(Logger.java:65)
at org.eclipse.emf.cdo.internal.server.TransactionCommitContext Impl.updateInfraStructure(TransactionCommitContextImpl.java: 569)
at org.eclipse.emf.cdo.internal.server.TransactionCommitContext Impl.commit(TransactionCommitContextImpl.java:287)
at org.eclipse.emf.cdo.internal.server.protocol.CommitTransacti onIndication$2.runLoop(CommitTransactionIndication.java:83)
at org.eclipse.emf.cdo.internal.server.protocol.CommitTransacti onIndication$2.runLoop(CommitTransactionIndication.java:1)
at org.eclipse.net4j.util.om.monitor.ProgressDistributor.run(Pr ogressDistributor.java:96)
at org.eclipse.emf.cdo.internal.server.protocol.CommitTransacti onIndication.indicatingCommit(CommitTransactionIndication.ja va:325)
at org.eclipse.emf.cdo.internal.server.protocol.CommitTransacti onIndication.indicating(CommitTransactionIndication.java:198 )
at org.eclipse.emf.cdo.internal.server.protocol.CommitTransacti onIndication.indicating(CommitTransactionIndication.java:140 )
at org.eclipse.net4j.signal.IndicationWithMonitoring.indicating (IndicationWithMonitoring.java:84)
at org.eclipse.net4j.signal.IndicationWithResponse.doExtendedIn put(IndicationWithResponse.java:90)
at org.eclipse.net4j.signal.Signal.doInput(Signal.java:312)
at org.eclipse.net4j.signal.IndicationWithResponse.execute(Indi cationWithResponse.java:63)
at org.eclipse.net4j.signal.IndicationWithMonitoring.execute(In dicationWithMonitoring.java:63)
at org.eclipse.net4j.signal.Signal.runSync(Signal.java:239)
at org.eclipse.net4j.signal.Signal.run(Signal.java:147)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Threa dPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo lExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
Exception in thread "Thread-10" java.lang.OutOfMemoryError: Java heap space
at java.io.BufferedWriter.<init>(BufferedWriter.java:87)
at java.io.BufferedWriter.<init>(BufferedWriter.java:70)
at org.eclipse.core.runtime.adaptor.EclipseLog.logForStream(Ecl ipseLog.java:463)
at org.eclipse.core.runtime.adaptor.EclipseLog.openFile(Eclipse Log.java:245)
at org.eclipse.core.runtime.adaptor.EclipseLog.log(EclipseLog.j ava:301)
at org.eclipse.core.internal.runtime.PlatformLogWriter.logging( PlatformLogWriter.java:35)
at org.eclipse.core.internal.runtime.RuntimeLog.log(RuntimeLog. java:95)
at org.eclipse.core.internal.runtime.InternalPlatform.log(Inter nalPlatform.java:711)
at org.eclipse.core.internal.runtime.Log.log(Log.java:56)
at org.eclipse.net4j.util.om.log.EclipseLoggingBridge.logged(Ec lipseLoggingBridge.java:41)
at org.eclipse.net4j.internal.util.bundle.AbstractPlatform.log( AbstractPlatform.java:222)
at org.eclipse.net4j.util.om.log.Logger.log(Logger.java:35)
at org.eclipse.net4j.util.om.log.Logger.log(Logger.java:91)
at org.eclipse.net4j.util.om.log.Logger.error(Logger.java:96)
at org.eclipse.emf.cdo.internal.server.TransactionCommitContext Impl.handleException(TransactionCommitContextImpl.java:305)
at org.eclipse.emf.cdo.internal.server.TransactionCommitContext Impl.commit(TransactionCommitContextImpl.java:295)
at org.eclipse.emf.cdo.internal.server.protocol.CommitTransacti onIndication$2.runLoop(CommitTransactionIndication.java:83)
at org.eclipse.emf.cdo.internal.server.protocol.CommitTransacti onIndication$2.runLoop(CommitTransactionIndication.java:1)
at org.eclipse.net4j.util.om.monitor.ProgressDistributor.run(Pr ogressDistributor.java:96)
at org.eclipse.emf.cdo.internal.server.protocol.CommitTransacti onIndication.indicatingCommit(CommitTransactionIndication.ja va:325)
at org.eclipse.emf.cdo.internal.server.protocol.CommitTransacti onIndication.indicating(CommitTransactionIndication.java:198 )
at org.eclipse.emf.cdo.internal.server.protocol.CommitTransacti onIndication.indicating(CommitTransactionIndication.java:140 )
at org.eclipse.net4j.signal.IndicationWithMonitoring.indicating (IndicationWithMonitoring.java:84)
at org.eclipse.net4j.signal.IndicationWithResponse.doExtendedIn put(IndicationWithResponse.java:90)
at org.eclipse.net4j.signal.Signal.doInput(Signal.java:312)
at org.eclipse.net4j.signal.IndicationWithResponse.execute(Indi cationWithResponse.java:63)
at org.eclipse.net4j.signal.IndicationWithMonitoring.execute(In dicationWithMonitoring.java:63)
at org.eclipse.net4j.signal.Signal.runSync(Signal.java:239)
at org.eclipse.net4j.signal.Signal.run(Signal.java:147)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Threa dPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo lExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
--nextPart21350895.kcvHbq28VH--
|
|
|
|
|
Re: [CDO] scalability test problem [message #500582 is a reply to message #500568] |
Thu, 26 November 2009 20:40 |
Lothar Werzinger Messages: 153 Registered: July 2009 Location: Bay Area |
Senior Member |
|
|
Víctor Roldán Betancort wrote:
some comments below:
> Lothar,
>
> some comments below:
>
> This problem is related to a timeout in the signaling protocol due to a
> very long commit operation. If you take a deeper look in the stack
> trace, you see:
>
> org.eclipse.net4j.util.transaction.TransactionException:
> org.eclipse.emf.cdo.common.util.TransportException:
> java.util.concurrent.TimeoutException: Timeout
>
> Yes, I know the EOFException is very misleading. Both I and Stefan went
> through the same problem in the past as well :P There is a bug to
> improve the exception message for it:
>
> 287812: [Net4j] Poor exception/message when a timeout occurs
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=287812
>
> To fix your problem, this should do the trick:
>
> ((org.eclipse.emf.cdo.net4j.CDOSession)session).options()
> .getProtocol().setTimeout(-1L);
>
> There is also the commit timeout value:
>
> ((org.eclipse.emf.cdo.net4j.CDOSession)session).options()
> .setCommitTimeout(Integer.MAX_VALUE);
Thanks, I will try to use these settings ASAP.
>> I would appreciate any help to create a configuration that works with
>> 500000 or more (millions) of objects.
>
> I would recommend you to read this:
>
> http://wiki.eclipse.org/Tweaking_CDO_Performance
>
> If you take a look there, it suggest an even bigger collection loading
> policy. Also I would suggest to increase server cache memory, it
> improves a lot the performance.
Actually I made all these parameters so small to make sure I don't get an
out of memory situation on the client/server. Performance is not what I was
trying to evaluate, but the scalability itself. Thanks for the pointers
anyway :-)
> HTH,
> Víctor.
Thanks!
Lothar
|
|
|
|
Powered by
FUDForum. Page generated in 0.03688 seconds