Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF "Technology" (Ecore Tools, EMFatic, etc)  » CDO performance issues
CDO performance issues [message #32368] Thu, 20 April 2006 21:32 Go to next message
Mark Geib is currently offline Mark GeibFriend
Messages: 432
Registered: July 2009
Senior Member
Eike,

I have just completed the first preliminary tests with the new CDO,
Eclipse 3.2M6, and migrated data.

With the old CDO, Eclipse 3.0.2, my client takes about 30secs to come up.

With the new CDO, Eclipse 3.2M6, my client takes about 5min45sec (345secs)
to come up. With such a decrease in performance I assume that I must have
something mis-tuned or mis-configured. Any ideas...?

During the startup both the server and client machines are using nearly
zero cpu, and the network traffic between them is max of about 8
kbits/sec, usually more like 4 kbits/sec. So the machines are not busy at
all, nor the network.

Mark.
Re: CDO performance issues [message #32681 is a reply to message #32368] Sat, 22 April 2006 12:52 Go to previous messageGo to next message
Eclipse UserFriend
Originally posted by: stepper.sympedia.com

hi mark,

hmm, without seeing it directly i can't imagine what that is ;-(
i tested on several machines (all with windows) and everything went well.
once i tried on a single laptop (mysql, host-eclipse, cdo-server + 2 cdo
clients). well that was slower than on my bigger machine, but really
acceptable nevertheless.

have you ever tried the shipped example?
is it also slow on your machines?

are you able to run with a profiler enabled?
in case you are running on linux, can you test your model on windows, too?

cheers
/eike

p.s. i just moved within switzerland (zurich -> bern where i am working
for some more months). so it might take a while unitl i'm connected again
;-( but on thursday i'll be back in berlin...
Re: CDO performance issues [message #32747 is a reply to message #32681] Wed, 26 April 2006 16:57 Go to previous messageGo to next message
Mark Geib is currently offline Mark GeibFriend
Messages: 432
Registered: July 2009
Senior Member
Eike,

I installed the tptp profiler and ran some tests, all on Linux of course.

On the client side the profiler shows two hot spots, one in
..socket.impl.SelectorManagerImpl$Controller.doWorkStep.

The same hotspot is identified on the server.

I did not include java.nio in the filter so I have no data for the
select...but I assume most of the time is spent there.


On the client there is also a hotspot at
..net4j.util.thread.DeadlockDetector.wait() called by transmit() in
ChannelImpl.

I can not attach files here, but I will email you the high level stats for
both the client and server.

Mark.
Re: CDO performance issues [message #32974 is a reply to message #32747] Sun, 30 April 2006 05:52 Go to previous message
Eclipse UserFriend
Originally posted by: stepper.sympedia.de

Mark,

Sorry again, that I answer lately. You know why...

I've read all your other mails. Thanks for all your effort. You're
really the user who makes open software development fun ;-)
Next I've to say sorry again. That 500millis thing is really incredible.
I told you about the thread sync problem once and during that
investigation I must have inserted this horrible piece of code! I must
have forgotten to remove it after testing.

Unfortunately the SocketTest did not run anymore after removing the
sleep() call ;-(
Btw. you could also wish to run that test case as it gives a quite good
picture of how good the net4j and socket layers work.
It has 2 modes (should be 2 classes, but i was temporarily tired of
refactoring). If you set PERFORMANCE to true, it connects a single
client with one test channel and transmits 1000 echo requests and prints
the measures. With PERFORMANCE == false, it performs a stress test with
4 clients and 4 channels each, sending 10000 echo requests.

I'm quite sure that the problem only occurs if client and server are on
the same (1 cpu) computer. It appears that the client receiver thread is
sometimes faster than the client transmitter thread (the main thread).
That results in getCurrentRequest() to return null. I was able to
eliminate the problem with the following code in processSignal():

if (signalId < 0)
{
// Signal is a Confirmation, use the original Request
RequestWithConfirmation confirmation =
(RequestWithConfirmation)getCurrentRequest();

while (confirmation == null)
{
confirmation = (RequestWithConfirmation)getCurrentRequest();
DeadlockDetector.sleep(5);
}

This way there is no more place in the selector loop code with a wait()
that has a positive minimum expiration time. In other words in the
unexceptional case the loop completes without waiting anywhere. That'
the most important thing at all, because ALL connectors and ALL channels
are triggered by this thread. In the end that's the precondition to be
"non-blocking" ;-)

I hope you can verify these changes and many thanks again!

Cheers
/Eike


Geib schrieb:
> Eike,
>
> I installed the tptp profiler and ran some tests, all on Linux of course.
>
> On the client side the profiler shows two hot spots, one in
> ..socket.impl.SelectorManagerImpl$Controller.doWorkStep.
> The same hotspot is identified on the server.
>
> I did not include java.nio in the filter so I have no data for the
> select...but I assume most of the time is spent there.
>
>
> On the client there is also a hotspot at
> ..net4j.util.thread.DeadlockDetector.wait() called by transmit() in
> ChannelImpl.
>
> I can not attach files here, but I will email you the high level stats
> for both the client and server.
>
> Mark.
>
Re: CDO performance issues [message #576074 is a reply to message #32368] Sat, 22 April 2006 12:52 Go to previous message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6683
Registered: July 2009
Senior Member
hi mark,

hmm, without seeing it directly i can't imagine what that is ;-(
i tested on several machines (all with windows) and everything went well.
once i tried on a single laptop (mysql, host-eclipse, cdo-server + 2 cdo
clients). well that was slower than on my bigger machine, but really
acceptable nevertheless.

have you ever tried the shipped example?
is it also slow on your machines?

are you able to run with a profiler enabled?
in case you are running on linux, can you test your model on windows, too?

cheers
/eike

p.s. i just moved within switzerland (zurich -> bern where i am working
for some more months). so it might take a while unitl i'm connected again
;-( but on thursday i'll be back in berlin...


Re: CDO performance issues [message #576129 is a reply to message #32681] Wed, 26 April 2006 16:57 Go to previous message
Mark Geib is currently offline Mark GeibFriend
Messages: 432
Registered: July 2009
Senior Member
Eike,

I installed the tptp profiler and ran some tests, all on Linux of course.

On the client side the profiler shows two hot spots, one in
..socket.impl.SelectorManagerImpl$Controller.doWorkStep.

The same hotspot is identified on the server.

I did not include java.nio in the filter so I have no data for the
select...but I assume most of the time is spent there.


On the client there is also a hotspot at
..net4j.util.thread.DeadlockDetector.wait() called by transmit() in
ChannelImpl.

I can not attach files here, but I will email you the high level stats for
both the client and server.

Mark.
Re: CDO performance issues [message #576417 is a reply to message #32747] Sun, 30 April 2006 05:52 Go to previous message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6683
Registered: July 2009
Senior Member
Mark,

Sorry again, that I answer lately. You know why...

I've read all your other mails. Thanks for all your effort. You're
really the user who makes open software development fun ;-)
Next I've to say sorry again. That 500millis thing is really incredible.
I told you about the thread sync problem once and during that
investigation I must have inserted this horrible piece of code! I must
have forgotten to remove it after testing.

Unfortunately the SocketTest did not run anymore after removing the
sleep() call ;-(
Btw. you could also wish to run that test case as it gives a quite good
picture of how good the net4j and socket layers work.
It has 2 modes (should be 2 classes, but i was temporarily tired of
refactoring). If you set PERFORMANCE to true, it connects a single
client with one test channel and transmits 1000 echo requests and prints
the measures. With PERFORMANCE == false, it performs a stress test with
4 clients and 4 channels each, sending 10000 echo requests.

I'm quite sure that the problem only occurs if client and server are on
the same (1 cpu) computer. It appears that the client receiver thread is
sometimes faster than the client transmitter thread (the main thread).
That results in getCurrentRequest() to return null. I was able to
eliminate the problem with the following code in processSignal():

if (signalId < 0)
{
// Signal is a Confirmation, use the original Request
RequestWithConfirmation confirmation =
(RequestWithConfirmation)getCurrentRequest();

while (confirmation == null)
{
confirmation = (RequestWithConfirmation)getCurrentRequest();
DeadlockDetector.sleep(5);
}

This way there is no more place in the selector loop code with a wait()
that has a positive minimum expiration time. In other words in the
unexceptional case the loop completes without waiting anywhere. That'
the most important thing at all, because ALL connectors and ALL channels
are triggered by this thread. In the end that's the precondition to be
"non-blocking" ;-)

I hope you can verify these changes and many thanks again!

Cheers
/Eike


Geib schrieb:
> Eike,
>
> I installed the tptp profiler and ran some tests, all on Linux of course.
>
> On the client side the profiler shows two hot spots, one in
> ..socket.impl.SelectorManagerImpl$Controller.doWorkStep.
> The same hotspot is identified on the server.
>
> I did not include java.nio in the filter so I have no data for the
> select...but I assume most of the time is spent there.
>
>
> On the client there is also a hotspot at
> ..net4j.util.thread.DeadlockDetector.wait() called by transmit() in
> ChannelImpl.
>
> I can not attach files here, but I will email you the high level stats
> for both the client and server.
>
> Mark.
>


Previous Topic:[Announce] EMFT JET 1.0.0 I200604271321 is available
Next Topic:small bug in CDO
Goto Forum:
  


Current Time: Sun May 19 05:33:22 GMT 2024

Powered by FUDForum. Page generated in 0.03507 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top