Eclipse Community Forums: EMF » [CDO] First pass through resource contents slow?

Home » Modeling » EMF » [CDO] First pass through resource contents slow?

[CDO] First pass through resource contents slow? [message #1749264]

Sun, 04 December 2016 18:12

Eclipse User

Hi,

I have a project here that uses CDO (latest 4.5 release) with a MySQL database as storage. All components (client, server, storage) run on the same host. The client does the following actions more or less periodically:
1. Open transaction
2. Find a particular resource (always the same)
3. Pass through the contents of the resource several times (checking various things)
4. Add a new object to its contents
5. Commit

I have noticed that the first pass in a transaction through the contents of a CDOResource is very slow. For a resource with 5000 objects this loop takes nearly 1 second:

for(EObject o : resource.getContents()) {
   ... some simple stuff inside the loop, for example checking the type of each "o"...
}

Subsequent passes (i.e. for-loops) through the contents only take a few milliseconds, as long as they happen in the same transaction.

Is this the expected behavior? I don't see any database read-accesses during the various passes, not even across transactions, which means the data is cached in memory, correct?

As you can probably see, I am completely new to CDO. Please don't hesitate to shout for more information Smile

Thanks,
ru

Re: [CDO] First pass through resource contents slow? [message #1749341 is a reply to message #1749264]

Mon, 05 December 2016 11:55

Eclipse User

During the first iteration the resource content objects are loaded one after the other with one server round-trip per object. Then they're cached on the client side, which speeds up further iterations.

You can optimize the first iteration by prefetching the entire resource contents from the server. You'd use the cdoPrefetch() method on the resource object (or any other CDOObject instance).

Re: [CDO] First pass through resource contents slow? [message #1749388 is a reply to message #1749341]

Tue, 06 December 2016 02:54

Eclipse User

Thank you for your answer. Is the client-side cache cleared after a transaction? Because I see this behavior (i.e. slow first iteration) at the beginning of each transaction. If yes, is there a way to avoid this?

Re: [CDO] First pass through resource contents slow? [message #1749398 is a reply to message #1749388]

Tue, 06 December 2016 04:21

Eclipse User

Ideally you'd not close a transaction after each commit, and of course you'd not close the session after each commit, either. Then the caches stay intact and you'd just call commit() on that single transaction (with its cache) when you need to save your changes atomically. You can compare a CDOTransaction with a JDBC Connection, i.e., you leave it open and call commit() on it multiple times.

Re: [CDO] First pass through resource contents slow? [message #1749399 is a reply to message #1749398]

Tue, 06 December 2016 04:34

Eclipse User

Thanks, very helpful! For me, the session was the equivalent to a JDBC connection and the transaction was just something that you opened in the moment when you needed it.

I was probably confused by the fact that CDOTransaction extends CDOView and the wiki says "Views, in contrast to sessions, are considered light weight entities in CDO, such that you can open and close them as often as you want."

Maybe the wiki should mention the caching issue?

Edit: Actually, in hindsight, I have to say that the wiki is very misleading on this point.

[Updated on: Tue, 06 December 2016 04:38] by Moderator

Re: [CDO] First pass through resource contents slow? [message #1749403 is a reply to message #1749399]

Tue, 06 December 2016 04:59

Eclipse User

The wiki is open source, too. Please feel free to enhance the content wherever you feel comfortable Wink

Re: [CDO] First pass through resource contents slow? [message #1749405 is a reply to message #1749403]

Tue, 06 December 2016 05:05

Eclipse User

> wherever you feel comfortable

Ask me again in one or two years Smile

Re: [CDO] First pass through resource contents slow? [message #1749407 is a reply to message #1749405]

Tue, 06 December 2016 05:17

Eclipse User

Sorry that I am asking again but I want to be sure that I understand things correctly because it might have some important implications on a project of mine:

I have some (remote) front-end clients and a web server that provides various services to the clients. Internally, the web server uses CDO as model repository.

At the moment, the whole system is structured like this:
1. The web server implements a CDO client with a permanent session to the CDO server.
2. The front-end clients send requests to the web server. For each request, the web server opens a CDO transaction, does whatever it has to do to process the client request, commits the changes and closes the transaction.

From your comments, I understand that this approach is really bad for the performance. It would be better if the web server internally kept a pool of transactions for the front-end clients and re-used them.

Correct?

Re: [CDO] First pass through resource contents slow? [message #1749474 is a reply to message #1749407]

Wed, 07 December 2016 02:48

Eclipse User

It's not really bad, but performance depends on so many things. I'll try to explain:

A CDOObject is a thin wrapper that contains only a few internal data values, i.e., CDOID, CDOView, CDOState, and a reference to the "real" data, a so-called CDORevision. The more expensive CDORevision is softly cached in the CDOSession. Recreating the CDOObject is cheap if the needed CDORevision is still cached. Cache-warming can help to speed up first iterations, and extra strong references can help to keep CDORevisions in the cache for future iterations.

It's hard for me to tell where your specific bottleneck really is, but maybe the following articles about CDO internals help you to better understand:

How Scalable are my Models?
Concurrent Access to Models

Previous Topic:	Creating OrderBy annotation for OneToMany
Next Topic:	[CDO] Object changes it's CDOID in an Undo\Redo Scenario using TransactionalEditingDomain

Goto Forum:

-=] Back to Top [=-

Current Time: Mon Jul 07 05:26:53 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter