Eclipse Community Forums: EMF » [CDO] Increasing prefetch performance

Home » Modeling » EMF » [CDO] Increasing prefetch performance

[CDO] Increasing prefetch performance [message #1848138]

Mon, 22 November 2021 08:24

Eclipse User

Let's say we have a very large tree of CDOResourceFolders and CDOResources. I would like to load this tree in a view with as few client<->server roundtrips as possible. I do not want to load the content of the CDOResources (otherwise, we could simply use the regular prefetching mechanism with infinite depth).

Can I achieve this with a CDORevisionPrefetchingPolicy in combination with a CDOCollectionLoadingPolicy? On the other hand, before I investigate more on this: I read here https://www.eclipse.org/forums/index.php?t=msg&th=1108148&goto=1842003&#msg_1842003 that the CDOCollectionLoadingPolicy should be avoided.

What is the best way to implement the prefetching in this case?

Re: [CDO] Increasing prefetch performance [message #1848214 is a reply to message #1848138]

Thu, 25 November 2021 04:30

Eclipse User

Sorry for the late reply. I was too busy preparing RC1.

I think you're probably best off writing a custom IQueryHandler. You can look at OCLQueryHandler for an example.

Note that you can respond CDORevisions back to the client with IQueryContext.addResult(Object). Make sure you properly intern these revision on the client and use the interned revisions, see CDORevisionInterner.internRevision().

Re: [CDO] Increasing prefetch performance [message #1848274 is a reply to message #1848214]

Fri, 26 November 2021 05:22

Eclipse User

Using the IQueryHandler seems very promising!

I am not finding any information on the CDORevisionInterner though. Is it spelled differently, maybe? Also: if we are returning the actual revisions, then it may become a bit complicated regarding resource based security (which we are using).

I tried the following approach: only return the CDOIDs of the objects that are required and then load them in the client via: CDOView.getObjects. But: for me, this method only returns a small fraction of the objects that I am trying to load. I am providing a list of ~800 objects and only get 4 loaded revisions as the return map. Could there be a bug in CDOView.getObjects method? (I am using R20210908-0655)

Edit: my bad! The CDOView.getObjects method does not return the newly loaded objects, but it caches them nevertheless! The performance is really good now!
Edit2: can I also prefetch all lock states? Calling CDOObject.cdoWriteLock().isLocked() still needs a client<->server roundtrip.

[Updated on: Fri, 26 November 2021 08:00] by Moderator

Re: [CDO] Increasing prefetch performance [message #1848375 is a reply to message #1848274]

Tue, 30 November 2021 04:39

Eclipse User

The CDORevisionInterner was only recently added as a replacement of CDORevisionAdder, see commit 694c3178f66a66ae6813e0c57b8214fb033e1177. You'd call:

revision = revisionManager.getCache().internRevision(revision);

Proper permissions are automatically applied by the server while sending them to the client, see BaseCDORevision.write(CDODataOutput, int, CDOBranchPoint) .

Regarding lock state prefetching as part of revision loading, I've just implemented that: https://bugs.eclipse.org/bugs/show_bug.cgi?id=577319 . See also See InternalCDORevisionManager.prefetchRevisions(CDOID, CDOBranchPoint, int, boolean, Consumer<CDORevision>) .

Re: [CDO] Increasing prefetch performance [message #1848510 is a reply to message #1848375]

Mon, 06 December 2021 07:50

Eclipse User

This is great stuff, thanks :). The performance is really good now.
A couple of points though if anybody wants to do the same:
* I needed to use InternalCDORevisionManager#getRevisions instead of the revisionInterner, because this seems to be the only way currently to prefetch the lock states
* I needed to add a List<CDOID> to the IQueryContext#addResult. If I added them without the additional List wrapper, then it seems like the Client initiates a server roundtrip for each CDOID that we iterate over the result.

Previous Topic:	Generate Tests for Non-plugin Projects
Next Topic:	How could I fully get EMF models from an XML resource?

Goto Forum:

-=] Back to Top [=-

Current Time: Sun Jun 22 15:09:18 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter