Eclipse Community Forums: EMF » [CDO] Increasing prefetch performance

Help

Home

Home » Modeling » EMF » [CDO] Increasing prefetch performance

Show: Today's Messages :: Show Polls :: Message Navigator

[CDO] Increasing prefetch performance [message #1848138]

Mon, 22 November 2021 13:24

Robert Schulk

Messages: 144
Registered: July 2015

Senior Member

Let's say we have a very large tree of CDOResourceFolders and CDOResources. I would like to load this tree in a view with as few client<->server roundtrips as possible. I do not want to load the content of the CDOResources (otherwise, we could simply use the regular prefetching mechanism with infinite depth).

Can I achieve this with a CDORevisionPrefetchingPolicy in combination with a CDOCollectionLoadingPolicy? On the other hand, before I investigate more on this: I read here https://www.eclipse.org/forums/index.php?t=msg&th=1108148&goto=1842003&#msg_1842003 that the CDOCollectionLoadingPolicy should be avoided.

What is the best way to implement the prefetching in this case?

Report message to a moderator

Re: [CDO] Increasing prefetch performance [message #1848214 is a reply to message #1848138]

Thu, 25 November 2021 09:30

Eike Stepper

Messages: 6682
Registered: July 2009

Senior Member

Sorry for the late reply. I was too busy preparing RC1.

I think you're probably best off writing a custom IQueryHandler. You can look at OCLQueryHandler for an example.

Note that you can respond CDORevisions back to the client with IQueryContext.addResult(Object). Make sure you properly intern these revision on the client and use the interned revisions, see CDORevisionInterner.internRevision().

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Report message to a moderator

Re: [CDO] Increasing prefetch performance [message #1848274 is a reply to message #1848214]

Fri, 26 November 2021 10:22

Robert Schulk

Messages: 144
Registered: July 2015

Senior Member

Using the IQueryHandler seems very promising!

I am not finding any information on the CDORevisionInterner though. Is it spelled differently, maybe? Also: if we are returning the actual revisions, then it may become a bit complicated regarding resource based security (which we are using).

I tried the following approach: only return the CDOIDs of the objects that are required and then load them in the client via: CDOView.getObjects. But: for me, this method only returns a small fraction of the objects that I am trying to load. I am providing a list of ~800 objects and only get 4 loaded revisions as the return map. Could there be a bug in CDOView.getObjects method? (I am using R20210908-0655)

Edit: my bad! The CDOView.getObjects method does not return the newly loaded objects, but it caches them nevertheless! The performance is really good now!
Edit2: can I also prefetch all lock states? Calling CDOObject.cdoWriteLock().isLocked() still needs a client<->server roundtrip.

[Updated on: Fri, 26 November 2021 13:00]

Report message to a moderator

Re: [CDO] Increasing prefetch performance [message #1848375 is a reply to message #1848274]

Tue, 30 November 2021 09:39

Eike Stepper

Messages: 6682
Registered: July 2009

Senior Member

The CDORevisionInterner was only recently added as a replacement of CDORevisionAdder, see commit 694c3178f66a66ae6813e0c57b8214fb033e1177. You'd call:

revision = revisionManager.getCache().internRevision(revision);

Proper permissions are automatically applied by the server while sending them to the client, see BaseCDORevision.write(CDODataOutput, int, CDOBranchPoint) .

Regarding lock state prefetching as part of revision loading, I've just implemented that: https://bugs.eclipse.org/bugs/show_bug.cgi?id=577319 . See also See InternalCDORevisionManager.prefetchRevisions(CDOID, CDOBranchPoint, int, boolean, Consumer<CDORevision>) .

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Report message to a moderator

Re: [CDO] Increasing prefetch performance [message #1848510 is a reply to message #1848375]

Mon, 06 December 2021 12:50

Robert Schulk

Messages: 144
Registered: July 2015

Senior Member

This is great stuff, thanks :). The performance is really good now.
A couple of points though if anybody wants to do the same:
* I needed to use InternalCDORevisionManager#getRevisions instead of the revisionInterner, because this seems to be the only way currently to prefetch the lock states
* I needed to add a List<CDOID> to the IQueryContext#addResult. If I added them without the additional List wrapper, then it seems like the Client initiates a server roundtrip for each CDOID that we iterate over the result.

Report message to a moderator

Previous Topic:	Generate Tests for Non-plugin Projects
Next Topic:	How could I fully get EMF models from an XML resource?

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Sat Apr 20 03:02:47 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter