Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [CDO] Increase prepared statement fetch size in DB Store
[CDO] Increase prepared statement fetch size in DB Store [message #1386362] Tue, 17 June 2014 06:03 Go to next message
Leonid Ripeynih is currently offline Leonid RipeynihFriend
Messages: 150
Registered: February 2012
Senior Member
Hello!

I wonder, if there is any way to configure CDO Db Store, to use user-specified fetch size for it's prepares statements?

I've into a major performance problem with OCL, where calling
SomeClass.allInstances() -> select(predicate)

was almost always leading to signal time out. After some profiling i've discovered that the problem lies into org.eclipse.emf.cdo.server.internal.db.mapping.horizontal.HorizontalBranchingClassMapping.handleRevisions(IDBStoreAccessor, CDOBranch, long, boolean, CDORevisionHandler).

ResultSet size there is almost 300,000 elements, which is perfectly fine, since it only contains CDOIDs, but result set iteration is extremely slow, due to a low fetch size.

I've measured the performance by performing the same query, on the same DB with different fetch sizes, and here is the result:

Quering 300,000 CDOIDs:
FetchSize/Time to iterate (ms)
10/38,580
100/2,918
1,000/1,196
5,000/600
10,000/530
20,000/521
100,000/406


I haven't find any way to configure fetch size in CDO Db Store. Is there any?

Thanks in advance!
Re: [CDO] Increase prepared statement fetch size in DB Store [message #1386409 is a reply to message #1386362] Tue, 17 June 2014 11:29 Go to previous messageGo to next message
Christophe Bouhier is currently offline Christophe BouhierFriend
Messages: 937
Registered: July 2009
Senior Member
On 17-06-14 08:03, Leonid Ripeynih wrote:
> Hello!
>
> I wonder, if there is any way to configure CDO Db Store, to use
> user-specified fetch size for it's prepares statements?
Have you checked out this?

http://wiki.eclipse.org/CDO/Tweaking_Performance


>
> I've into a major performance problem with OCL, where calling
> SomeClass.allInstances() -> select(predicate)
>
> was almost always leading to signal time out. After some profiling i've
> discovered that the problem lies into
> org.eclipse.emf.cdo.server.internal.db.mapping.horizontal.HorizontalBranchingClassMapping.handleRevisions(IDBStoreAccessor,
> CDOBranch, long, boolean, CDORevisionHandler).
>
> ResultSet size there is almost 300,000 elements, which is perfectly
> fine, since it only contains CDOIDs, but result set iteration is
> extremely slow, due to a low fetch size.
>
> I've measured the performance by performing the same query, on the same
> DB with different fetch sizes, and here is the result:
>
>
> Quering 300,000 CDOIDs:
> FetchSize/Time to iterate (ms)
> 10/38,580
> 100/2,918
> 1,000/1,196
> 5,000/600
> 10,000/530
> 20,000/521
> 100,000/406
>
>
> I haven't find any way to configure fetch size in CDO Db Store. Is there
> any?
> Thanks in advance!
Re: [CDO] Increase prepared statement fetch size in DB Store [message #1386439 is a reply to message #1386409] Tue, 17 June 2014 14:35 Go to previous messageGo to next message
Leonid Ripeynih is currently offline Leonid RipeynihFriend
Messages: 150
Registered: February 2012
Senior Member
Christophe Bouhier

Have you checked out this?

http://wiki.eclipse.org/CDO/Tweaking_Performance



Sure, but i haven't found any way to set JDBC Statement's fetchSize via cdo-server.xml

Here is a related topic to this problem:
http://www.eclipse.org/forums/index.php/t/781797/

To set fetch size author if this topic was forced to patch CDO as well...
Re: [CDO] Increase prepared statement fetch size in DB Store [message #1386538 is a reply to message #1386439] Wed, 18 June 2014 08:05 Go to previous message
Per Sterner is currently offline Per SternerFriend
Messages: 91
Registered: October 2011
Member
In my case the problem was that everything was fetched. This means, as soon as the iterator started it should run fast.

I don't know how the OCL-statement internally works:
SomeClass.allInstances() -> select(predicate)

But further more the CDOServerExporter uses repository.handleRevisions. This really streams all data from some stored EClasses.

The default procedure for fetching revisions works differently.

[Updated on: Wed, 18 June 2014 08:06]

Report message to a moderator

Previous Topic:[EMF] Consuming org.eclipse.emf.ecore.generated_package
Next Topic:[CDO] Problem loading older revisions of object with 4.3
Goto Forum:
  


Current Time: Sat Aug 17 18:00:21 GMT 2019

Powered by FUDForum. Page generated in 0.01657 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top