Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [CDO] Commit memory footprint
[CDO] Commit memory footprint [message #1736229] Mon, 27 June 2016 15:24 Go to next message
Philippe Fry is currently offline Philippe FryFriend
Messages: 8
Registered: April 2012
Junior Member
Hello,

We import data from various files to fill a EMF model. During the import we commit the modifications regularly.
At the end of the import we create 3 CDO resources and we commit. The model is then persisted by CDO into a h2 database. The resulting database has a size of 2 GB.

During the import memory consumption goes up to 2 GB.
During the last commit (into the database) memory consumption continues to rise up to 4 GB. This last commit is 10 times longer than the import itself.

We would like to reduce the memory footprint and the time taken by the last commit into the database.

We tried creating the CDO resources before the import and then commiting regularly during the import but we were surprised by the drop in performance: each commit to the DB takes more and more time and we couldn't even reach the end of the import.

Are we missing something ? What is the right way to manage the transactions and commits efficiently with CDO ?

Thanks!
Re: [CDO] Commit memory footprint [message #1736240 is a reply to message #1736229] Mon, 27 June 2016 18:58 Go to previous messageGo to next message
Pavel Vlasov is currently offline Pavel VlasovFriend
Messages: 23
Registered: December 2015
Junior Member
I had a similar poroblem recently. In my case I did not close transactions after commit and on large imports my app wold become gradually slower and slower and then eventually would come to halt.

As soon as the app code was changed to close transactions everything became much faster.
Re: [CDO] Commit memory footprint [message #1737472 is a reply to message #1736240] Fri, 08 July 2016 13:46 Go to previous message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Am 27.06.2016 um 21:58 schrieb Pavel Vlasov:
> I had a similar poroblem recently. In my case I did not close transactions after commit and on large imports my app
> wold become gradually slower and slower and then eventually would come to halt.
> As soon as the app code was changed to close transactions everything became much faster.
Philippe, did Pavel's advice help you?

What mapping strategy are you using?

Note that, when you're using auditing, objects will never be overwritten. Each commit will always create modified full
copies. Especially if you're appending elements to long lists you create many big revisions of the same object. In that
case (and probably ingeneral) it might be better to use a "range-based" mapping strategy, see:

http://help.eclipse.org/neon/topic/org.eclipse.emf.cdo.doc/html/operators/Doc01_ConfiguringRepositories.html#Property_withRanges

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper


Previous Topic:[CDO] Update an historical revision with modification to an up-to-date and commitable revision
Next Topic:[CDO] CDOViews and controlling updates
Goto Forum:
  


Current Time: Thu Mar 28 11:18:00 GMT 2024

Powered by FUDForum. Page generated in 0.05532 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top