Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » Compare » Scalability of EMF Compare (Scalability of EMF Compare for large size models)
Scalability of EMF Compare [message #1018855] Thu, 14 March 2013 15:16 Go to next message
Asha Mising name is currently offline Asha Mising nameFriend
Messages: 18
Registered: July 2009
Junior Member
Hi,

I have around 18 MB file of model. I have tried increasing memory using -Xms1024m -Xmx1024m options while starting eclipse.

But I am observing slowdown in performance and heap error continuously.

What is the typical size of model with which EMFCompare is recommended to use?

Thanks in advance.

Regards,
Asha
Re: Scalability of EMF Compare [message #1020503 is a reply to message #1018855] Mon, 18 March 2013 10:21 Go to previous messageGo to next message
Laurent Goubet is currently offline Laurent GoubetFriend
Messages: 1639
Registered: July 2009
Senior Member
Asha,

If you fragmented your model correctly, EMF Compare can be used on extremely large models, you can find some of the numbers on the wiki, and some more were presented at Eclipse con Europe (and will be presented at Eclipse con US next week). The slides can be found on slideshare, the numbers starting from page 44.

Basically, we've used EMF Compare 2.1 on files "weighing" nearly 1 GB of XML on disk, containing more than a million elements... with only 512 of -Xmx.

Such figures require you to have properly fragmented your models. If you have a single XML file of 1GB on disk, EMF Compare will still need to load that one file in memory, twice (or thrice if comparing with an ancestor). If you have fragmented that file in 4 fragments of 250MB each, EMF Compare will first determine which fragments have really changed : if all four have changed, we'll need to load the four in memory twice (or three times), which results in no gain. However, if only one of these fragments has changed, we only need to load those 250MB of data, which requires roughly four times less memory.

This, of course, assume you are using EMF Compare 2.1. EMF Compare 1.3 was more memory-hungry and did not differentiate between fragmented or monolithic xml files. A typical use of EMF Compare will not require settings different than "-Xms256m -Xmx512m". Then, you will increase the "-Xmx" setting depending on the size of the model you're comparing. If you try and compare a monolithic model that weighs 750MB on disk, it is quite obvious that "-Xmx1024" won't be enough : to compare files, we need them in memory. Comparing two such files together will thus need at least "-Xmx1536m". If you are comparing files that weigh 18MB on disk, and EMF Compare cannot manage with 1024m ... then there is a real issue.

If you are using EMF Compare 1.x, please update to EMF Compare 2.x: M5 can be downloaded from the 2.1 milestones update site, M6 will be available for download some time tomorrow from the same update site.

[edit: if you are using EMF Compare 2.x and it still fails to handle your files, do you think you could provide us a copy?]

Laurent Goubet
Obeo

[Updated on: Mon, 18 March 2013 10:23]

Report message to a moderator

Re: Scalability of EMF Compare [message #1020942 is a reply to message #1020503] Tue, 19 March 2013 08:27 Go to previous messageGo to next message
Asha Mising name is currently offline Asha Mising nameFriend
Messages: 18
Registered: July 2009
Junior Member
Thanks,

I think its 1.2 version bundled with eclipse-modeling-indigo-win32.

From where can I download EMFCompare 2.0 compatible with eclipse-modeling-indigo-win32?

Is there any way of automatically fragmenting existing models? How fragments are defined which can ensure less model dependencies across fragments?

Does EMF 2.X version uses some sort of model change information to identify the
difference in models which can speed up computation of differences?

Regards,
Asha.
Re: Scalability of EMF Compare [message #1020956 is a reply to message #1020942] Tue, 19 March 2013 08:52 Go to previous message
Laurent Goubet is currently offline Laurent GoubetFriend
Messages: 1639
Registered: July 2009
Senior Member
Asha,

I provided you with the update site for the 2.1 milestones. It currently holds version 2.1.0M5, it will hold 2.1.0M6 (with a month worth of improvements) later today. EMF Compare 2 is compatible with Eclipse Galileo, Helios, Indigo, Juno and Kepler (the latter is the release planned for June 2013).

- Open your eclipse
- Click on Help > Install new software...
- In the dialog that pops up, paste the following URL : http://download.eclipse.org/modeling/emf/compare/updates/milestones/2.1
- In the bottom half of the dialog, you will see a category named "EMF Compare". Check the box beside it and hit "Next" then follow the installation instructions.

There are no ways I know of to automatically fragment models. How you fragment your models really depends on the models themselves, but it is usually done with very large models on the containers to "isolate" small isles of elements with few to no dependencies to other parts of the model.

I won't go in details over the various optimizations EMF Compare 2. The wiki page I pointed above roughly explains how we determine what parts of the models do not need to be compared. We use a very basic test : if the two fragments are binary identical, then there is no way anything changed within them... and we thus do not need to load and compare them as EMF models. The comparison algorithms have also been enhanced in version 2.* as compared to version 1.*, which offers a second set of optimizations.

Laurent Goubet
Obeo
Previous Topic:Invalid Comparison result with orphan fields on both sides
Next Topic:Configuring EMF Compare
Goto Forum:
  


Current Time: Mon Dec 22 07:48:47 GMT 2014

Powered by FUDForum. Page generated in 0.08444 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software