Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF "Technology" (Ecore Tools, EMFatic, etc)  » [TEXO] Performance issue during Serializing Objects into XMI using Texo's ModelXMLSaver([TEXO] Performance issue during Serializing Objects into XMI using Texo's ModelXMLSaver)
[TEXO] Performance issue during Serializing Objects into XMI using Texo's ModelXMLSaver [message #1422792] Sat, 13 September 2014 11:34 Go to next message
Sunny T is currently offline Sunny TFriend
Messages: 9
Registered: June 2013
Junior Member
Hi Martin,

I am using Texo's ModelXMLSaver class to serialize Objects to XMI. I have around 90 entity classes in my project. And the relationship between these entities is pretty complex and all of the references are bi-directional (have their EOpposite set). There are approximately 50K records in total for all these entities in DB. These 50K records are fetched entity by entity and added to a List. This list of objects is then passed to the ModelXMLSaver and the write method is executed. To fetch rows of all entities the DB calls take only 5 minutes but to serialize this into XMI, the API is taking 96 minutes. Is there way to improve this performance? Does the order in which the entities are added to the List matter?

[Updated on: Sat, 13 September 2014 11:34]

Report message to a moderator

Re: [TEXO] Performance issue during Serializing Objects into XMI using Texo's ModelXMLSaver [message #1422864 is a reply to message #1422792] Sat, 13 September 2014 14:16 Go to previous messageGo to next message
Sunny T is currently offline Sunny TFriend
Messages: 9
Registered: June 2013
Junior Member
index.php/fa/19115/0/

I ran the visualvm profiler while the Serialization was in progress. Attaching the screenshot.

[Updated on: Sat, 13 September 2014 19:46]

Report message to a moderator

Re: [TEXO] Performance issue during Serializing Objects into XMI using Texo's ModelXMLSaver [message #1423352 is a reply to message #1422864] Sun, 14 September 2014 09:01 Go to previous messageGo to next message
Martin Taal is currently offline Martin TaalFriend
Messages: 5468
Registered: July 2009
Senior Member
Hi Sunny,
To be sure, you didn't see additional sql queries during the xmi serialization. Afaycs the code spends all its time in
the serialization code and nowhere else?

Are you objects special in the sense that they are huge, have large number of string eattributes or other specialities
(you already noted the eopposite ereferences)?

gr. Martin

On 13-09-14 16:16, Sunny T wrote:
> I ran the visualvm profiler while the Serialization was in progress. Attaching the screenshot.
>


--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@xxxxxxxx - mtaal@xxxxxxxx
Web: www.springsite.com - www.elver.org
Re: [TEXO] Performance issue during Serializing Objects into XMI using Texo's ModelXMLSaver [message #1423359 is a reply to message #1423352] Sun, 14 September 2014 09:19 Go to previous messageGo to next message
Sunny T is currently offline Sunny TFriend
Messages: 9
Registered: June 2013
Junior Member
Hi Martin,

There are no additional sql queries that are being fired during the serialization. Yes, most of the time is spent in the serialize call. The strings are regular eattributes and nothing special about them and most of them are just name and description attributes.

Also, I am using StringWriter as the writer, will this have any impact?

Regards,
Sunny
Re: [TEXO] Performance issue during Serializing Objects into XMI using Texo's ModelXMLSaver [message #1423368 is a reply to message #1423359] Sun, 14 September 2014 09:38 Go to previous messageGo to next message
Martin Taal is currently offline Martin TaalFriend
Messages: 5468
Registered: July 2009
Senior Member
Hi Sunny,
Also replying to your email here, if a lot of time is spend in the createTarget method of the ModelEMFConverter. You can
replace the ModelEMFConverter with your own class (extending ModelEMFConverter) and then override the createTarget
method to print some information, or check the call count of the method.

https://wiki.eclipse.org/Texo/Components

Can you let me know what you see? Is createTarget called maybe a 'zillion' times?

gr. Martin

On 14-09-14 11:19, Sunny T wrote:
> Hi Martin,
>
> There are no additional sql queries that are being fired during the serialization. Yes, most of the time is spent in the
> serialize call. The strings are regular eattributes and nothing special about them and most of them are just name and
> description attributes.
>
> Also, I am using StringWriter as the writer, will this have any impact?
>
> Regards,
> Sunny
>


--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@xxxxxxxx - mtaal@xxxxxxxx
Web: www.springsite.com - www.elver.org
Re: [TEXO] Performance issue during Serializing Objects into XMI using Texo's ModelXMLSaver [message #1423404 is a reply to message #1423368] Sun, 14 September 2014 10:59 Go to previous messageGo to next message
Sunny T is currently offline Sunny TFriend
Messages: 9
Registered: June 2013
Junior Member
Hi Martin,

I extended the ModelEMFConverter and implemented by own class as you suggested. And yes, the createTarget() method was called "3.2 million" times in 15 minutes and it was still going on.

Regards,
Sunny

[Updated on: Sun, 14 September 2014 10:59]

Report message to a moderator

Re: [TEXO] Performance issue during Serializing Objects into XMI using Texo's ModelXMLSaver [message #1423846 is a reply to message #1422864] Mon, 15 September 2014 04:31 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 33107
Registered: July 2009
Senior Member
Sunny,

It's usually best to do a screen shot with something that captures that
display or window image rather than a photo of the monitor; I can't read
the contents. But I gather the specific issue relates to the "EMF
Reference Cleaner" thread. Whether such a thread is create is
determined as follows:

// A daemon thread is created only if there is no security manager.
// The system property
"org.eclipse.emf.common.util.ReferenceClearingQueue" can be used to
override the default.
//
String hasReferenceClearingQueue =
System.getProperty("org.eclipse.emf.common.util.ReferenceClearingQueue");
if (System.getSecurityManager() == null ?
!"false".equalsIgnoreCase(hasReferenceClearingQueue) :
"true".equalsIgnoreCase(hasReferenceClearingQueue))
{

So you can certainly turn it off if it causes a problems, but note that
this thread does work (cleaning garbage from pools such as the URI pool,
the String pool, and the SegmentSequence pool) on a background/daemon
thread that will otherwise be done on your main thread, so it's highly
unlikely that this thread is actually slowing down the overall progress
of your application because it works in parallel---most modern computers
have multiple cores---to what your application is actually
accomplishing, i.e., cleaning up garbage that the application is
apparently creating at a very high rate.

On 13/09/2014 4:16 PM, Sunny T wrote:
> I ran the visualvm profiler while the Serialization was in progress. Attaching the screenshot.


Ed Merks
Professional Support: https://www.macromodeling.com/
Re: [TEXO] Performance issue during Serializing Objects into XMI using Texo's ModelXMLSaver [message #1423904 is a reply to message #1423404] Mon, 15 September 2014 06:43 Go to previous message
Martin Taal is currently offline Martin TaalFriend
Messages: 5468
Registered: July 2009
Senior Member
Hi Sunny,
How many ereferences does each object have? And can you check the size of the objectmap hashmap (see the method
getObjectMapping)? It should not be larger than the number of objects being converted.

gr. Martin

On 14-09-14 12:59, Sunny T wrote:
> Hi Martin,
>
> I extend the ModelEMFConverter and implemented by own as you suggested. And yes, the createTarget() method was called
> "3.2 million" times in 15 minutes and it was still going on.
>
> Regards,
> Sunny
>


--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@xxxxxxxx - mtaal@xxxxxxxx
Web: www.springsite.com - www.elver.org
Previous Topic:[ECP] Problems in migrating target platform from ECP v. 1.2 to v. 1.4
Next Topic:[EMF Forms] How to add actions to a form
Goto Forum:
  


Current Time: Tue Mar 19 04:34:36 GMT 2024

Powered by FUDForum. Page generated in 0.02536 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top