Eclipse Community Forums: EMF » Archive serialization performance

Help

Home

Home » Modeling » EMF » Archive serialization performance

Show: Today's Messages :: Show Polls :: Message Navigator

Archive serialization performance [message #528296]

Tue, 20 April 2010 02:19

Greg Babcock

Messages: 53
Registered: July 2009

Member

My test model is about 1.1Meg in size and consists one main resource (75K)
and 180 resources that each contain an array of 660 doubles. It takes 15 -
35 seconds and 350Meg of disk space to store it as an archive. It takes
350ms and consumes 2Meg of disk space to use the default XMI serialization.
EMF 2.6M6 is slightly faster than 2.5 but not significantly.

I didn't find any optional settings that would improve performance, am I
missing something?
Are there any plans to address the performance issues associated with
serializing to and archive file?

Thank you,

GB

Report message to a moderator

Re: Archive serialization performance [message #528344 is a reply to message #528296]

Tue, 20 April 2010 08:29

Jonas Helming

Messages: 699
Registered: July 2009

Senior Member

Hi,
Maybe you have a look at CDO. Objects can then be garbage collected. The
only preliminary is that you have to regenerate your model, because
afaik the legacy mode of CDO does not support gargabe collections.
Cheers
Jonas
Greg Babcock wrote:
> My test model is about 1.1Meg in size and consists one main resource (75K)
> and 180 resources that each contain an array of 660 doubles. It takes 15 -
> 35 seconds and 350Meg of disk space to store it as an archive. It takes
> 350ms and consumes 2Meg of disk space to use the default XMI serialization.
> EMF 2.6M6 is slightly faster than 2.5 but not significantly.
>
> I didn't find any optional settings that would improve performance, am I
> missing something?
> Are there any plans to address the performance issues associated with
> serializing to and archive file?
>
> Thank you,
>
> GB
>
>

Report message to a moderator

Re: Archive serialization performance [message #528388 is a reply to message #528296]

Tue, 20 April 2010 11:01

Ed Merks

Messages: 33142
Registered: July 2009

Senior Member

Greg,

Comments below.

Greg Babcock wrote:
> My test model is about 1.1Meg in size and consists one main resource (75K)
> and 180 resources that each contain an array of 660 doubles. It takes 15 -
> 35 seconds and 350Meg of disk space to store it as an archive. It takes
> 350ms and consumes 2Meg of disk space to use the default XMI serialization.
> EMF 2.6M6 is slightly faster than 2.5 but not significantly.
>
It's a little bit difficult to imagine a zip file taking more space than
a flat file. Especially 175 times more. What would explain numbers
like that?
> I didn't find any optional settings that would improve performance, am I
> missing something?
> Are there any plans to address the performance issues associated with
> serializing to and archive file?
>
It's the same serialization except that the streams being used are
things like ZipInputStream and we can't control the performance of
that. Note that to write a file to an archive, the entire archive needs
to be read in and written out, along with actually serializing the one
zip entry, so that will take significant time
> Thank you,
>
> GB
>
>
>

Ed Merks
Professional Support: https://www.macromodeling.com/

Report message to a moderator

Re: Archive serialization performance [message #528424 is a reply to message #528388]

Tue, 20 April 2010 13:18

Greg Babcock

Messages: 53
Registered: July 2009

Member

I appologize for the typo, the Zip file consumes 350K, so the size is what I
expected. The problem is the serialization performance. I was able to
resolve the problem by managing the stream myself. Serializing to a Zip
file now takes about the same amount of time as standard serialization.

fileOutputStream= new FileOutputStream("c:/temp/test.zip");
ZipOutputStream out= new ZipOutputStream(fileOutputStream);
for(Resource r:resourceSet.getResources()){
out.putNextEntry(getZipEntry(r));
r.save(out, null);
}

Thanks,

GB

"Ed Merks" <Ed.Merks@gmail.com> wrote in message
news:hqk1ib$8c3$1@build.eclipse.org...
> Greg,
>
> Comments below.
>
>
> Greg Babcock wrote:
>> My test model is about 1.1Meg in size and consists one main resource
>> (75K) and 180 resources that each contain an array of 660 doubles. It
>> takes 15 - 35 seconds and 350Meg of disk space to store it as an archive.
>> It takes 350ms and consumes 2Meg of disk space to use the default XMI
>> serialization. EMF 2.6M6 is slightly faster than 2.5 but not
>> significantly.
>>
> It's a little bit difficult to imagine a zip file taking more space than a
> flat file. Especially 175 times more. What would explain numbers like
> that?
>> I didn't find any optional settings that would improve performance, am I
>> missing something?
>> Are there any plans to address the performance issues associated with
>> serializing to and archive file?
>>
> It's the same serialization except that the streams being used are things
> like ZipInputStream and we can't control the performance of that. Note
> that to write a file to an archive, the entire archive needs to be read in
> and written out, along with actually serializing the one zip entry, so
> that will take significant time
>> Thank you,
>>
>> GB
>>
>>

Report message to a moderator

Previous Topic:	EMFObservables Provisional Status
Next Topic:	option to prohibit mixed contents

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Fri Apr 26 13:00:41 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter