Eclipse Community Forums: EMF

Help

Home

Home » Modeling » EMF » Very Big Model(PB Save Model)

Show: Today's Messages :: Show Polls :: Message Navigator

Very Big Model [message #523508]

Fri, 26 March 2010 16:14

Benoit Ferrero

Messages: 1
Registered: March 2010

Junior Member

We are a Very Big Instance of Model .

When we save a model , We are a Memory Error

the file is incomplete and have a size of 130MByte.

It's possible to split a Model in many file ?
It's possible to have a part of Model in Memory and the other on the disk ?

Thanks for any help Smile

Benoit

PS Example of Error Stack :

java.lang.OutOfMemoryError: Java heap space

at java.util.Arrays.copyOf(Unknown Source)
at java.io.ByteArrayOutputStream.write(Unknown Source)
at org.eclipse.emf.ecore.xmi.impl.StringSegment.writeAscii(Stri ngSegment.java:333)
at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.writeAscii(XMLSav eImpl.java:997)
at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.save(XMLSaveImpl. java:260)
at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doSave(XMLRes ourceImpl.java:205)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour ceImpl.java:1344)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour ceImpl.java:986)
at fr.inria.aoste.ccslmodel.debug.model.MySimulationEngine.writ eTrace(MySimulationEngine.java:819)
.....

Report message to a moderator

Re: Very Big Model [message #523571 is a reply to message #523508]

Fri, 26 March 2010 16:54

Ed Merks

Messages: 33140
Registered: July 2009

Senior Member

This is a multi-part message in MIME format.
--------------030309040002070004040205
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Benoit,

Probably this option will help:

/**
* Write the intermediate results of serialization to the file
system rather to an in-memory buffer.
* This helps to reduce memory requirements for serializing a
large file, but it's slower.
* The default is <code>Boolean.FALSE</code> unless set to
<code>Boolean.TRUE</code> explicitly.
*/
String OPTION_USE_FILE_BUFFER = "USE_FILE_BUFFER";

Benoit Ferrero wrote:
> We are a Very Big Instance of Model .
>
>
> When we save a model , We are a Memory Error
>
> the file is incomplete and have a size of 130MByte.
> It's possible to split a Model in many file ?
> It's possible to have a part of Model in Memory and the other on the
> disk ?
>
>
>
>
> Thanks for any help :)
> Benoit
>
> PS Example of Error Stack :
>
> java.lang.OutOfMemoryError: Java heap space
>
> at java.util.Arrays.copyOf(Unknown Source)
> at java.io.ByteArrayOutputStream.write(Unknown Source)
> at org.eclipse.emf.ecore.xmi.impl.StringSegment.writeAscii(Stri
> ngSegment.java:333)
> at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.writeAscii(XMLSav
> eImpl.java:997)
> at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.save(XMLSaveImpl.
> java:260)
> at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doSave(XMLRes
> ourceImpl.java:205)
> at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
> ceImpl.java:1344)
> at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
> ceImpl.java:986)
> at fr.inria.aoste.ccslmodel.debug.model.MySimulationEngine.writ
> eTrace(MySimulationEngine.java:819)
> ....

--------------030309040002070004040205
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Benoit, 
 
Probably this option will help: 
<blockquote> /** 
 * Write the intermediate results of serialization to the
file system rather to an in-memory buffer. 
 * This helps to reduce memory requirements for serializing
a large file, but it's slower. 
 * The default is <code>Boolean.FALSE</code>
unless set to <code>Boolean.TRUE</code> explicitly. 
 */ 
 String OPTION_USE_FILE_BUFFER = "USE_FILE_BUFFER"; 
</blockquote>
 
 
Benoit Ferrero wrote:
<blockquote cite="mid:hoimi1$gm2$1@build.eclipse.org" type="cite">We
are a Very Big Instance of Model .
 
 
 
When we save a model , We are a Memory Error
 
 
the file is incomplete and have a size of 130MByte. 
It's possible to split a Model in many file ?
 
It's possible to have a part of Model in Memory and the other on the
disk ?
 
 
 
 
 
Thanks for any help :)
 
Benoit
 
 
PS Example of Error Stack :
 
 
java.lang.OutOfMemoryError: Java heap space
 
 
 at java.util.Arrays.copyOf(Unknown Source)
 
 at java.io.ByteArrayOutputStream.write(Unknown Source)
 
 at org.eclipse.emf.ecore.xmi.impl.StringSegment.writeAscii(Stri
ngSegment.java:333)
 
 at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.writeAscii(XMLSav
eImpl.java:997)
 
 at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.save(XMLSaveImpl.
java:260)
 
 at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doSave(XMLRes
ourceImpl.java:205)
 
 at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
ceImpl.java:1344)
 
 at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
ceImpl.java:986)
 
 at fr.inria.aoste.ccslmodel.debug.model.MySimulationEngine.writ
eTrace(MySimulationEngine.java:819)
 
.....
 
</blockquote>
</body>
</html>

--------------030309040002070004040205--

Ed Merks
Professional Support: https://www.macromodeling.com/

Report message to a moderator

Re: Very Big Model [message #523680 is a reply to message #523508]

Sun, 28 March 2010 10:34

Eike Stepper

Messages: 6682
Registered: July 2009

Senior Member

Hi Benoit,

In case you don't care so much about the format of the instance storage
you could try to use CDO. Then you are completely independent of the
size as CDO can lazily load and *unload* single objects. Let me know if
that's something you're interested in.

Cheers
/Eike

----
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Am 26.03.2010 17:14, schrieb Benoit Ferrero:
> We are a Very Big Instance of Model .
>
>
> When we save a model , We are a Memory Error
>
> the file is incomplete and have a size of 130MByte.
> It's possible to split a Model in many file ?
> It's possible to have a part of Model in Memory and the other on the
> disk ?
>
>
>
>
> Thanks for any help :)
> Benoit
>
> PS Example of Error Stack :
>
> java.lang.OutOfMemoryError: Java heap space
>
> at java.util.Arrays.copyOf(Unknown Source)
> at java.io.ByteArrayOutputStream.write(Unknown Source)
> at org.eclipse.emf.ecore.xmi.impl.StringSegment.writeAscii(Stri
> ngSegment.java:333)
> at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.writeAscii(XMLSav
> eImpl.java:997)
> at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.save(XMLSaveImpl.
> java:260)
> at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doSave(XMLRes
> ourceImpl.java:205)
> at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
> ceImpl.java:1344)
> at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
> ceImpl.java:986)
> at fr.inria.aoste.ccslmodel.debug.model.MySimulationEngine.writ
> eTrace(MySimulationEngine.java:819)
> .....

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Report message to a moderator

Re: Very Big Model [message #524014 is a reply to message #523508]

Tue, 30 March 2010 05:32

Stefan Winkler

Messages: 307
Registered: July 2009
Location: Germany

Senior Member

Hi,

you can have your model distributed over several files by using
non-containment relationships for your EReferences.
This way you can have subtrees stored in different resources (they must
be added to separate resources using
resource_x.getConteint().add(subtreeNode)).
If you load the "root" object tree, then the referenced subtrees from
the separate files remain transparent proxies and only when you access
them, they are loaded.
The drawback is, however, they are not automatically unloaded, you have
to unload them manually yourself (using resource.unload()).

As Eike said, you could use CDO instead. CDO stores the object tree in a
backend (e.g., a database) and manages the proxies automatically, so
only the objects you currently use are contained in your memory and if
you don't use them anymore, they are unloaded and transparently proxied
again.

Cheers,
Stefan

Am 26.03.2010 17:14, schrieb Benoit Ferrero:
> We are a Very Big Instance of Model .
>
>
> When we save a model , We are a Memory Error
>
> the file is incomplete and have a size of 130MByte.
> It's possible to split a Model in many file ?
> It's possible to have a part of Model in Memory and the other on the
> disk ?
>
>
>
>
> Thanks for any help :)
> Benoit
>
> PS Example of Error Stack :
>
> java.lang.OutOfMemoryError: Java heap space
>
> at java.util.Arrays.copyOf(Unknown Source)
> at java.io.ByteArrayOutputStream.write(Unknown Source)
> at org.eclipse.emf.ecore.xmi.impl.StringSegment.writeAscii(Stri
> ngSegment.java:333)
> at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.writeAscii(XMLSav
> eImpl.java:997)
> at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.save(XMLSaveImpl.
> java:260)
> at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doSave(XMLRes
> ourceImpl.java:205)
> at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
> ceImpl.java:1344)
> at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
> ceImpl.java:986)
> at fr.inria.aoste.ccslmodel.debug.model.MySimulationEngine.writ
> eTrace(MySimulationEngine.java:819)
> ....

Report message to a moderator

Previous Topic:	[CDO dadabinding] possible to observe eContainer?
Next Topic:	Turning EMF into XML/XMI without using a resource

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Tue Apr 23 12:21:13 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter