Eclipse Community Forums: EMF

Home » Modeling » EMF » Very Big Model(PB Save Model)

Fri, 26 March 2010 12:14

Eclipse User

We are a Very Big Instance of Model .

When we save a model , We are a Memory Error

the file is incomplete and have a size of 130MByte.

It's possible to split a Model in many file ?
It's possible to have a part of Model in Memory and the other on the disk ?

Thanks for any help Smile

Benoit

PS Example of Error Stack :

java.lang.OutOfMemoryError: Java heap space

at java.util.Arrays.copyOf(Unknown Source)
at java.io.ByteArrayOutputStream.write(Unknown Source)
at org.eclipse.emf.ecore.xmi.impl.StringSegment.writeAscii(Stri ngSegment.java:333)
at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.writeAscii(XMLSav eImpl.java:997)
at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.save(XMLSaveImpl. java:260)
at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doSave(XMLRes ourceImpl.java:205)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour ceImpl.java:1344)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour ceImpl.java:986)
at fr.inria.aoste.ccslmodel.debug.model.MySimulationEngine.writ eTrace(MySimulationEngine.java:819)
.....

Re: Very Big Model [message #523571 is a reply to message #523508]

Fri, 26 March 2010 12:54

Eclipse User

This is a multi-part message in MIME format.
--------------030309040002070004040205
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Benoit,

Probably this option will help:

/**
* Write the intermediate results of serialization to the file
system rather to an in-memory buffer.
* This helps to reduce memory requirements for serializing a
large file, but it's slower.
* The default is <code>Boolean.FALSE</code> unless set to
<code>Boolean.TRUE</code> explicitly.
*/
String OPTION_USE_FILE_BUFFER = "USE_FILE_BUFFER";

Benoit Ferrero wrote:
> We are a Very Big Instance of Model .
>
>
> When we save a model , We are a Memory Error
>
> the file is incomplete and have a size of 130MByte.
> It's possible to split a Model in many file ?
> It's possible to have a part of Model in Memory and the other on the
> disk ?
>
>
>
>
> Thanks for any help :)
> Benoit
>
> PS Example of Error Stack :
>
> java.lang.OutOfMemoryError: Java heap space
>
> at java.util.Arrays.copyOf(Unknown Source)
> at java.io.ByteArrayOutputStream.write(Unknown Source)
> at org.eclipse.emf.ecore.xmi.impl.StringSegment.writeAscii(Stri
> ngSegment.java:333)
> at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.writeAscii(XMLSav
> eImpl.java:997)
> at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.save(XMLSaveImpl.
> java:260)
> at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doSave(XMLRes
> ourceImpl.java:205)
> at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
> ceImpl.java:1344)
> at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
> ceImpl.java:986)
> at fr.inria.aoste.ccslmodel.debug.model.MySimulationEngine.writ
> eTrace(MySimulationEngine.java:819)
> ....

--------------030309040002070004040205
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Benoit, 
 
Probably this option will help: 
<blockquote> /** 
 * Write the intermediate results of serialization to the
file system rather to an in-memory buffer. 
 * This helps to reduce memory requirements for serializing
a large file, but it's slower. 
 * The default is <code>Boolean.FALSE</code>
unless set to <code>Boolean.TRUE</code> explicitly. 
 */ 
 String OPTION_USE_FILE_BUFFER = "USE_FILE_BUFFER"; 
</blockquote>
 
 
Benoit Ferrero wrote:
<blockquote cite="mid:hoimi1$gm2$1@build.eclipse.org" type="cite">We
are a Very Big Instance of Model .
 
 
 
When we save a model , We are a Memory Error
 
 
the file is incomplete and have a size of 130MByte. 
It's possible to split a Model in many file ?
 
It's possible to have a part of Model in Memory and the other on the
disk ?
 
 
 
 
 
Thanks for any help :)
 
Benoit
 
 
PS Example of Error Stack :
 
 
java.lang.OutOfMemoryError: Java heap space
 
 
 at java.util.Arrays.copyOf(Unknown Source)
 
 at java.io.ByteArrayOutputStream.write(Unknown Source)
 
 at org.eclipse.emf.ecore.xmi.impl.StringSegment.writeAscii(Stri
ngSegment.java:333)
 
 at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.writeAscii(XMLSav
eImpl.java:997)
 
 at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.save(XMLSaveImpl.
java:260)
 
 at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doSave(XMLRes
ourceImpl.java:205)
 
 at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
ceImpl.java:1344)
 
 at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
ceImpl.java:986)
 
 at fr.inria.aoste.ccslmodel.debug.model.MySimulationEngine.writ
eTrace(MySimulationEngine.java:819)
 
.....
 
</blockquote>
</body>
</html>

--------------030309040002070004040205--

Re: Very Big Model [message #523680 is a reply to message #523508]

Sun, 28 March 2010 06:34

Eclipse User

Hi Benoit,

In case you don't care so much about the format of the instance storage
you could try to use CDO. Then you are completely independent of the
size as CDO can lazily load and *unload* single objects. Let me know if
that's something you're interested in.

Cheers
/Eike

----
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Am 26.03.2010 17:14, schrieb Benoit Ferrero:
> We are a Very Big Instance of Model .
>
>
> When we save a model , We are a Memory Error
>
> the file is incomplete and have a size of 130MByte.
> It's possible to split a Model in many file ?
> It's possible to have a part of Model in Memory and the other on the
> disk ?
>
>
>
>
> Thanks for any help :)
> Benoit
>
> PS Example of Error Stack :
>
> java.lang.OutOfMemoryError: Java heap space
>
> at java.util.Arrays.copyOf(Unknown Source)
> at java.io.ByteArrayOutputStream.write(Unknown Source)
> at org.eclipse.emf.ecore.xmi.impl.StringSegment.writeAscii(Stri
> ngSegment.java:333)
> at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.writeAscii(XMLSav
> eImpl.java:997)
> at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.save(XMLSaveImpl.
> java:260)
> at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doSave(XMLRes
> ourceImpl.java:205)
> at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
> ceImpl.java:1344)
> at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
> ceImpl.java:986)
> at fr.inria.aoste.ccslmodel.debug.model.MySimulationEngine.writ
> eTrace(MySimulationEngine.java:819)
> .....

Re: Very Big Model [message #524014 is a reply to message #523508]

Tue, 30 March 2010 01:32

Eclipse User

Hi,

you can have your model distributed over several files by using
non-containment relationships for your EReferences.
This way you can have subtrees stored in different resources (they must
be added to separate resources using
resource_x.getConteint().add(subtreeNode)).
If you load the "root" object tree, then the referenced subtrees from
the separate files remain transparent proxies and only when you access
them, they are loaded.
The drawback is, however, they are not automatically unloaded, you have
to unload them manually yourself (using resource.unload()).

As Eike said, you could use CDO instead. CDO stores the object tree in a
backend (e.g., a database) and manages the proxies automatically, so
only the objects you currently use are contained in your memory and if
you don't use them anymore, they are unloaded and transparently proxied
again.

Cheers,
Stefan

Am 26.03.2010 17:14, schrieb Benoit Ferrero:
> We are a Very Big Instance of Model .
>
>
> When we save a model , We are a Memory Error
>
> the file is incomplete and have a size of 130MByte.
> It's possible to split a Model in many file ?
> It's possible to have a part of Model in Memory and the other on the
> disk ?
>
>
>
>
> Thanks for any help :)
> Benoit
>
> PS Example of Error Stack :
>
> java.lang.OutOfMemoryError: Java heap space
>
> at java.util.Arrays.copyOf(Unknown Source)
> at java.io.ByteArrayOutputStream.write(Unknown Source)
> at org.eclipse.emf.ecore.xmi.impl.StringSegment.writeAscii(Stri
> ngSegment.java:333)
> at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.writeAscii(XMLSav
> eImpl.java:997)
> at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.save(XMLSaveImpl.
> java:260)
> at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doSave(XMLRes
> ourceImpl.java:205)
> at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
> ceImpl.java:1344)
> at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(Resour
> ceImpl.java:986)
> at fr.inria.aoste.ccslmodel.debug.model.MySimulationEngine.writ
> eTrace(MySimulationEngine.java:819)
> ....

Previous Topic:	[CDO dadabinding] possible to observe eContainer?
Next Topic:	Turning EMF into XML/XMI without using a resource

Goto Forum:

-=] Back to Top [=-

Current Time: Mon Jul 14 11:32:28 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter