Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Web Tools Project (WTP) » StructuredTextEditor consumes huge amount of memory
StructuredTextEditor consumes huge amount of memory [message #204048] Fri, 09 November 2007 16:24 Go to next message
Eclipse User
Originally posted by: ying_sun.yahoo.com

Hi All,

We have a multi-page editor that uses StructuredTextEditor as an
embedded page to display the source. However, we are experiencing a high
memory usage problem when opening a large (>4M) XML file. We tried to
open the same XML file in RAD (7.0.0.5) and noticed that the
StrcturedTextEditor uses close to 80 meg of memory. We understood that
opening such a large file probably requires quite a bit memory but still
feel shocked.

Has anyone experience the similar problem? Any suggestion?

Ying
Re: StructuredTextEditor consumes huge amount of memory [message #204063 is a reply to message #204048] Fri, 09 November 2007 16:43 Go to previous messageGo to next message
Eclipse User
Originally posted by: merks.ca.ibm.com

Ying,

Have you measured how much space is used by just plain old DOM to
represent this file? Keep in mind that the file is read into a text
buffer where each character takes two bytes but on disk it's likely
taking just one byte, so that makes ~4M grow into ~8M, and you've not
build a DOM structure for it yet. DOM elements and attributes take a
lot of space. Even a java.lang.String with a single character takes
roughly 30 bytes to represent, so that's like a 30 times growth (albeit
it at an addition character causes only two bytes of growth). In the
end, it doesn't seem so surprising to me you should see so much growth...


Ying wrote:
> Hi All,
>
> We have a multi-page editor that uses StructuredTextEditor as an
> embedded page to display the source. However, we are experiencing a
> high memory usage problem when opening a large (>4M) XML file. We
> tried to open the same XML file in RAD (7.0.0.5) and noticed that the
> StrcturedTextEditor uses close to 80 meg of memory. We understood that
> opening such a large file probably requires quite a bit memory but
> still feel shocked.
>
> Has anyone experience the similar problem? Any suggestion?
>
> Ying
Re: StructuredTextEditor consumes huge amount of memory [message #204077 is a reply to message #204048] Fri, 09 November 2007 16:49 Go to previous message
Eclipse User
Originally posted by: dcarver.starstandard.org

Ying wrote:
> Hi All,
>
> We have a multi-page editor that uses StructuredTextEditor as an
> embedded page to display the source. However, we are experiencing a high
> memory usage problem when opening a large (>4M) XML file. We tried to
> open the same XML file in RAD (7.0.0.5) and noticed that the
> StrcturedTextEditor uses close to 80 meg of memory. We understood that
> opening such a large file probably requires quite a bit memory but still
> feel shocked.
>
> Has anyone experience the similar problem? Any suggestion?
>

You may want to add your comments to the following bugs:

Bug 136935
Bug 156564
Bug 182552

These are some of the performance and memory issues. The bugs can be
accessed at http://bugs.eclipse.org






> Ying
Re: StructuredTextEditor consumes huge amount of memory [message #204198 is a reply to message #204063] Wed, 07 November 2007 01:50 Go to previous message
David Williams is currently offline David Williams
Messages: 696
Registered: July 2009
Senior Member
On Fri, 09 Nov 2007 11:43:06 -0500, Ed Merks <merks@ca.ibm.com> wrote:

> In the
> end, it doesn't seem so surprising to me you should see so much growth...
>

Ed's right. The StructuredEditor's huge memory use is not so much determined
by number of bytes in the input file, as it is the number of Elements or Nodes.

Years ago, I did some informal "study" and found the limit was about 10000 Elements.
and, still be able to edit the file with reasonable performance.

In other words, if you had a 100 Elements in a megabyte file, you'd be fine, but if
you had 10000 elements in a 100K file you'd still run into trouble.

I'm sure we've improved things since then, but there will always be some limit like this
in a model based editor.

Usually, whenever I see the contents of 1Meg or 4Meg XML like files, it's because they are
in some form other a representation of a database. Perhaps even directly the result of a database
query.

And, as you can imagine, the StructuredTextEditors were not designed with database quantities
in mind ... but rather with users typing in content.

I would be really cool to get some contributions that traded-off the right functions to be
able to do both -- database quantities and user input -- but suspect its a fair amount of work.
Previous Topic:There are no projects that can be added or removed from the server issue
Next Topic:Q about MethodElement.getEnterpriseBean()
Goto Forum:
  


Current Time: Sat Sep 20 18:02:20 GMT 2014

Powered by FUDForum. Page generated in 0.13390 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software