Hi,
I have a C/C++ project created out of a huge code base.
In this code base, there are certain files under a directory which are very
huge compared to the normal source files for any regular C project. Their file
size is some where between 19 MB to 44 MB. During the indexing of this
project, I was seeing out of memory exceptions and at this time, the PDOM
created so far is around 986 MB which is well within the limits of max pdom
size.
In order to reproduce the above OOM error, I took one of the 44
MB files and created a project out of it. Surprisingly this file as only 3
variable declarations and one of the variable is an unsigned char array
initialized with nearly 43 MB of data. When I created the above project I
could see the heap size increasing and getting close to the max heap limit
(2048M). Some times it throws the OOM during indexing and sometimes during
opening of this file in the editor and while performing a selection of certain
portions of this opened file.
My environment setup is as
follows:-
===========================
Eclipse - 3.4.1
CDT -
5.0.1
-Xmx2048M
java.version=1.6.0_11
java.vendor=Sun Microsystems
Inc.
BootLoader constants: OS=linux, ARCH=x86, WS=gtk,
NL=en_US
Command-line arguments: -os linux -ws gtk -arch x86
I
searched the bugzilla to see if there was a similar problem reported and found
the below bug close to what I wanted to report. I provided a sample example
appended to the below bug on how to reproduce it.
https://bugs.eclipse.org/bugs/show_bug.cgi?id=236838I
have the following questions with respect to this issue:-
1. Is there any
limit on the file size that the indexer can handle with certain memory
setting? I mean, for example let's say, if 2 GB is the -XmX value for an
Eclipse run, then can we know whether the indexer can handle a file of xMB
without throwing an OOM?
2. Any recommendations on how to handle large
files? other than applying an exclusion filter to remove them from getting
indexed? Even if I have to apply an exclusion filter, can I do that prior to
creating a project out of a already existing code base location?
3. If
these huge need to be part of the index, how can we get them indexed without
running into OOM exceptions?
I tried varying the Indexer cache
settings in the preferences, but it didn't help. Please let me know if you
need more information?
Thanks,
-Sri.
