| RE: [cdt-dev] Indexing/Opening a file causing out of memory |
Hi,
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Srikanth Ranganamyna
Sent: Thursday, July 09, 2009 4:06 AM
To: CDT General developers list.
Subject: [cdt-dev] Indexing/Opening a file causing out of memory
I have a C/C++ project created out of a huge code base. In this code base, there are certain files under a directory which are very huge compared to the normal source files for any regular C project. Their file size is some where between 19 MB to 44 MB. During the indexing of this project, I was seeing out of memory exceptions and at this time, the PDOM created so far is around 986 MB which is well within the limits of max pdom size.
In order to reproduce the above OOM error, I took one of the 44 MB files and created a project out of it. Surprisingly this file as only 3 variable declarations and one of the variable is an unsigned char array initialized with nearly 43 MB of data. When I created the above project I could see the heap size increasing and getting close to the max heap limit (2048M). Some times it throws the OOM during indexing and sometimes during opening of this file in the editor and while performing a selection of certain portions of this opened file.
My environment setup is as follows:-
===========================
Eclipse - 3.4.1
CDT - 5.0.1
-Xmx2048M
java.version=1.6.0_11
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=linux, ARCH=x86, WS=gtk, NL=en_US
Command-line arguments: -os linux -ws gtk -arch x86
I searched the bugzilla to see if there was a similar problem reported and found the below bug close to what I wanted to report. I provided a sample example appended to the below bug on how to reproduce it.
https://bugs.eclipse.org/bugs/show_bug.cgi?id=236838
I have the following questions with respect to this issue:-
1. Is there any limit on the file size that the indexer can handle with certain memory setting? I mean, for example let's say, if 2 GB is the -XmX value for an Eclipse run, then can we know whether the indexer can handle a file of xMB without throwing an OOM?
2. Any recommendations on how to handle large files? other than applying an exclusion filter to remove them from getting indexed? Even if I have to apply an exclusion filter, can I do that prior to creating a project out of a already existing code base location?
3. If these huge need to be part of the index, how can we get them indexed without running into OOM exceptions?
I tried varying the Indexer cache settings in the preferences, but it didn't help. Please let me know if you need more information?
Thanks,
-Sri.