[cdt-dev] Re: OutOfMemory debugging in Eclipse

["Dmitry Smirnov" <divis1969@xxxxxxxxx>]

Hi,

with the help of http://www.eclipse.org/mat/ I had detected that the
problem is caused by the huge size of
org.eclipse.cdt.internal.core.pdom.WritablePDOM object.
Unfortunately, the hprof file, generated by JVM is very big (about 540
MB; JVM was configured to run application with 512 MB). Even ZIPped it
is about 150 MB.
If I increase the size (I've tried 768 and 1024 MB), the problem persists.
So, I'm suspecting memory leak and need you advice how to find the root cause.

Let me describe the algorithm I'm using to parse the code.
First, I'm creating a project and configure its source and include directories.
Second, I start the indexer and wait for it to complete. Indexer
creates the .pdom file which is about 60 MB.
Third, for each indexed file, I'm creating an ASTTranslationUnit and
use it to find functions, macro, references to other functions from
the function body (with the help of Visior pattern), references to
function from the other files.
The information is stored in org.w3c.dom.Document and it does not
contain refernces to index, AST or similar objects. Only strings
(function, macro names), numbers (source line).

Indexer had processed about 2400 files. Depending on the order (I can
reaarange this order before processing files in my application) and
the size of heap, the OutOfMemoryError can happen after parsing from
200 to 500 files.

What could be the reason for such growth?

BTW, I've noticed that the getIndex() of my ASTTranslationUnit returns
null. Is it correct? Can I instruct AST to use PDOM generated
previously?
Here is how I create the unit:
	private IASTTranslationUnit getASTTranslationUnit(IIndexFile indexFile) {
		CDOM cdom = CCorePlugin.getDefault().getDOM();
		IWorkspace workspace = ResourcesPlugin.getWorkspace();
		IPath ipath = null;
		IASTTranslationUnit itu = null;
		try {
			if( indexFile.getLocation().getFullPath() != null )
			{
				ipath = new Path(indexFile.getLocation().getFullPath());
				IFile file = workspace.getRoot().getFile(ipath);
				itu = cdom.getTranslationUnit(file, false);
			}
		} catch (CoreException e) {
			e.printStackTrace();
		} catch (UnsupportedDialectException e) {
			e.printStackTrace();
		}
		catch (StackOverflowError e)
		{
			e.printStackTrace();
		}
		catch (OutOfMemoryError e)
		{
			e.printStackTrace();
		}

		return itu;
	}



2008/10/8 Dmitry Smirnov <divis1969@xxxxxxxxx>:
> Hi,
>
> I'm using AST to parse files of the big project. Once parsed, some
> info is extracted from translation unit and stored in
> org.w3c.dom.Document.
> When I first encountered the OutOfMemoryError, I suspected this
> Document to become very large. So, I've changed the way I use it and
> discard all the additons once they are not needed.
>
> But the OutOfMemoryError still persist! Perhaps I'm not releasing
> something, but I cannot figure out that...
>
> So, I would like to ask you, is there any way to debug this problem?
> Can I know what is the memory status (i.e. how much is occupied, how
> much is total)? I'm running my application as an application, not a
> product, so I suppose the numbers in status line of Eclipse IDE is the
> memory heap of this IDE process).
> Can I find somehow any "huge" objects in heap?
>
> P.S. Obviously, this maillist is not for this question, but maybe you can help.
>
> Dmitry
>