Eclipse Community Forums: Memory Analyzer

Help

Home

Home » Eclipse Projects » Memory Analyzer » Requested features

Show: Today's Messages :: Show Polls :: Message Navigator

Requested features [message #636034]

Fri, 29 October 2010 06:35

No real name

Messages: 4
Registered: October 2010

Junior Member

Thanks for a wonderful program! I hope I can contribute some day.

Especially the leak-suspect analysis is very good and often finds the most important leaks.

But very very small leaks is that grows big over time/usages in a server environment is hard to find.. (the leaks is small compared to static memory that should be in the heap)

I have a suggestion to a tool that could be very good to pinpoint the small leaks (and maybe built into MAT):

1. Take two dump-files and create a new file with only the difference but in the normal dump-format. Dump B Dump A => Dump C
2. Run your the leak-suspect analysis on the differences (Dump C)

Do anybody know about a tool that can create a dumpfile with only the differens between two dumps?

Why not make a analyze-function that can take an arbitary number of dumps and make an analyze of that and find the leaks. Its hard work to analyze it manually.

Report message to a moderator

Re: Requested features [message #636112 is a reply to message #636034]

Fri, 29 October 2010 11:02

Krum Tsvetkov

Messages: 164
Registered: July 2009

Senior Member

Hi,

Thanks for the nice feedback!

I wanted to start with a comment on your statement about contributing to the project. We, the developers in these project are very open for any kind of contributions (at least I think so, and I tried to go for such openness).
And here by contributions I don't mean just code, but all kind of feedback, critique, ideas, bug reports and bug patches, and so on.
When it comes to contributing code, I know that our current API documentation is no as good as it should be. But we are trying to improve in the area - we are adding some Wiki pages (still under construction) which should serve like a developer guide to writing extensions to MAT. I'll post the links once they are in a better shape.

Now about your concrete suggestion - the problem with heap dumps is that we in general cannot recognize one and the same object in two dumps. The objects do not have stable IDs, they are identified by the addresses, which change when a GC occurs. So what we can compare are some aggregations (say on class level).
I have to look into more details into your idea and see if this can fit somehow. If you think it is useful, despite the limitation I just mentioned - we'll discuss it.

So far for now from me. I'll spend some more time looking at this and comment again.

Regards,
Krum

Report message to a moderator

Re: Requested features [message #637344 is a reply to message #636112]

Fri, 05 November 2010 07:32

No real name

Messages: 4
Registered: October 2010

Junior Member

Yeah I read another post about the ID issue. I thought the ID was the same during the whole lifetime of the object, to bad it is'nt like that.

I thougt about it little more about it anyway and say that we can find some good method anyway to find the differenses, we still need not just the differenses we also need to keep all the new objects "parents" up to the gc-root. Otherwise the differentdump will be to hard to follow. But these objects that is the same between two dumps should be colored or something when you look at the result.

I will take a look at the details in the dumps and try to find some kind of idea of an analyze that tries to find what objects that are the same without any reliable ID.

Report message to a moderator

Re: Requested features [message #637347 is a reply to message #637344]

Fri, 05 November 2010 07:49

No real name

Messages: 4
Registered: October 2010

Junior Member

Is this maybe something that I'm looking for:

From http://perfinsp.sourceforge.net/hdump.html

"Delta HDUMP

Delta HDUMP is available when using JPROF with JVMTI. Each log-hd.n file contains information about the highest object tag in use at the time of each preceding heap dump. Thus, we can easily tell which objects were created between heapdumps. By using the "-diff m n" parameter, the user can select which pair of heap dumps to compare. The result of this comparison is the b3rd file.
The format of the b3rd file is the same as the b1st file. However, the roots used in the b3rd file are those found using the b2nd algorithm with all of the objects created between heapdumps m and n. Furthermore, only those objects referenced in the b2nd search that determined those roots are included in the b3rd output. The result is a greatly reduced subset of the b1st file that includes only those references that are keeping these new objects alive."

Report message to a moderator

Re: Requested features [message #637352 is a reply to message #637347]

Fri, 05 November 2010 08:06

No real name

Messages: 4
Registered: October 2010

Junior Member

Ok it needed a profiler... and of course its better. But I have a production environment at a client that can't accept that, but multiple heapdumps over time is not a problem....

Some other more fuzzy ideas of memory-changes over time.

1. Take for example 20 heapdumps
2. Make a "color-map" or some other visualization over the whole heap so you can "look at it" exactly what to color and how I dont know without doing some experiments...
3. Animate this visualization over time (some kind of slider-control)

Could this maybe let you "see" what parts of the heap that seems to grow. You should be able to click on the part that grows to get some kind of list of the objects....

i.e. the visualization lets your own brain do a more fuzzy analyze... it should be better at that than comparing a huge amount of figures...

Report message to a moderator

Re: Requested features [message #640554 is a reply to message #637352]

Mon, 22 November 2010 13:47

Krum Tsvetkov

Messages: 164
Registered: July 2009

Senior Member

Hi,

I am also pretty sure that one can develop interesting analysis features based on comparison and analysis of a series of heap dumps. This topic is something on our TODO list, but it is slipping already several times.

The idea with the coloring I found interesting. I never thought about it. There the crucial part will be to find the important "things" to display/color, as the full dump could be huge - I've seen some with 200 million objects and about 3x references. May be a starting poit to play with could be the query "Leak Identification" -> "Big drops in dominator tree". This is something I've done a long time ago, and so far haven't seen many people using it (useless or too difficult to interpret??). The query lists the places in the dominator tree, where the parent is much bigger than any of it's children, i.e. kind of accumulation points where a set of small objects make up a big one. I haven't tried it, but I guess that observing how the number children of such places varies with the time could bring some interesting results.

Well, this is still as I said in the beginning an area where we need to spend time. Thank you for sharing your ideas with us! Any further comments are welcome.

Krum

Report message to a moderator

Re: Requested features [message #641245 is a reply to message #636034]

Wed, 24 November 2010 16:01

Andrew Johnson

Messages: 205
Registered: July 2009

Senior Member

This paper looks interesting - I'll read through to see if we could use any ideas.

Diagnosing Memory Leaks using Graph Mining on Heap Dumps by Evan K. Maxwell, Godmar Back, Naren Ramakrishnan
Department of Computer Science
Virginia Tech

We demonstrate several synthetic as well as real-world examples of heap dumps for which our approach provides more
insight into the problem than state-of-the-art tools such as Eclipse's MAT.

Here is a video presentation too.
Diagnosing Memory Leaks using Graph Mining on Heap Dumps

Report message to a moderator

Previous Topic:	Post-mortem analysis
Next Topic:	Untraceable GC path?

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Fri Apr 19 16:03:29 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter