Eclipse Community Forums: Memory Analyzer (MAT) » Extending MAT by providing a new heap dump reader

Home » Archived » Memory Analyzer (MAT) » Extending MAT by providing a new heap dump reader(Outbound and inbound indexes, sizes)

Extending MAT by providing a new heap dump reader [message #516364]

Tue, 23 February 2010 12:45

Eclipse User

Hi,

I'm trying to extend MAT by providing a new heap dump reader.

Can anyone explain to me how the writing of the inbound/outbound indexes works and what the KeyWriterImpl is responsible for?

I'm having trouble at this point because I'm getting a NullPointerException in storeKey in KeyWriterImpl. The KeyWriter tries to access an id that does not belong to a ClassImpl. I assume that I am missing some references in my heap dump and/or using the Memory Analyzer APIs incorrectly but i have no idea what my mistake is.

PS: i'm developing against a 4xx revision of the memory analyzer

[Updated on: Tue, 02 March 2010 09:45] by Moderator

Re: Extending MAT by providing a new heap dump reader [message #516817 is a reply to message #516364]

Thu, 25 February 2010 04:43

Eclipse User

I'm not sure what you have done wrong, but MAT is quite sensitive to having the indexes being consistent.

DTFJIndexBuilder.validateIndices() does some checking for the DTFJ Parser as I found debugging index problems hard. You could try using this as a basis for checking your indexes. Perhaps this should be moved into the core of MAT.

Some things it checks:
1.indexToAddress index is in ascending order without duplicates.
2.all objects have a class index in objectToClass
3.all the class indexes found by objectToClass find a valid ClassImpl from idToClass
4.If the object is an ordinary object, i.e. idToClass == null then arrayToSize >= 0
5.for classes objId -> classImpl -> objectAddress is the same as objId -> address, also classImpl->getObjectId == objId
6.For classImpl getClassId == objId of class -> objectToClass
7.every class has a class loaderId
8.GC roots in range

Also for outbound refs, the first ref must be to the objects's class. This isn't checked, but is assumed by MAT.

Re: Extending MAT by providing a new heap dump reader [message #517247 is a reply to message #516364]

Fri, 26 February 2010 11:39

Eclipse User

Andrew Johnson

Also for outbound refs, the first ref must be to the objects's class. This isn't checked, but is assumed by MAT.

Thanks a lot, that solved my problem.

Re: Extending MAT by providing a new heap dump reader [message #517929 is a reply to message #516364]

Tue, 02 March 2010 09:44

Eclipse User

I have some more questions, this time about the sizes in ClassImpl. Is it correct that the "usedHeapSize" (as used in setUsedHeapSize and getUsedHeapSize) is the number of bytes used by the object representing the class? Also, is it possible to have objects of a class with different instance sizes? The addInstance and removeInstance methods provide parameters for the size but the existence of the heapSizePerInstance variable suggests that every instance of a class must have the same size.

I also want to add some dummy classes for unboxed types. Can I do this by providing a classImp with both usedHeapSize and heapSizePerInstance being zero?

Finally, is the forum a good place to ask these kind of questions or should I ask on the mailing list?

Re: Extending MAT by providing a new heap dump reader [message #518191 is a reply to message #516364]

Wed, 03 March 2010 07:28

Eclipse User

Yes, ClassImpl.usedHeapSize is the number of bytes used by the class object itself.

What this actually means varies with dump type. The DTFJ dump parser includes the size of bytecode and JITted code as well as the java.lang.Class object on the heap. HPROF dumps include a size based on the size of static fields.

http://dev.eclipse.org/blogs/memoryanalyzer/2010/02/25/heap- dump-analysis-with-memory-analyzer-part-2-shallow-size/

Including the byte code and JITted code sizes is a bit odd as those don't actually consume the heap, and so the MAT total heap size could exceed the -Xmx value, but it is also useful to know the big classes for non-heap memory reasons.

heapSizePerInstance is the default size of instances for non-array classes. For array classes this is the size of a pointer. The array pointer size used to be used to calculate the array size for ObjectArrayImpl/PrimitiveArrayImpl, but that is something that only the parser knows about, so now MAT always uses the array size table.

Normally all simple objects are of the same size. I've recently added a feature to allow simple objects of the same type but different sizes:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=301228
If a simple object is not of the default instance size for a class then the parser adds an entry for it to the array size map.

Dummy classes with zero class and instance sizes do work. I've used them in the DTFJ parser. Every object in MAT must have a type, so you still need to call addInstance(0) on the dummy class's type.

Running the DTFJ parser with -Dmat.methods_as_classes=true creates some dummy classes, in a completely different hierarchy from Java.

Java:
instance java.lang.Object, type class java.lang.Object
instance java.lang.String, type class java.lang.String
class java.lang.Object - type class java.lang.Class, superclass null
class java.lang.String - type class java.lang.Class, superclass class java.lang.Object

native memory/stacks/methods:
class <native memory>, type class <native memory type>, superclass null
class <native memory type>, type class <native memory type>, superclass class <native memory>

so
class <native memory> is like class java.lang.Object
class <native memory type> is like class java.lang.Class

I think here is a good place for discussions about extending MAT. You are an 'adopter', and it is good to see adopters as well as 'users'.

http://www.eclipse.org/projects/dev_process/development_proc ess.php#2_3_Three_Communities

Re: Extending MAT by providing a new heap dump reader [message #519316 is a reply to message #516364]

Mon, 08 March 2010 08:54

Eclipse User

Hello Erik,

I couldn't resist to ask you: for what kind of heap dump format you are trying to extend MAT? We haven't had many adopters so far, and I'm really curious to learn more. Of course, only if you can talk about this and if you feel like sharing the info.

Regards,
Krum

Re: Extending MAT by providing a new heap dump reader [message #519537 is a reply to message #519316]

Tue, 09 March 2010 04:28

Eclipse User

Krum Tsvetkov

I couldn't resist to ask you: for what kind of heap dump format you are trying to extend MAT?

I do not want to raise expectations, because I'm a student and my time for the implementation is limited.

I'm trying to extend MAT for use with the metacircular Jikes Research Virtual Machine ( http://www.jikesrvm.org/ ). I'm doing this for a practical course at my university.

The Jikes RVM does not contain any way to do a heap dump at the moment so I am also trying to add that feature. This includes the definition of a new heap dump format. I use a text-based format defined by myself. I'm trying to dump only the information that is absolutely necessary.

I would use a binary format but that would require more complex changes and I'm already having enough trouble with the Jikes RVM. The dump feature is not working correctly at the moment, so I do not know how much progress I will be able to make with extending MAT. I'm not even done with the HeapObjectReader.

Re: Extending MAT by providing a new heap dump reader [message #520999 is a reply to message #519537]

Tue, 16 March 2010 03:13

Eclipse User

Thanks a lot for answering my question! It is very interesting to see people trying to adopt MAT in different areas.
Have fun with your projects and if you have some further questions about the reader I hope we can answer them.

Previous Topic:	class loader held only by finalzation queue ??
Next Topic:	download MAT for RAP

Goto Forum:

-=] Back to Top [=-

Current Time: Wed May 07 08:41:57 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter