Eclipse Community Forums: Memory Analyzer » Further analysis on memory heap

Help

Home

Home » Eclipse Projects » Memory Analyzer » Further analysis on memory heap

Show: Today's Messages :: Show Polls :: Message Navigator

Further analysis on memory heap [message #1066881]

Fri, 05 July 2013 03:43

ss ss

Messages: 7
Registered: April 2013

Junior Member

We have an output from the memory analyser as below. We want go to further to know the root cause. How to drill on this ?

21 instances of "com.mysql.jdbc.JDBC4Connection", loaded by "sun.misc.Launcher$ExtClassLoader @ 0xf000a4f0" occupy 17,339,632 (80.79%) bytes. 

Biggest instances:

•com.mysql.jdbc.JDBC4Connection @ 0xf016d728 - 1,228,048 (5.72%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf0094478 - 1,144,056 (5.33%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf0201ba0 - 1,126,120 (5.25%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf04ab3f0 - 1,074,552 (5.01%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf006ee40 - 993,912 (4.63%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf0076a90 - 931,512 (4.34%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf015fcf0 - 930,952 (4.34%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf04ad490 - 918,176 (4.28%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf00bea28 - 874,512 (4.07%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf0467bc0 - 846,368 (3.94%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf00b7bd0 - 838,448 (3.91%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf021c830 - 799,184 (3.72%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf0097818 - 791,256 (3.69%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf01a27c0 - 763,264 (3.56%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf01c2d80 - 745,088 (3.47%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf012b158 - 710,704 (3.31%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf04693e0 - 704,072 (3.28%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf147f438 - 680,400 (3.17%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf1603618 - 458,472 (2.14%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf148fd70 - 390,984 (1.82%) bytes. 
•com.mysql.jdbc.JDBC4Connection @ 0xf15ff7d0 - 389,552 (1.82%) bytes. 


Keywords
com.mysql.jdbc.JDBC4Connection
sun.misc.Launcher$ExtClassLoader @ 0xf000a4f0
 Reference Pattern
Class Name Shallow Heap Retained Heap Percentage 
class java.lang.Thread @ 0xf0003840 40 208 0.00%

Report message to a moderator

Re: Further analysis on memory heap [message #1067925 is a reply to message #1066881]

Thu, 11 July 2013 09:20

Andrew Johnson

Messages: 205
Registered: July 2009

Senior Member

Use Path To GC roots on each of the objects in turn (or Merge Shortest Paths to GC Roots on a set of objects) to find out why they are retained.

Your total used heap size of 17,339,632 / (80.79%) = 21,462,596 bytes is very small by modern standards so are you sure you have a memory leak? What is the -mx value for the VM?

Report message to a moderator

Re: Further analysis on memory heap [message #1067927 is a reply to message #1067925]

Thu, 11 July 2013 09:36

ss ss

Messages: 7
Registered: April 2013

Junior Member

Dear Andrew,
Ok I went into dominator_tree and just picked one of the objects that is com.mysql.jdbc.JDBC4Connection then I ran both the Path to Gc root and Merge shortest path to GC roots below are both results. Iam not good in analysing it can you see what could be the issue here?

Path to Gc root

Class Name | Shallow Heap | Retained Heap
------------------------------------------------------------------------------------------------------------------------------------------------
| |
com.mysql.jdbc.JDBC4Connection @ 0xf016d728 | 1,120 | 2,020,312
|- connection com.jolbox.bonecp.ConnectionHandle @ 0xf01c2bf0 | 112 | 264
| '- dbconn commServer9000$ConnectionHandler @ 0xfe48d788 | 400 | 400
| '- <Java Local>, target java.lang.Thread @ 0xfe48d720 Thread-73818 Thread | 104 | 55,216
|- key java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf016d708 | 32 | 72
| '- next java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf016d6c0 | 32 | 144
| '- [2] java.util.concurrent.ConcurrentHashMap$HashEntry[4] @ 0xf016d6a0 | 32 | 176
| '- table java.util.concurrent.ConcurrentHashMap$Segment @ 0xf016d658 | 40 | 248
| '- [9] java.util.concurrent.ConcurrentHashMap$Segment[16] @ 0xf00769b8 | 80 | 3,080
| '- segments java.util.concurrent.ConcurrentHashMap @ 0xf0076988 | 48 | 3,128
| |- finalizableRefs com.jolbox.bonecp.BoneCP @ 0xf006ecc0 | 128 | 304
| | |- pool com.jolbox.bonecp.ConnectionTesterThread @ 0xf0002778 | 48 | 48
| | | |- <Java Local> java.lang.Thread @ 0xf00024c0 BoneCP-keep-alive-scheduler Thread | 104 | 432
| | | |- task java.util.concurrent.Executors$RunnableAdapter @ 0xfe876538 | 24 | 24
| | | | |- <Java Local> java.lang.Thread @ 0xf00024c0 BoneCP-keep-alive-scheduler Thread | 104 | 432
| | | | |- callable java.util.concurrent.FutureTask$Sync @ 0xfe876508 | 48 | 48
| | | | '- Total: 2 entries | |
| | | '- Total: 2 entries | |
| | |- pool com.jolbox.bonecp.ConnectionReleaseHelperThread @ 0xf0003590 | 24 | 24
| | | '- <Java Local> java.lang.Thread @ 0xf0003528 BoneCP-release-thread-helper-thread Thread| 104 | 560
| | |- pool com.jolbox.bonecp.ConnectionReleaseHelperThread @ 0xf0003828 | 24 | 24
| | | '- <Java Local> java.lang.Thread @ 0xf00063c0 BoneCP-release-thread-helper-thread Thread| 104 | 560
| | |- pool com.jolbox.bonecp.ConnectionReleaseHelperThread @ 0xf0006108 | 24 | 24
| | | '- <Java Local> java.lang.Thread @ 0xf0006088 BoneCP-release-thread-helper-thread Thread| 104 | 592
| | |- pool com.jolbox.bonecp.PoolWatchThread @ 0xf000a0c8 | 40 | 40
| | |- connectionPool commServer9000 @ 0xf0134b40 | 16 | 16
| | '- Total: 6 entries | |
| |- finalizableRefs com.jolbox.bonecp.ConnectionHandle @ 0xf00be898 | 112 | 264
| | '- dbconn commServer9000$ConnectionHandler @ 0xfe7c9c18 | 400 | 1,656
| | '- <Java Local>, target java.lang.Thread @ 0xfe7c9bb0 Thread-73821 Thread | 104 | 57,016
| |- finalizableRefs com.jolbox.bonecp.ConnectionHandle @ 0xf015fc80 | 112 | 264
| | '- dbconn commServer9000$ConnectionHandler @ 0xfe5f8238 | 400 | 1,736
| | '- <Java Local>, target java.lang.Thread @ 0xfe5f81d0 Thread-73819 Thread | 104 | 57,232
| |- finalizableRefs com.jolbox.bonecp.ConnectionHandle @ 0xf0205560 | 112 | 264
| | '- dbconn commServer9000$ConnectionHandler @ 0xfe5f8680 | 400 | 1,704
| | '- <Java Local>, target java.lang.Thread @ 0xfe5f8618 Thread-73820 Thread | 104 | 57,200
| |- finalizableRefs com.jolbox.bonecp.ConnectionHandle @ 0xf0147b10 | 112 | 264
| | '- dbconn commServer9000$ConnectionHandler @ 0xf0147838 | 400 | 1,928
| | '- this$1 commServer9000$ConnectionHandler$1 @ 0xf01477d8 | 40 | 1,968
| | '- authenticator javax.mail.Session @ 0xf01474d0 | 48 | 6,240
| '- Total: 5 entries | |
'- Total: 2 entries | |
------------------------------------------------------------------------------------------------------------------------------------------------

Merge shortest path to GC roots

Class Name | Ref. Objects | Shallow Heap | Ref. Shallow Heap | Retained Heap
---------------------------------------------------------------------------------------------------------------------------------------
| | | |
java.lang.Thread @ 0xfe48d720 Thread-73818 Thread | 1 | 104 | 1,120 | 55,216
'- <Java Local>, target commServer9000$ConnectionHandler @ 0xfe48d788| 1 | 400 | 1,120 | 400
'- dbconn com.jolbox.bonecp.ConnectionHandle @ 0xf01c2bf0 | 1 | 112 | 1,120 | 264
'- connection com.mysql.jdbc.JDBC4Connection @ 0xf016d728 | 1 | 1,120 | 1,120 | 2,020,312
---------------------------------------------------------------------------------------------------------------------------------------

Report message to a moderator

Re: Further analysis on memory heap [message #1069093 is a reply to message #1067927]

Mon, 15 July 2013 06:42

Krum Tsvetkov

Messages: 165
Registered: July 2009

Senior Member

>> Your total used heap size of 17,339,632 / (80.79%) = 21,462,596 bytes is very small by modern standards so are you sure you have a memory leak? What is the -mx value for the VM?
+1
Try giving more heap to your application.

It is better to execute the merged shortest path on all suspect objects together (in this case on the JDBC4Connection objects). This query will try to find common paths to the objects, which could be of interest.

Report message to a moderator

Re: Further analysis on memory heap [message #1069121 is a reply to message #1069093]

Mon, 15 July 2013 07:50

ss ss

Messages: 7
Registered: April 2013

Junior Member

Dear Krum,
Currently the -mx is set to 256Mb only. I dont get you here well "It is better to execute the merged shortest path on all suspect objects together (in this case on the JDBC4Connection objects). " What I did I selected all the JDBC4Connection objects and then I ran the merge shortest path. Below is the results any comments please?

Quote:

Class Name | Ref. Objects | Shallow Heap | Ref. Shallow Heap | Retained Heap
------------------------------------------------------------------------------------------------------------------------------------------------------------
| | | |
java.lang.Thread @ 0xf00024c0 BoneCP-keep-alive-scheduler Thread | 18 | 104 | 20,160 | 432
'- <Java Local> com.jolbox.bonecp.ConnectionTesterThread @ 0xf0002778 | 18 | 48 | 20,160 | 48
|- pool com.jolbox.bonecp.BoneCP @ 0xf006ecc0 | 16 | 128 | 17,920 | 304
| '- finalizableRefs java.util.concurrent.ConcurrentHashMap @ 0xf0076988 | 16 | 48 | 17,920 | 3,536
| '- segments java.util.concurrent.ConcurrentHashMap$Segment[16] @ 0xf00769b8 | 16 | 80 | 17,920 | 3,488
| |- [11] java.util.concurrent.ConcurrentHashMap$Segment @ 0xf00be9a0 | 2 | 40 | 2,240 | 320
| | '- table java.util.concurrent.ConcurrentHashMap$HashEntry[4] @ 0xf00be9e8 | 2 | 32 | 2,240 | 248
| | '- [1] java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf16035f8 | 2 | 32 | 2,240 | 144
| | |- key com.mysql.jdbc.JDBC4Connection @ 0xf1603618 | 1 | 1,120 | 1,120 | 1,375,416
| | |- next java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf00bea08| 1 | 32 | 1,120 | 72
| | | '- key com.mysql.jdbc.JDBC4Connection @ 0xf00bea28 | 1 | 1,120 | 1,120 | 1,469,016
| | '- Total: 2 entries | | | |
| |- [7] java.util.concurrent.ConcurrentHashMap$Segment @ 0xf01c2cf8 | 2 | 40 | 2,240 | 408
| | '- table java.util.concurrent.ConcurrentHashMap$HashEntry[8] @ 0xf40d1ad8 | 2 | 48 | 2,240 | 336
| | |- [5] java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf40d1b08 | 1 | 32 | 1,120 | 72
| | | '- key com.mysql.jdbc.JDBC4Connection @ 0xf04693e0 | 1 | 1,120 | 1,120 | 1,223,328
| | |- [1] java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf01c2d60 | 1 | 32 | 1,120 | 72
| | | '- key com.mysql.jdbc.JDBC4Connection @ 0xf01c2d80 | 1 | 1,120 | 1,120 | 1,508,024
| | '- Total: 2 entries | | | |
| |- [2] java.util.concurrent.ConcurrentHashMap$Segment @ 0xf049b808 | 2 | 40 | 2,240 | 248
| | '- table java.util.concurrent.ConcurrentHashMap$HashEntry[4] @ 0xf1602508 | 2 | 32 | 2,240 | 176
| | |- [2] java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf1602528 | 1 | 32 | 1,120 | 72
| | | '- key com.mysql.jdbc.JDBC4Connection @ 0xf15ff7d0 | 1 | 1,120 | 1,120 | 1,164,272
| | |- [1] java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf049b868 | 1 | 32 | 1,120 | 72
| | | '- key com.mysql.jdbc.JDBC4Connection @ 0xf0467bc0 | 1 | 1,120 | 1,120 | 1,691,712
| | '- Total: 2 entries | | | |
| |- [9] java.util.concurrent.ConcurrentHashMap$Segment @ 0xf016d658 | 2 | 40 | 2,240 | 248
| | '- table java.util.concurrent.ConcurrentHashMap$HashEntry[4] @ 0xf016d6a0 | 2 | 32 | 2,240 | 176
| | '- [2] java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf016d6c0 | 2 | 32 | 2,240 | 144
| | |- key com.mysql.jdbc.JDBC4Connection @ 0xf00b7bd0 | 1 | 1,120 | 1,120 | 1,384,112
| | |- next java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf016d708| 1 | 32 | 1,120 | 72
| | | '- key com.mysql.jdbc.JDBC4Connection @ 0xf016d728 | 1 | 1,120 | 1,120 | 2,455,840
| | '- Total: 2 entries | | | |
| |- [14] java.util.concurrent.ConcurrentHashMap$Segment @ 0xf04aa138 | 2 | 40 | 2,240 | 248
| | '- table java.util.concurrent.ConcurrentHashMap$HashEntry[4] @ 0xf27b4ab8 | 2 | 32 | 2,240 | 176
| | |- [0] java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf04aa198 | 1 | 32 | 1,120 | 72
| | | '- key com.mysql.jdbc.JDBC4Connection @ 0xf04ab3f0 | 1 | 1,120 | 1,120 | 1,987,792
| | |- [1] java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf27b4ad8 | 1 | 32 | 1,120 | 72
| | | '- key com.mysql.jdbc.JDBC4Connection @ 0xf27ad758 | 1 | 1,120 | 1,120 | 491,592
| | '- Total: 2 entries | | | |
| |- [10] java.util.concurrent.ConcurrentHashMap$Segment @ 0xf012b0d0 | 2 | 40 | 2,240 | 248
| | '- table java.util.concurrent.ConcurrentHashMap$HashEntry[4] @ 0xf012b118 | 2 | 32 | 2,240 | 176
| | '- [1] java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf012b138 | 2 | 32 | 2,240 | 144
| | |- next java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf016d610| 1 | 32 | 1,120 | 72
| | | '- key com.mysql.jdbc.JDBC4Connection @ 0xf006ee40 | 1 | 1,120 | 1,120 | 1,475,776
| | |- key com.mysql.jdbc.JDBC4Connection @ 0xf012b158 | 1 | 1,120 | 1,120 | 1,238,240
| | '- Total: 2 entries | | | |
| |- [4] java.util.concurrent.ConcurrentHashMap$Segment @ 0xf04ad410 | 1 | 40 | 1,120 | 248
| | '- table java.util.concurrent.ConcurrentHashMap$HashEntry[4] @ 0xf40c9aa0 | 1 | 32 | 1,120 | 176
| | '- [1] java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf40c9ac0 | 1 | 32 | 1,120 | 144
| | '- next java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf04ad470| 1 | 32 | 1,120 | 72
| | '- key com.mysql.jdbc.JDBC4Connection @ 0xf04ad490 | 1 | 1,120 | 1,120 | 2,033,984
| |- [12] java.util.concurrent.ConcurrentHashMap$Segment @ 0xf0097798 | 1 | 40 | 1,120 | 168
| | '- table java.util.concurrent.ConcurrentHashMap$HashEntry[2] @ 0xf00977e0 | 1 | 24 | 1,120 | 96
| | '- [1] java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf00977f8 | 1 | 32 | 1,120 | 72
| | '- key com.mysql.jdbc.JDBC4Connection @ 0xf0097818 | 1 | 1,120 | 1,120 | 1,371,248
| |- [3] java.util.concurrent.ConcurrentHashMap$Segment @ 0xf02428a8 | 1 | 40 | 1,120 | 248
| | '- table java.util.concurrent.ConcurrentHashMap$HashEntry[4] @ 0xf02428f0 | 1 | 32 | 1,120 | 176
| | '- [0] java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf0242910 | 1 | 32 | 1,120 | 144
| | '- next java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf0242958| 1 | 32 | 1,120 | 72
| | '- key com.mysql.jdbc.JDBC4Connection @ 0xf015fcf0 | 1 | 1,120 | 1,120 | 2,141,152
| |- [6] java.util.concurrent.ConcurrentHashMap$Segment @ 0xf021c760 | 1 | 40 | 1,120 | 176
| | '- table java.util.concurrent.ConcurrentHashMap$HashEntry[4] @ 0xf021c7a8 | 1 | 32 | 1,120 | 104
| | '- [0] java.util.concurrent.ConcurrentHashMap$HashEntry @ 0xf021c810 | 1 | 32 | 1,120 | 72
| | '- key com.mysql.jdbc.JDBC4Connection @ 0xf021c830 | 1 | 1,120 | 1,120 | 1,451,408
| '- Total: 10 entries | | | |
|- partition com.jolbox.bonecp.ConnectionPartition @ 0xf006ed58 | 2 | 80 | 2,240 | 1,040
| '- freeConnections com.jolbox.bonecp.BoundedLinkedTransferQueue @ 0xf006eda8 | 2 | 40 | 2,240 | 840
'- Total: 2 entries | | | |
java.lang.Thread @ 0xfea44e28 Thread-101188 Thread | 1 | 104 | 1,120 | 57,240
'- <Java Local>, target commServer9000$ConnectionHandler @ 0xfea44e90 | 1 | 400 | 1,120 | 1,736
'- dbconn com.jolbox.bonecp.ConnectionHandle @ 0xf01a2750 | 1 | 112 | 1,120 | 264
'- connection com.mysql.jdbc.JDBC4Connection @ 0xf01a27c0 | 1 | 1,120 | 1,120 | 1,461,584
java.lang.Thread @ 0xfea45280 Thread-101189 Thread | 1 | 104 | 1,120 | 57,208
'- <Java Local>, target commServer9000$ConnectionHandler @ 0xfea452e8 | 1 | 400 | 1,120 | 1,704
'- dbconn com.jolbox.bonecp.ConnectionHandle @ 0xf0205560 | 1 | 112 | 1,120 | 264
'- connection com.mysql.jdbc.JDBC4Connection @ 0xf0201ba0 | 1 | 1,120 | 1,120 | 1,745,376
java.lang.Thread @ 0xfeb356b8 Thread-101190 Thread | 1 | 104 | 1,120 | 57,232
'- <Java Local>, target commServer9000$ConnectionHandler @ 0xfeb35720 | 1 | 400 | 1,120 | 1,728
'- dbconn com.jolbox.bonecp.ConnectionHandle @ 0xf148fc48 | 1 | 112 | 1,120 | 264
'- connection com.mysql.jdbc.JDBC4Connection @ 0xf147f438 | 1 | 1,120 | 1,120 | 1,375,352
java.lang.Thread @ 0xfe8da988 Thread-101187 Thread | 1 | 104 | 1,120 | 56,600
'- <Java Local>, target commServer9000$ConnectionHandler @ 0xfe8da9f0 | 1 | 400 | 1,120 | 1,232
'- dbconn com.jolbox.bonecp.ConnectionHandle @ 0xf0094408 | 1 | 112 | 1,120 | 264
'- connection com.mysql.jdbc.JDBC4Connection @ 0xf0094478 | 1 | 1,120 | 1,120 | 1,720,104
Total: 5 entries | 22 | 520 | 24,640 |
------------------------------------------------------------------------------------------------------------------------------------------------------------

Report message to a moderator

Re: Further analysis on memory heap [message #1069573 is a reply to message #1069121]

Tue, 16 July 2013 06:48

Krum Tsvetkov

Messages: 165
Registered: July 2009

Senior Member

Hi,

the output means that the common part to 16 of the objects in question is this one:
java.lang.Thread @ 0xf00024c0 BoneCP-keep-alive-scheduler Thread | 18 | 104 | 20,160 | 432
'- <Java Local> com.jolbox.bonecp.ConnectionTesterThread @ 0xf0002778 | 18 | 48 | 20,160 | 48
|- pool com.jolbox.bonecp.BoneCP @ 0xf006ecc0 | 16 | 128 | 17,920 | 304
| '- finalizableRefs java.util.concurrent.ConcurrentHashMap @ 0xf0076988 | 16 | 48 | 17,920 | 3,536

i.e. the connections are kept by bonecp connection pool, which, I think, is exactly what you would like a connection pool to do.
Keep the application running with some with more heap and see if there are further problems.

Further hint - was the app was already running with Xmx 256Mb when the heap dump was triggered? The heap dump shows something like 25Mb used. This would mean that there could be another reason for an OutOfMemoryError (if you had once at all).

Krum

Report message to a moderator

Re: Further analysis on memory heap [message #1069754 is a reply to message #1069573]

Tue, 16 July 2013 13:46

ss ss

Messages: 7
Registered: April 2013

Junior Member

Dear Krum,
I actually got this results from the memory leak result was worries me for instance is that

1st Test I saw this

•com.mysql.jdbc.JDBC4Connection @ 0xf016d728 - 2,455,840 (6.10%) bytes.

2nd Test I Saw this

•com.mysql.jdbc.JDBC4Connection @ 0xf016d728 - 2,473,032 (5.81%) bytes.

If you looks the bytes it show increase in them that worries me but you say that it keep running more heap is that its normal behavior of the pooling? Yes the app is already running with Xmx 256Mb when I take snapshot of the heaps. This heap are not triggered from a outofmemoryerror I take them on different time snap shots. I am just trying to learn and be proactive before anything occur looking out for any possible problems. How do you look that the heap dump used is 25Mb. If more memory use does it indicate any problem? What else can I look from memory analyser to indicate of any problem? Actually what are these path to root suppose to indicate?

Report message to a moderator