Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [tracecompass-dev] Fw: Heap storage consumption for traces

Hi Dave,

I see nothing wrong with this heap dump. It only shows a 200MB memory usage, with nothing outstanding and nothing relating to the 2GB memory usage of the callstack analysis you were mentioning. Maybe you can manually acquire a heap dump when memory usage is high, using Memory Analyzer. You could trigger a garbage collection just before to make sure it's actual memory and not lingering unreachable objects.

Cheers,

Geneviève

On 2019-01-17 3:46 p.m., David Wootton wrote:

Genevieve
I uploaded the file again
Download link https://framadrop.org/r/4_o3TANYNE#wA/uZVxcaNjzpwEh6TCLahcs+nzX4kkPGyi+eYkKf7o=
File delete link https://framadrop.org/d/4_o3TANYNE/6KC4F56ytyJPEB63i0aJIP2ioOcelt_M

Dave

Inactive hide details for Genevieve Bastien ---01/17/2019 03:34:02 PM---Hi Dave, Thanks, but the second link was to delete the Genevieve Bastien ---01/17/2019 03:34:02 PM---Hi Dave, Thanks, but the second link was to delete the file... I clicked on it, the file was deleted

From: Genevieve Bastien <gbastien+lttng@xxxxxxxxxxxx>
To: tracecompass-dev@xxxxxxxxxxx
Date: 01/17/2019 03:34 PM
Subject: Re: [tracecompass-dev] Fw: Heap storage consumption for traces
Sent by: tracecompass-dev-bounces@xxxxxxxxxxx




Hi Dave,

Thanks, but the second link was to delete the file... I clicked on it, the file was deleted, I don't have it... Can you upload it again please? Sorry about that.

Geneviève

On 2019-01-17 2:37 p.m., David Wootton wrote:

      Genevieve
      I created a heap dump using memory sampling where I started sampling at the point just before I started loading the trace and stopped sampling after the trace was fully loaded and messages in the Eclipse status bar were no longer changing. I did not click the garbage collect icon in the Eclipse toolbar so there's probably leftover objects in the dump.

      The original trace file was a bit more than 300MB. I compressed it using gzip before uploading.

      These are the links I got from the website       https://framadrop.org/r/G96lAPv4xo#0cxhLW3Q9UQBZAeOQHy6YmwN85GQ4hPROpL1TDqAzlQ= to download and https://framadrop.org/d/G96lAPv4xo/KfH4omGHh8gvDLDhK0PlsutR8dFteLvJ to delete. The website says the file expires on Jan 31.

      Dave


      Inactive hide details for "Geneviève Bastien" ---01/16/2019 03:21:41 PM---Hi Dave, Sorry to hear those improvements didn't impa"Geneviève Bastien" ---01/16/2019 03:21:41 PM---Hi Dave, Sorry to hear those improvements didn't impact the problems you have. If you could share yo

      From:       "Geneviève Bastien" <gbastien@xxxxxxxxxxxx>
      To:       tracecompass-dev@xxxxxxxxxxx
      Date:       01/16/2019 03:21 PM
      Subject:       Re: [tracecompass-dev] Fw: Heap storage consumption for traces
      Sent by:       tracecompass-dev-bounces@xxxxxxxxxxx



      Hi Dave,

      Sorry to hear those improvements didn't impact the problems you have. If you could share your heap dump with us, we could take a look. You can use https://framadrop.org/ to share the file. If it's more than 200Mo, it will only be valid for 24 hours, but we'll read our mails by then and pick it up ;-)

      Cheers,

      Geneviève

      On 2019-01-15 5:12 p.m., David Wootton wrote:

              I built code using these instructions and ran some crude side by side tests using the Eclipse heap storage monitor in the status line. I was puzzled that I didn't see any change in storage utilization (around 2GB for 1.5 million events with about 10 attributes per event) or time (around 1.5 minutes)

              It turns out my analysis modules extend CallStackAnalysis and the changes apply to CallGraphAnalysis where the type hierarchies don't intersect.

              Dave

              Inactive hide details for Matthew Khouzam ---01/09/2019 09:43:01 AM---Hi David, Sorry for the long delay, the easiest way to maMatthew Khouzam ---01/09/2019 09:43:01 AM---Hi David, Sorry for the long delay, the easiest way to make a trace compass baseline is here.

              From:               Matthew Khouzam <matthew.khouzam@xxxxxxxxxxxx>
              To:               tracecompass developer discussions <tracecompass-dev@xxxxxxxxxxx>
              Date:               01/09/2019 09:43 AM
              Subject:               Re: [tracecompass-dev] Fw: Heap storage consumption for traces
              Sent by:               tracecompass-dev-bounces@xxxxxxxxxxx



              Hi David,

              Sorry for the long delay, the easiest way to make a trace compass baseline is here.

              First check out this page.

              https://projects.eclipse.org/projects/tools.tracecompass/developer
              Eclipse Trace Compass | projects.eclipse.org
              projects.eclipse.org
              You can use the code from these repositories to experiment, test, build, create patches, issue pull requests, etc. This project uses Gerrit Code Review; please see contributing via Gerrit.
              Then checkout the patch in your git repo like this.
              $ git fetch               ssh://yourname@xxxxxxxxxxxxxxx:29418/tracecompass/org.eclipse.tracecompass refs/changes/63/134363/4 && git checkout FETCH_HEAD


              then run a maven build (mvn clean install -Pdeploy-update-site) more info is here.
              https://git.eclipse.org/c/tracecompass/org.eclipse.tracecompass.git/tree/README.md

              You should then have a local update site you can use.

              From:               tracecompass-dev-bounces@xxxxxxxxxxx <tracecompass-dev-bounces@xxxxxxxxxxx> on behalf of David Wootton <dwootton@xxxxxxxxxx>
              Sent:               Monday, January 7, 2019 3:22:55 PM
              To:               tracecompass developer discussions
              Subject:               Re: [tracecompass-dev] Fw: Heap storage consumption for traces

              Matthew
              I don't have an easy way to build a workspace with pth Trace Compass and my plugin code. At the moment, trying this is not urgent, so I'll wait until this code gets into a repository I can work with.
              Dave

              Inactive hide details for Matthew Khouzam ---01/07/2019 11:21:17 AM---The patch set is not yet fully merged, you can get a sneaMatthew Khouzam ---01/07/2019 11:21:17 AM---The patch set is not yet fully merged, you can get a sneaky preview here. https://urldefense.proofpo

              From:               Matthew Khouzam <matthew.khouzam@xxxxxxxxxxxx>
              To:               tracecompass developer discussions <tracecompass-dev@xxxxxxxxxxx>
              Date:               01/07/2019 11:21 AM
              Subject:               Re: [tracecompass-dev] Fw: Heap storage consumption for traces
              Sent by:               tracecompass-dev-bounces@xxxxxxxxxxx



              The patch set is not yet fully merged, you can get a sneaky preview here.               https://git.eclipse.org/r/#/c/134363/
              Gerrit Code Review - git.eclipse.org
              git.eclipse.org
              Loading Gerrit Code Review... ... Home; Privacy Policy; Terms of Use; Copyright Agent; Legal

              From:               tracecompass-dev-bounces@xxxxxxxxxxx <tracecompass-dev-bounces@xxxxxxxxxxx> on behalf of David Wootton <dwootton@xxxxxxxxxx>
              Sent:               Monday, January 7, 2019 10:42:48 AM
              To:               tracecompass developer discussions
              Subject:               Re: [tracecompass-dev] Fw: Heap storage consumption for traces

              Matthew.
              That's good news. Will this be in the Simrel 2019-03 release? Will there be a Trace Compass repository site I can point to at some point in time to give this a quick look?
              Thanks to Genevieve for the optimization work.
              Dave

              Inactive hide details for Matthew Khouzam ---01/07/2019 10:20:52 AM---Hey, Dave, Happy Holidays! Hope Santa was good for you, sMatthew Khouzam ---01/07/2019 10:20:52 AM---Hey, Dave, Happy Holidays! Hope Santa was good for you, she was for me, her name is Genevieve and sh

              From:               Matthew Khouzam <matthew.khouzam@xxxxxxxxxxxx>
              To:               tracecompass developer discussions <tracecompass-dev@xxxxxxxxxxx>
              Date:               01/07/2019 10:20 AM
              Subject:               Re: [tracecompass-dev] Fw: Heap storage consumption for traces
              Sent by:               tracecompass-dev-bounces@xxxxxxxxxxx



              Hey, Dave,

              Happy Holidays! Hope Santa was good for you, she was for me, her name is Genevieve and she really optimized the callgraph over the vacations. I think you will be pleasantly surprised with the results. 😊

              BR
              Matthew

              From:               tracecompass-dev-bounces@xxxxxxxxxxx <tracecompass-dev-bounces@xxxxxxxxxxx> on behalf of David Wootton <dwootton@xxxxxxxxxx>
              Sent:               Thursday, January 3, 2019 7:20:16 AM
              To:               tracecompass developer discussions
              Subject:               [tracecompass-dev] Fw: Heap storage consumption for traces

              Matthew
              I hope you had a good holiday. I'm following up on this now that I am back at work. Do you need anything from me at this point, and how can I get it to you?
              Dave
              ----- Forwarded by David Wootton/Poughkeepsie/Contr/IBM on 01/03/2019 07:18 AM -----

              From:               David Wootton/Poughkeepsie/Contr/IBM
              To:               tracecompass developer discussions <tracecompass-dev@xxxxxxxxxxx>
              Date:               12/05/2018 02:28 PM
              Subject:               Re: [tracecompass-dev] Heap storage consumption for traces


              Matthew
              I think I can send you my heap dump, but I need to check. It's about 200MB and I don't have any public site I can upload it to. How do I get it to you?

              I'm not really familiar with VisualVM so I'm not sure how I check what owns an object.

              I am a little surprised that I seem to only have 52 TmfStateValue objects when I have 4000 ITmfEventField objects. Maybe my placement for the printf call is wrong and most of the objects have been released, I placed it in my class overriding the call stack view, in the traceOpened method. But I am using a hash table to keep only a single instance of a value.

              If I understand, in TraceCompass 4.1, at the point where I create a TmfStateValue to wrap an object (String, Long, Integer, etc), I can now just use that object itself in the call to the state builder pushAttribute call. Is that correct?

              Happy holidays

              Dave

              Inactive hide details for Matthew Khouzam ---12/05/2018 10:30:53 AM---Hi Dave, these are some pretty cool findings. Now one thiMatthew Khouzam ---12/05/2018 10:30:53 AM---Hi Dave, these are some pretty cool findings. Now one thing to keep in mind, we have several caches,

              From:               Matthew Khouzam <matthew.khouzam@xxxxxxxxxxxx>
              To:               tracecompass developer discussions <tracecompass-dev@xxxxxxxxxxx>
              Date:               12/05/2018 10:30 AM
              Subject:               Re: [tracecompass-dev] Heap storage consumption for traces
              Sent by:               tracecompass-dev-bounces@xxxxxxxxxxx



              Hi Dave,

              these are some pretty cool findings. Now one thing to keep in mind, we have several caches, so they do trigger visualVM and rightly so. Can you look up who is the owner of the mapEntries? Maybe we have a leak with a manager or something.

              Also, please note, tmfstatevalues are on the road to deprecation. The state system now handles objects, so you can save a few megabytes of useless pointers by just using these objects. 😊

              If you are allowed to share your heapdump, we can look at that and try to optimize a bit here.

              There is finally a possibility, but not a large one. Does your EventField have a backreference to the event, which contains a map of its fields? That would be a prime candidate for where things can go wrong.

              BR,
              Matthew. (Is it too soon to wish a happy holiday season?)

              From:               tracecompass-dev-bounces@xxxxxxxxxxx <tracecompass-dev-bounces@xxxxxxxxxxx> on behalf of David Wootton <dwootton@xxxxxxxxxx>
              Sent:               Wednesday, December 5, 2018 8:58:57 AM
              To:               tracecompass developer discussions
              Subject:               [tracecompass-dev] Heap storage consumption for traces

              A while back, Matthew and I discussed heap storage utilization by traces. My traces have a number of attributes associated with event, with the number of attributes varying by type of trace. I create ITmfEventField and TmfStateValue objects for each attribute.

              I have a trace with about 88,000 events. In that trace, I ended up with close to 800,000 ITmfEventField and 500,000 TmfStateValue objects.

              I created a hash table for ITmfEventField objects that contained a single instance for each unique field name and value pair. In this trace, I ended up with about 4,000 unique ITmfEventField objects.

              I created a hash table for the TmfStateValue objects and end up with 52 unique objects.

              I ran my code with VisialVM, and I noticed that the top class for heap usage was org.eclipse.tracecompass.internal.statesystem.core.backend.historytree.HTInterval. From one run to the next the total number of objects varies somewhere between 1.5 million and 2.5 million. If I click the Perform-GC button in VisualVM, the count drops to around 700,000, fairly consistently.

              I also see a large number of instances of com.google.com.collect.ImmutableMapEntry (355,000), com.google.com.collect.ImmutableMapEntry[] (178,000) classes and com.google.com.collect.ImmutableMapEntry$NonTerminalImmutableMapEntry (355,000)

              Looking at the reference pane in VisualVM for a few (< 10) of each of these classes, it looks like the ImmutableMapEntry and ImmutableMapEntry$nonTerminalImmutableMap objects might be related to StateItem objects and the ImmutableMapEntry[] objects might be related to ITmfEventField objects.

              I'm wondering if it is possible to use a hash table to maintain single copies of unique instances of those objects or whether the data in them guarantees each instance is unique. If it is possible, then is the performance tradeoff for ushing a hash table worth it?

              Dave_______________________________________________
              tracecompass-dev mailing list
              tracecompass-dev@xxxxxxxxxxx
              To change your delivery options, retrieve your password, or unsubscribe from this list, visit
              https://www.eclipse.org/mailman/listinfo/tracecompass-dev



              _______________________________________________
              tracecompass-dev mailing list
              tracecompass-dev@xxxxxxxxxxx
              To change your delivery options, retrieve your password, or unsubscribe from this list, visit
              https://www.eclipse.org/mailman/listinfo/tracecompass-dev
              _______________________________________________
              tracecompass-dev mailing list
              tracecompass-dev@xxxxxxxxxxx
              To change your delivery options, retrieve your password, or unsubscribe from this list, visit
              https://www.eclipse.org/mailman/listinfo/tracecompass-dev


              _______________________________________________
              tracecompass-dev mailing list
              tracecompass-dev@xxxxxxxxxxx
              To change your delivery options, retrieve your password, or unsubscribe from this list, visit
              https://www.eclipse.org/mailman/listinfo/tracecompass-dev


              _______________________________________________
              tracecompass-dev mailing list
              tracecompass-dev@xxxxxxxxxxx
              To change your delivery options, retrieve your password, or unsubscribe from this list, visit
              https://www.eclipse.org/mailman/listinfo/tracecompass-dev
              _______________________________________________
              tracecompass-dev mailing list
              tracecompass-dev@xxxxxxxxxxx
              To change your delivery options, retrieve your password, or unsubscribe from this list, visit
              https://www.eclipse.org/mailman/listinfo/tracecompass-dev



      _______________________________________________
      tracecompass-dev mailing list
      tracecompass-dev@xxxxxxxxxxx
      To change your delivery options, retrieve your password, or unsubscribe from this list, visit
      https://www.eclipse.org/mailman/listinfo/tracecompass-dev
      _______________________________________________
      tracecompass-dev mailing list
      tracecompass-dev@xxxxxxxxxxx
      To change your delivery options, retrieve your password, or unsubscribe from this list, visit
      https://www.eclipse.org/mailman/listinfo/tracecompass-dev




_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/tracecompass-dev

Back to the top