Re: [tracecompass-dev] Extending filtering for CallStackView

[Patrick Tasse <patrick.tasse@xxxxxxxxx>]

Hi Dave,

I would try it with wrapped traces instead of cloned traces.

If I understand correctly, your trace has two call stack analyses. Are they using different output files for the state system? If they are using the same file it might explain some of the problems.

In your overloaded CallStackView.getTracesToBuild(), create a new list and for each trace returned by the super.getTracesToBuild() (in case of an experiment with multiple traces), if it is one of your traces create two wrappers around the original trace, and add them to the list of traces to build.

The wrappers could just be a subclass of TmfTrace that have the original trace as a private field, and another field to indicate which analysis it should use.

Override getName() to append something (the selected analysis number?) to the original trace name (so you can use that in your filtering content provider).

Override getAnalysisModules() so that it returns only the call stack analysis from the original trace that corresponds to the selected analysis.

Override getFunctionName() so that it returns whatever the original trace returns.

I think you can return anything for all the other abstract methods that need to be implemented, I don't think the call stack view (the only object that has visibility to those wrappers) will call anything else on them.

Patrick

On Mon, Aug 8, 2016 at 5:56 PM, David Wootton <dwootton@xxxxxxxxxx> wrote:

Patrick
It looks like this approach to preserving filtering may not be practical.

Once I fixed the trace name so the IOException and TMFTraceException no longer occurred, I still was not able to get my 2nd analysis data to display.

In trying to debug by stepping thru CallStackView.buildEntryList, it seems that the processing to build the set of TimeGraphEntry objects does not result in any ProcessEntry, ThreadEntry, or lower level objects being created, only the TraceEntry object with no children, which explains the blank view.

There seems to be something wrong with the state system for this analysis resulting in the failure to create Process Entry, ThreadEntry, etc objects. I don't know if there is an assumption that the trace events used as input to the analysis are owned by the same trace object as the one I'm trying to use in my view or if something else is going wrong. I did try to clone a new set of events with ownership set to the new trace but that didn't seem to help.

I also suspect this has something to do with the tree in the left hand side of the CallStackView not populating properly even when the intervals do display (in my first trace/analysis). I haven't tracked down the code which manages the creation of this tree.

Another problem I discovered is that something in the code is not closing the files that contain the state system data in the full history model. This means that even if I reload the trace, I cannot regenerate the analysis data, since the attempt to rewrite these files fails somewhere with a 'files in use IOException'. at least on Windows. I can clean up the analysis data files for the original trace after ensuring the trace editor view for the trace is closed, but I have no editor for the second trace.

If you have suggestions for something else to try, I'll try them.

Otherwise I think I need to accept that being unable to preserve filtering is a limitation in the current release.

Thanks for your help.

Dave
David Wootton---08/04/2016 05:59:33 PM---Patrick I created a scratch workspace this afternoon containing my plugin source +

From: David Wootton/Poughkeepsie/Contr/IBM@IBMUS
To: tracecompass developer discussions <tracecompass-dev@xxxxxxxxxxx>
Date: 08/04/2016 05:59 PM

Subject: Re: [tracecompass-dev] Extending filtering for CallStackView
Sent by: tracecompass-dev-bounces@eclipse.org

---

Patrick
I created a scratch workspace this afternoon containing my plugin source + Trace Compass source. It turns out the reason the analysis modules were not loading was that CallStackView.getCallStackModule() was returning null due to a TmfTraceException that was in turned caused by an IOException. The cause of the IOException was the way I build the alternate name for the cloned trace. I appended ':' and the analysis module ID. Since I'm using a full (disk) state system, the ':' was part of the filename, which is invalid on Windows.

Once I fixed that, I don't have that problem, and the state system history files created on disk are each 69KB, so there is data in them. However, the view does not display any intervals, so I need to do some more debugging.

It also turns out the reason I thought the thread numbers were displaying as ':' is that the Function column was not quite wide enough by default to display the number, so the text was clipped. Once I widened the column, the numbers were visible.

There was an additional problem which I didn't realize, that with my modifications, the tree does not properly align with the timelines. The indicators I can click to expand and collapse a timeline are gone, so the tree with thread numbers is fully collapsed, as if some child nodes are missing.

I'm off now, and on Fridays, so I will continue debugging on Monday.

Dave

Patrick Tasse ---08/04/2016 05:44:17 PM---Hi Dave, 1) When I switch to my second analysis, the view updates, but no intervals

From: Patrick Tasse <patrick.tasse@xxxxxxxxx>
To: tracecompass developer discussions <tracecompass-dev@xxxxxxxxxxx>
Date: 08/04/2016 05:44 PM
Subject: Re: [tracecompass-dev] Extending filtering for CallStackView
Sent by: tracecompass-dev-bounces@eclipse.org

---

Hi Dave,

1) When I switch to my second analysis, the view updates, but no intervals are shown and the tree in the left hand side of my view, where the processes and threads tree would be shown only has a line 'Stack info not available ...'. It looks like this results from two places in CallStackView.buildEntryList, when the result of getCallStackModule() is null and when module.getStateSystem() returns null. As far as I can tell by debugging, getAnalysisModules() should be returning an analysis module since my implementation of TmfTrace.getAnalysisModules() is returning a list of modules, and my modules have a valid state system in the fStateSystem field.

I suspect your cloned trace doesn't have its fAnalysisModules set, maybe executeAnalysis() was never called on it, but should it, or should it reuse the original's trace analyses?

2) When my first analysis displays, showing thread timelines, the thread id in the process/thread tree is a ':' instead of a thread number.

 That should come from the state system's attribute name, is it the same you see in the State System Explorer view?

I think my two trace objects (original and clone) have to have unique names. When the TraceEntry objects are created they are assigned the trace name. So if the names are the same I have no way to tell the TraceEntry objects apart in TimeGraphContentProvider.getElements();

Indeed, the TraceEntry doesn't have any reference to the ITmfTrace object. Perhaps you can have an algorithm that filters out the n'th occurrence of trace entries with the same name, where 'n' depends on the selected analysis?

One thing I ran into when trying to add events to an existing trace is that the TmfEvent objects contain a reference to the trace that contains them. In order to be able to add events, I had to create a new trace and then create new TmfEvent objects to add to the new trace. I don't know if anything is looking at TmfEvents in the process of building the data for the CallStackView and failing.

The Call Stack view only uses the data from the state system provided by the module, it does not read the trace events itself.

The module reads events to build the state system, but this should happen before the changes you are doing here.

Patrick


On Thu, Aug 4, 2016 at 12:13 PM, David Wootton <dwootton@xxxxxxxxxx> wrote:

Patrick
I think I've done what you suggested and have this partially working. If I switch between analyses the view does update, and the filtering of thread timelines is preserved.

However, I have two problems at this point
1) When I switch to my second analysis, the view updates, but no intervals are shown and the tree in the left hand side of my view, where the processes and threads tree would be shown only has a line 'Stack info not available ...'. It looks like this results from two places in CallStackView.buildEntryList, when the result of getCallStackModule() is null and when module.getStateSystem() returns null. As far as I can tell by debugging, getAnalysisModules() should be returning an analysis module since my implementation of TmfTrace.getAnalysisModules() is returning a list of modules, and my modules have a valid state system in the fStateSystem field.
2) When my first analysis displays, showing thread timelines, the thread id in the process/thread tree is a ':' instead of a thread number.

In order to do this, I implemented a copy constructor for my TmfTrace class and then cloned the initial trace, setting the cloned trace such that it returned the analysis module for the second analysis and giving it a unique name.

The wrapper class contains my initial trace object and the cloned trace object, and depending on which analysis I want to view, determines which trace object to query. My wrapper class also extends TmfTrace, where the methods I had to implement are just passthrus to the selected trace object. My wrapper trace object also has a unique name.

I think my two trace objects (original and clone) have to have unique names. When the TraceEntry objects are created they are assigned the trace name. So if the names are the same I have no way to tell the TraceEntry objects apart in TimeGraphContentProvider.getElements();

One thing I ran into when trying to add events to an existing trace is that the TmfEvent objects contain a reference to the trace that contains them. In order to be able to add events, I had to create a new trace and then create new TmfEvent objects to add to the new trace. I don't know if anything is looking at TmfEvents in the process of building the data for the CallStackView and failing.

Dave

Patrick Tasse ---08/03/2016 01:32:01 PM---Hi Dave, 1) I actually meant the view's content provider (you can call

From: Patrick Tasse <patrick.tasse@xxxxxxxxx>
To: tracecompass developer discussions <tracecompass-dev@xxxxxxxxxxx>
Date: 08/03/2016 01:32 PM


Subject: Re: [tracecompass-dev] Extending filtering for CallStackView
Sent by: tracecompass-dev-bounces@eclipse.org


---




Hi Dave,

1) I actually meant the view's content provider (you can call setTimeGraphContentProvider from the constructor), but you'd probably also need to call setFilterContentProvider and change that one too.

Yes what I had in mind is to override getElements(Object input) which receives the root (trace) entry list as input so that it returns a reduced array of trace entries.

2) Wrapping or copying the trace looks like the hardest part. But looking at the Call Stack view code, it looks like we actually don't need much from that trace. It needs to handle methods ITmfTrace.getName(), and it needs to be able to be used as input parameter to CallStackView.getCallStackModule() and CallStackView.getFunctionName(). I might be wrong, but if you have those covered you might not need to really worry about anything else from ITmfTrace in your wrapper.

3) I guess either the traces/wrappers have both modules and getCallStackModule() returns the right one, or each trace/wrapper returns the single correct module. Whatever works best ;)

Patrick

On Wed, Aug 3, 2016 at 12:48 PM, David Wootton <dwootton@xxxxxxxxxx> wrote:
Patrick
I'm willing to try this. It sounds like the intent is to make the single trace look like two traces so the code which currently manages the filter objects for multiple traces just things it is handling more unique traces, which sounds feasible.

To clarify some things I'm not sure about
1) The only content provider I have is to handle the input for the TimeGraphFilterDialog. (extending TimeGraphContentProvider) Is that the one you mean? If so, I'm not sure how I hide subtrees. The only way I see to select input is to override TimeGraphContentProvider.getElements() which looks like it should be given a list or array of CallStackEntry objects that I figure out which trace they belong to and filter accordingly.
2) It sounds like I need to implement a copy constructor in my class extending TmfTrace and then call that at an appropriate time. I'm thinking if I do that in my method that overrides AbstractTimeGraphView.traceOpened(), then set one trace instance as if it's using analysis 'A' and the other using analysis 'B', that will set things up properly.
3) I keep my method which overrides CallStackView.getCallStackModule(), so it returns the IAnalysis module I want as the first element (which should be harmless) so that my class extending CallStackViewer gets the state system matching the analysis I expect.
Dave

Patrick Tasse ---08/03/2016 11:11:50 AM---Hi Dave, Here's an idea, not entirely sure it will work.

From: Patrick Tasse <patrick.tasse@xxxxxxxxx>
To: tracecompass developer discussions <tracecompass-dev@xxxxxxxxxxx>
Date: 08/03/2016 11:11 AM


Subject: Re: [tracecompass-dev] Extending filtering for CallStackView
Sent by: tracecompass-dev-bounces@eclipse.org

---




Hi Dave,

Here's an idea, not entirely sure it will work.

You said that you have your own content provider, let's see if that can help.

In your extended Call Stack view, override getTracesToBuild() so that for each trace (under an experiment if applicable), it returns not one but two instance of ITmfTrace, which would be wrappers of the same trace but that provide a different call stack analysis module.

Then you have two sub-trees in the Call Stack view, with elements that persist and are not rebuilt when you switch analysis. Their respective filters also persist.

Finally you make it so that your content provider completely hides the whole sub-tree of the wrapped trace which corresponds to the not-selected analysis. When switching analysis you just call refresh() and make sure the content provider returns different roots (wrapped traces).

Patrick


On Tue, Aug 2, 2016 at 5:59 PM, David Wootton <dwootton@xxxxxxxxxx> wrote:
Patrick
Yes, the problem is that if I switch between the two analyses, that when I switch back to the first analysis, the filtering settings are gone and all of the threads in the trace are visible again. If I open the TimeGraphFilterDialog, then all of the thread entries are present and checked.

I tried changing my rebuild() calls to refresh() calls. If I make only that change, then my view does not update with the analysis that I wanted to switch to. I still see the results of the same analysis.

The problem might be that rebuild is recreating the CallStackEntry instances as you state. I did see that the fFiltersMap field is updated when a new trace is loaded, with the ITmfTrace object being the map key, and then that map is used to reset the filter when refresh or rebuild are called, so saving and restoring filters across different traces does work.

(The reason I call rebuild has to do with the way I switch between analyses. I specified both analysis to be run automatically when a trace is loaded. Then my class which extends TmfTrace implements an overriding implementation for getAnalysisModules() which determines which analysis is requested and makes sure the corresponding IAnalysisModule is element zero in the list it returns. I did this because there's a private method CallStackView.getCallStackModule(ITmfTrace) which gets the requested analysis module, and is currently implemented so that it returns element 0 of the list.)

Dave

Patrick Tasse ---08/02/2016 05:28:37 PM---Hi Dave, I'm not sure I understand correctly, is your problem that entries that the

From: Patrick Tasse <patrick.tasse@xxxxxxxxx>
To: tracecompass developer discussions <tracecompass-dev@xxxxxxxxxxx>
Date: 08/02/2016 05:28 PM
Subject: Re: [tracecompass-dev] Extending filtering for CallStackView
Sent by: tracecompass-dev-bounces@eclipse.org


---





Hi Dave,

I'm not sure I understand correctly, is your problem that entries that the user has filtered out with the dialog come back visible when you switch analyses?

I think the problem is that when you switch analyses and call rebuild(), your are creating a whole set of brand new CallStackEntry instances. They do not overload Object's equals(), so they are considered different. Consequently, after calling rebuild() the RawViewerFilter contains obsolete entries that do not match against anything anymore.

Can you avoid calling rebuild() when switching analyses? If you just make sure the content provider hides some entries it should work by just calling refresh()?

Patrick


On Tue, Aug 2, 2016 at 2:10 PM, David Wootton <dwootton@xxxxxxxxxx> wrote:
I have figured out how to run multiple analyses extended from AbstractCallStackAnalysis for the same trace. I've also figured out how to set a content provider extending TimeGraphContentProvider to my class extending CallStackView so I can make timelines for individual threads visible or not visible like the existing ability to make an entire process visible or not visible using the TimeGraphFilterDialog. I potentially have a lot of threads per process, so this helps the user reduce clutter.

The problem is that the filtering to hide or show threads does not persist when I switch between analyses for the same thread. If I invoke the TimeGraphFilterDialog to pick a set of threads to hide, the view is updated when I close that dialog.

I invoke CallStackView.rebuild() when I switch between analyses.

It seems that this method and CallStackView.refresh() use a private (AbstractTimeGraphView.fFiltersMap) map to track the specified filter active for each trace when multiple traces are open.

Since I don't seem to have access to this map, I can't update it, so whatever I set using the TimeGraphFilterDialog gets overwritten when I switch analyses.

If I had a way to update this map via getter and setter methods then I think what I am trying will work. Alternatively, if I had access to the actual RawViewerFilter object then I could update the set of objects filtered in or out. However, that is a private class defined in ShowFiulterDialogAction so I don't see any way to get access to it.

Is there any way to get access to these objects? Is there another way I could make this work?

Thanks

Dave



_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tracecompass-dev
_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tracecompass-dev




_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tracecompass-dev
_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tracecompass-dev




_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tracecompass-dev

_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tracecompass-dev




_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tracecompass-dev

_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tracecompass-dev


_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tracecompass-dev

_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tracecompass-dev