Home » Archived » Test and Performance Tools Platform (TPTP) » Trace log file
| Trace log file [message #38755] |
Thu, 03 November 2005 07:35  |
Eclipse User |
|
|
|
Originally posted by: Pablo.Virolainen.nomovok.com
Hello,
Is there easy way to implement trace file format importer/converter? I
need it because my trace file's are binary dumps and I would like to use
tptp as a gui for them. (any class to fill with the data or example to
begin with?)
Basucally the files include call trace and/or sampled profile data with
some information about resource usage.
Is it true that tptp cannot join two (or more) sampled profile data, so
one could analyse at them as one entity?
Pablo Virolainen
|
|
| | |
| Re: Trace log file [message #38954 is a reply to message #38755] |
Thu, 03 November 2005 15:40  |
 Marius Slavescu
Messages: 67
Registered: July 2009 |
Member |
|
|
Hi Pablo,
One way to do this is to convert them in XML4Profiling and Statistical
events (as XML fragments) and then use the
org.eclipse.hyades.loaders.util.XMLTraceDataProcessor (or
org.eclipse.hyades.loaders.util.XMLLoader) to load them in the EMF trace and
statistical models.
If first you push the events into a file (as XML fragments) then you should
be able to use the Import profile file to load them in the corresponding EMF
model, make sure you add nodeCreate, processCreate and agentCreate events
and also set agentType attribute appropriately (e.g. agentName="Java
Profiling Agent" agentName="" agentType="Profiler" or
agentName="PerfmonAgent" agentType="Statistical").
Another way (this event processing should be much faster in this case) is to
write your import wizard where you instantiate and initialize an XMLLoader
(in the profiling trace case you also need set the collectionMode, use
XMLLoader.setCollectionMode()) and then use the following methods to feed it
(instead of loadEvent methods):
public abstract void attributeName(String name);
public abstract void attributeValueCharacters(String attributeValue);
public abstract void characters(char[] ch, int start, int length);
public abstract void endDocument(Object object, int i);
public abstract void endElement(String elementName, int currentOffset);
public abstract void startDocument();
public abstract void startElement(String elementName, boolean
hasAttributes, boolean isEmpty);
To see how to use these methods and also how to initialize the XMLLoader
please check org.eclipse.hyades.trace.ui.internal.util.PDCoreUtil and
org.eclipse.hyades.loaders.util.SAXFragmentHandler and how they interact
with XMLLoader.
Regarding correlations between the trace and statistical model, TPTP doesn't
have yet and engine or UI to do that, although we have a generic correlation
model that can be used to create one-to-many EObjects correlations (those
EObjects can come from different models).
We have a view that can visualize a generic correlation resource, but it is
currently handling only correlations between CBECommonBaseEvents object.
I opened feature https://bugs.eclipse.org/bugs/show_bug.cgi?id=114923 to
extend that view, so have a generic behavior that can handle any
correlations between any TPTP model class (for both ends of the correlation
entry - key and value entry).
Please let me know if you need more info.
Regards,
Marius
"Pablo Virolainen" <Pablo.Virolainen@nomovok.com> wrote in message
news:pan.2005.11.03.07.35.38.771076@nomovok.com...
>
> Hello,
>
> Is there easy way to implement trace file format importer/converter? I
> need it because my trace file's are binary dumps and I would like to use
> tptp as a gui for them. (any class to fill with the data or example to
> begin with?)
>
> Basucally the files include call trace and/or sampled profile data with
> some information about resource usage.
>
> Is it true that tptp cannot join two (or more) sampled profile data, so
> one could analyse at them as one entity?
>
> Pablo Virolainen
|
|
|
| Re: Trace log file [message #38988 is a reply to message #38954] |
Thu, 03 November 2005 15:51 |
Marius Slavescu
Messages: 67
Registered: July 2009 |
Member |
|
|
I forgot to add that in both cases you need to load the stream of events
using two different instances of XMLLoader, one for each agent type (virtual
agent in your case, although you could also build a real agent that could do
the conversion on the fly to XML events, then connect remotely through the
Agent Controller to that agent).
Regards,
Marius
"Marius Slavescu" <slavescu@ca.ibm.com> wrote in message
news:dkdb1m$hr2$1@news.eclipse.org...
> Hi Pablo,
>
> One way to do this is to convert them in XML4Profiling and Statistical
> events (as XML fragments) and then use the
> org.eclipse.hyades.loaders.util.XMLTraceDataProcessor (or
> org.eclipse.hyades.loaders.util.XMLLoader) to load them in the EMF trace
and
> statistical models.
>
> If first you push the events into a file (as XML fragments) then you
should
> be able to use the Import profile file to load them in the corresponding
EMF
> model, make sure you add nodeCreate, processCreate and agentCreate events
> and also set agentType attribute appropriately (e.g. agentName="Java
> Profiling Agent" agentName="" agentType="Profiler" or
> agentName="PerfmonAgent" agentType="Statistical").
>
> Another way (this event processing should be much faster in this case) is
to
> write your import wizard where you instantiate and initialize an XMLLoader
> (in the profiling trace case you also need set the collectionMode, use
> XMLLoader.setCollectionMode()) and then use the following methods to feed
it
> (instead of loadEvent methods):
>
> public abstract void attributeName(String name);
> public abstract void attributeValueCharacters(String attributeValue);
> public abstract void characters(char[] ch, int start, int length);
> public abstract void endDocument(Object object, int i);
> public abstract void endElement(String elementName, int
currentOffset);
> public abstract void startDocument();
> public abstract void startElement(String elementName, boolean
> hasAttributes, boolean isEmpty);
>
> To see how to use these methods and also how to initialize the XMLLoader
> please check org.eclipse.hyades.trace.ui.internal.util.PDCoreUtil and
> org.eclipse.hyades.loaders.util.SAXFragmentHandler and how they interact
> with XMLLoader.
>
> Regarding correlations between the trace and statistical model, TPTP
doesn't
> have yet and engine or UI to do that, although we have a generic
correlation
> model that can be used to create one-to-many EObjects correlations (those
> EObjects can come from different models).
> We have a view that can visualize a generic correlation resource, but it
is
> currently handling only correlations between CBECommonBaseEvents object.
>
> I opened feature https://bugs.eclipse.org/bugs/show_bug.cgi?id=114923 to
> extend that view, so have a generic behavior that can handle any
> correlations between any TPTP model class (for both ends of the
correlation
> entry - key and value entry).
>
> Please let me know if you need more info.
>
> Regards,
> Marius
>
> "Pablo Virolainen" <Pablo.Virolainen@nomovok.com> wrote in message
> news:pan.2005.11.03.07.35.38.771076@nomovok.com...
> >
> > Hello,
> >
> > Is there easy way to implement trace file format importer/converter? I
> > need it because my trace file's are binary dumps and I would like to use
> > tptp as a gui for them. (any class to fill with the data or example to
> > begin with?)
> >
> > Basucally the files include call trace and/or sampled profile data with
> > some information about resource usage.
> >
> > Is it true that tptp cannot join two (or more) sampled profile data, so
> > one could analyse at them as one entity?
> >
> > Pablo Virolainen
>
>
|
|
|
| Re: Trace log file [message #39032 is a reply to message #38988] |
Thu, 03 November 2005 16:14 |
Marius Slavescu
Messages: 67
Registered: July 2009 |
Member |
|
|
I missed the third method (because is the most complex one), you could load
the trace/statistical model directly and not through the XMLLoader. In this
case you will end up rewriting the loaders logic.
Are you binary traces Java specific, if not you will also need to use the
language attribute to tag them appropriately, then you could also provide
the UI application context that will handle appropriately your language in
the trace views. See Profiling and Logging -> Application Context preference
page, by default only Java is supported.
We provide an incomplete C/C++ UI application context as a sample, see
New -> Example -> UI -> Context Language Sample.
Regards,
Marius
"Marius Slavescu" <slavescu@ca.ibm.com> wrote in message
news:dkdbm8$j0t$1@news.eclipse.org...
> I forgot to add that in both cases you need to load the stream of events
> using two different instances of XMLLoader, one for each agent type
(virtual
> agent in your case, although you could also build a real agent that could
do
> the conversion on the fly to XML events, then connect remotely through the
> Agent Controller to that agent).
>
> Regards,
> Marius
>
> "Marius Slavescu" <slavescu@ca.ibm.com> wrote in message
> news:dkdb1m$hr2$1@news.eclipse.org...
> > Hi Pablo,
> >
> > One way to do this is to convert them in XML4Profiling and Statistical
> > events (as XML fragments) and then use the
> > org.eclipse.hyades.loaders.util.XMLTraceDataProcessor (or
> > org.eclipse.hyades.loaders.util.XMLLoader) to load them in the EMF trace
> and
> > statistical models.
> >
> > If first you push the events into a file (as XML fragments) then you
> should
> > be able to use the Import profile file to load them in the corresponding
> EMF
> > model, make sure you add nodeCreate, processCreate and agentCreate
events
> > and also set agentType attribute appropriately (e.g. agentName="Java
> > Profiling Agent" agentName="" agentType="Profiler" or
> > agentName="PerfmonAgent" agentType="Statistical").
> >
> > Another way (this event processing should be much faster in this case)
is
> to
> > write your import wizard where you instantiate and initialize an
XMLLoader
> > (in the profiling trace case you also need set the collectionMode, use
> > XMLLoader.setCollectionMode()) and then use the following methods to
feed
> it
> > (instead of loadEvent methods):
> >
> > public abstract void attributeName(String name);
> > public abstract void attributeValueCharacters(String
attributeValue);
> > public abstract void characters(char[] ch, int start, int length);
> > public abstract void endDocument(Object object, int i);
> > public abstract void endElement(String elementName, int
> currentOffset);
> > public abstract void startDocument();
> > public abstract void startElement(String elementName, boolean
> > hasAttributes, boolean isEmpty);
> >
> > To see how to use these methods and also how to initialize the XMLLoader
> > please check org.eclipse.hyades.trace.ui.internal.util.PDCoreUtil and
> > org.eclipse.hyades.loaders.util.SAXFragmentHandler and how they interact
> > with XMLLoader.
> >
> > Regarding correlations between the trace and statistical model, TPTP
> doesn't
> > have yet and engine or UI to do that, although we have a generic
> correlation
> > model that can be used to create one-to-many EObjects correlations
(those
> > EObjects can come from different models).
> > We have a view that can visualize a generic correlation resource, but it
> is
> > currently handling only correlations between CBECommonBaseEvents object.
> >
> > I opened feature https://bugs.eclipse.org/bugs/show_bug.cgi?id=114923 to
> > extend that view, so have a generic behavior that can handle any
> > correlations between any TPTP model class (for both ends of the
> correlation
> > entry - key and value entry).
> >
> > Please let me know if you need more info.
> >
> > Regards,
> > Marius
> >
> > "Pablo Virolainen" <Pablo.Virolainen@nomovok.com> wrote in message
> > news:pan.2005.11.03.07.35.38.771076@nomovok.com...
> > >
> > > Hello,
> > >
> > > Is there easy way to implement trace file format importer/converter? I
> > > need it because my trace file's are binary dumps and I would like to
use
> > > tptp as a gui for them. (any class to fill with the data or example to
> > > begin with?)
> > >
> > > Basucally the files include call trace and/or sampled profile data
with
> > > some information about resource usage.
> > >
> > > Is it true that tptp cannot join two (or more) sampled profile data,
so
> > > one could analyse at them as one entity?
> > >
> > > Pablo Virolainen
> >
> >
>
>
|
|
|
| Re: Trace log file [message #39397 is a reply to message #39032] |
Fri, 04 November 2005 11:43 |
| Eclipse User |
|
|
|
Originally posted by: Pablo.Virolainen.nomovok.com
Thu, 03 Nov 2005 11:14:19 -0500, Marius Slavescu kirjoitti:
> I missed the third method (because is the most complex one), you could load
> the trace/statistical model directly and not through the XMLLoader. In this
> case you will end up rewriting the loaders logic.
>
> Are you binary traces Java specific, if not you will also need to use the
> language attribute to tag them appropriately, then you could also provide
> the UI application context that will handle appropriately your language in
> the trace views. See Profiling and Logging -> Application Context preference
> page, by default only Java is supported.
>
> We provide an incomplete C/C++ UI application context as a sample, see
> New -> Example -> UI -> Context Language Sample.
>
> Regards,
> Marius
My binary traces are not Java specific. The code has been writen in some
programming language gcc supports (some parts could be writen with
assembler). Usually I get symbols from pointers with add2line (and
c++filt to demangle the symbols). If not then I'm planning to use
sprintf(symbol_name,"%p",my_pointer); as symbol.
There is at least one progam, which has some parts writen with Java, some
with C++ and some with C. Will this generate problems? (I saw
agentName="java Profiling Agent" in the Mariuses post)
One may ask why I use binary traces. It's because the target system
doesn't have too much cpu-cycles to consume for log file formetting nor
storage space. I would like to send the log with some networking
interface, but it seems to affect too much to profiling data and I don't
know how to compensate the effect.
Ultimate goal is to have environment, where I could use eclipse to build,
run and profile a program with the real and/or emulated hardware.
Building seems to be working with some hw targets.
ps. Some things are just so easy to say with programming language when you
don't know the words. It's true that I'm more familar with C/C++ than with
Java.
Pablo Virolainen
|
|
|
| Re: Trace log file [message #39460 is a reply to message #38852] |
Fri, 04 November 2005 14:44 |
| Eclipse User |
|
|
|
Originally posted by: Pablo.Virolainen.nomovok.com
Thu, 03 Nov 2005 09:36:25 -0500, Paul Slauenwhite kirjoitti:
> Hi Pablo,
> You need to write a log (or trace) file parser using the Generic Log
> Adapter to convert your binary dumps to Common Base Events, assuming you
> know how to convert your binary data to ASCII. Then, the log (or trace)
> file can be imported into the Log and Trace Analyzer for viewing, analysis
> and correlation. Any number of log and trace files may be grouped together
> (monitor, node and process levels of granularity) as one entity for viewing,
> analysis and correlation. For more information:
>
> 1) Help >> Help Contents >> Generic Log Adapter >>Creating a log parser
> 2) File >> New >> Example >> Examples >> Logging >> Log Parser Sample
> 3) http://www.eclipse.org/tptp/ >> Documentation >> Conference Materials >>
> EclipseWorld 2005 >> Using and Extending the Test and Performance Tools
> Platform [Download: source] (sample code for creating log parsers)
>
> Paul
Hi Paul,
Are you saying that I have to/it's more easy to import ASCII log that
binry log? I looked "Log parser sample" example and I couldn't find way to
read binary data.. (other thah opening file in preParse and closing it in
postParse.. It might be possible to open the binary logfile in
setConfiguration method, but I didn't find document/source code which
tells what key to use.)
It seems that the demo implements (virtual?) agent to feed the log file to
eclipse.
Pablo Virolainen
|
|
|
| Re: Trace log file [message #39520 is a reply to message #39460] |
Fri, 04 November 2005 15:06 |
Paul Slauenwhite
Messages: 975
Registered: July 2009 |
Senior Member |
|
|
Hi Pablo,
As the log file parser writer, you are responsible for converting the
records in the log file to Common Base Events. Typically, this involves
reading the lines of the log file and creating a Common Base Event when a
complete record is found. The log parser infrastructure provides the line
reading function but in your case, you need to do a conversion to ASCII
first -or- know how to consume the binary data in your log file parser. If
you have a utility that creates an ASCII version of the binary log file, you
can set the converter property on the sensor and have the sensor read the
generated ASCII log file. For example:
directory: <ASCII output file directory>
file: <ASCII output file name>
converter: <binary to ASCII converter utility> <binary input file> <ASCII
output file>
Paul
"Pablo Virolainen" <Pablo.Virolainen@nomovok.com> wrote in message
news:pan.2005.11.04.14.43.59.325204@nomovok.com...
> Thu, 03 Nov 2005 09:36:25 -0500, Paul Slauenwhite kirjoitti:
>
> > Hi Pablo,
> > You need to write a log (or trace) file parser using the Generic Log
> > Adapter to convert your binary dumps to Common Base Events, assuming you
> > know how to convert your binary data to ASCII. Then, the log (or trace)
> > file can be imported into the Log and Trace Analyzer for viewing,
analysis
> > and correlation. Any number of log and trace files may be grouped
together
> > (monitor, node and process levels of granularity) as one entity for
viewing,
> > analysis and correlation. For more information:
> >
> > 1) Help >> Help Contents >> Generic Log Adapter >>Creating a log parser
> > 2) File >> New >> Example >> Examples >> Logging >> Log Parser Sample
> > 3) http://www.eclipse.org/tptp/ >> Documentation >> Conference Materials
>>
> > EclipseWorld 2005 >> Using and Extending the Test and Performance Tools
> > Platform [Download: source] (sample code for creating log parsers)
> >
> > Paul
>
> Hi Paul,
> Are you saying that I have to/it's more easy to import ASCII log that
> binry log? I looked "Log parser sample" example and I couldn't find way to
> read binary data.. (other thah opening file in preParse and closing it in
> postParse.. It might be possible to open the binary logfile in
> setConfiguration method, but I didn't find document/source code which
> tells what key to use.)
>
> It seems that the demo implements (virtual?) agent to feed the log file to
> eclipse.
>
> Pablo Virolainen
|
|
|
| Re: Trace log file [message #39551 is a reply to message #39520] |
Fri, 04 November 2005 15:54 |
Marius Slavescu
Messages: 67
Registered: July 2009 |
Member |
|
|
Hi Pablo,
I'm not sure if you need to implement a log parser, if your data needs to be
loaded in the trace and statistical models (which are responsible to capture
profiling and perfmon related information) the log parser won't help you at
all, the output of a log parser goes in the log model (CommonBaseEvent
based).
Please see my previous notes where I explain how you can load your data in
the trace and statistical models (directly from binary files or by
converting them to XML fragments).
If you also want to load the binary logs in the log model (as Common Base
Events) then the easiest and most optimal way (for your case) is to write a
static parser for your log type which will basically implement the
sensor/extractor/parser/formatter blades from the GLA stack. You should be
able to reuse the existing outputters. This way your static parser would be
able to consume the binary log directly without any conversion to ASCII as
Paul mention it as an alternate method.
See
/org.eclipse.hyades.logging.parsers/src/org/eclipse/hyades/l ogging/parsers/A
pacheAccessLogParser.java and
/org.eclipse.hyades.logging.parsers/config/Apache/access/v1. 3.26/static.adap
ter as an example on how to implement the log parser (you will need to
change the
org.eclipse.hyades.logging.adapter.config.sensors.StaticPars erSensor with
your own that knows to read your binary log instead of an ASCII file).
Please let me know if you need more info.
Regards,
Marius
"Paul Slauenwhite" <paules@ca.ibm.com> wrote in message
news:dkftd9$r46$1@news.eclipse.org...
> Hi Pablo,
> As the log file parser writer, you are responsible for converting the
> records in the log file to Common Base Events. Typically, this involves
> reading the lines of the log file and creating a Common Base Event when a
> complete record is found. The log parser infrastructure provides the line
> reading function but in your case, you need to do a conversion to ASCII
> first -or- know how to consume the binary data in your log file parser.
If
> you have a utility that creates an ASCII version of the binary log file,
you
> can set the converter property on the sensor and have the sensor read the
> generated ASCII log file. For example:
>
> directory: <ASCII output file directory>
> file: <ASCII output file name>
> converter: <binary to ASCII converter utility> <binary input file> <ASCII
> output file>
>
> Paul
> "Pablo Virolainen" <Pablo.Virolainen@nomovok.com> wrote in message
> news:pan.2005.11.04.14.43.59.325204@nomovok.com...
> > Thu, 03 Nov 2005 09:36:25 -0500, Paul Slauenwhite kirjoitti:
> >
> > > Hi Pablo,
> > > You need to write a log (or trace) file parser using the Generic
Log
> > > Adapter to convert your binary dumps to Common Base Events, assuming
you
> > > know how to convert your binary data to ASCII. Then, the log (or
trace)
> > > file can be imported into the Log and Trace Analyzer for viewing,
> analysis
> > > and correlation. Any number of log and trace files may be grouped
> together
> > > (monitor, node and process levels of granularity) as one entity for
> viewing,
> > > analysis and correlation. For more information:
> > >
> > > 1) Help >> Help Contents >> Generic Log Adapter >>Creating a log
parser
> > > 2) File >> New >> Example >> Examples >> Logging >> Log Parser Sample
> > > 3) http://www.eclipse.org/tptp/ >> Documentation >> Conference
Materials
> >>
> > > EclipseWorld 2005 >> Using and Extending the Test and Performance
Tools
> > > Platform [Download: source] (sample code for creating log parsers)
> > >
> > > Paul
> >
> > Hi Paul,
> > Are you saying that I have to/it's more easy to import ASCII log that
> > binry log? I looked "Log parser sample" example and I couldn't find way
to
> > read binary data.. (other thah opening file in preParse and closing it
in
> > postParse.. It might be possible to open the binary logfile in
> > setConfiguration method, but I didn't find document/source code which
> > tells what key to use.)
> >
> > It seems that the demo implements (virtual?) agent to feed the log file
to
> > eclipse.
> >
> > Pablo Virolainen
>
>
|
|
|
Goto Forum:
Current Time: Thu Apr 25 16:34:08 GMT 2024
Powered by FUDForum. Page generated in 0.04432 seconds |
] 
Search
Help
Register
Login
Home
