RE: [dsdp-tcf-dev] Standardprotocols/interfaces/formatsfor performance t

["Tarassov, Eugene" <eugene.tarassov@xxxxxxxxxxxxx>]

Hi Aaron,

> Is there anywhere that I can find any details of using the TCF
extensions for binary data?

There is no separate document that describes extensions for binary data.
The API is very simple:

In Java:
  wrap a byte array into JSON.Binary class to sent it as binary data
  when receive binary data, use JSON.toByteArray() to convert it to a
byte array
In C or C++:
  use json_write_binary*() to send binary data
  use json_read_binary*() to receive binary data

C version also support "zero-copy" - sending data directly from a buffer
in kernel space, see json_splice_binary*()

We have a generic service that can be used to transmit large streams of
binary data:
http://dev.eclipse.org/svnroot/dsdp/org.eclipse.tm.tcf/trunk/docs/TCF%20
Service%20-%20Streams.html


Regards,
Eugene

-----Original Message-----
From: dsdp-tcf-dev-bounces@xxxxxxxxxxx
[mailto:dsdp-tcf-dev-bounces@xxxxxxxxxxx] On Behalf Of Spear, Aaron
Sent: Friday, February 26, 2010 7:58 AM
To: DSDP TCF dev list
Cc: linuxtools-dev@xxxxxxxxxxx
Subject: RE: [dsdp-tcf-dev] Standardprotocols/interfaces/formatsfor
performance tools (TCF, LTTng, ...)

Hi Michel,

Thanks for the reply.  Is there anywhere that I can find any details of
using the TCF extensions for binary data?  I am very interested to
understand how it works and how efficient it is for streaming large
amounts of binary data.  I am also interested to understand the LTTng
specific usage of the binary protocol.  I saw the git repo with the
Linux agent side code to support this, but I could not seem to find a
java host side implementation of the binary TCF support, but I may not
have been looking in the right place.

Thanks!
Aaron  

> -----Original Message-----
> From: dsdp-tcf-dev-bounces@xxxxxxxxxxx 
> [mailto:dsdp-tcf-dev-bounces@xxxxxxxxxxx] On Behalf Of Michel Dagenais
> Sent: Tuesday, February 23, 2010 3:04 PM
> To: DSDP TCF dev list
> Cc: linuxtools-dev@xxxxxxxxxxx
> Subject: Re: [dsdp-tcf-dev] Standard
> protocols/interfaces/formatsfor performance tools (TCF, LTTng, ...)
> 
> 
>  > The focus currently
> > is using real-time trace data as the event source for analysis.  In 
> > the future this will expand as we desire to be able to correlate 
> > analysis of heterogeneous systems, e.g. embedded Linux +
> LTTng events
> > on one machine, correlated with real-time trace data
> collected from a
> > bunch of DSP's.
> 
> A posteriori traces synchronization is an important problem in such a 
> setup. Benjamin Poirier contributed a very nice procedure to 
> synchronize traces in the latest LTTV.
> 
> > -What protocols are currently in use that we might consider as a 
> > starting point?  I see that the linuxtools project
> apparently has one
> > for transferring LTTng event data.  Are there any docs for this 
> > protocol?
> 
> We extended TCF (Michael Sills-Lavoie) to handle raw binary data, 
> offer zero copy transfer and accept plugin modules, and we built a 
> module to list and activate tracepoints. This has been contributed and

> integrated into TCF and should be the base for LTTng and Eclipse Linux

> Tools
> 
> > -file formats: event log file formats is another obvious
> candidate for
> > standardization.  Mentor has a file format we use that was
> inspired by
> > LTTng's format but is optimized for extremely large real-time trace 
> > logs.  I intend to throw this into the mix.  Any others we should 
> > think about? (The LTTng format obviously...)
> 
> A number of formats are optimized for system independance, ease of 
> parsing and other such considerations. However, for something like 
> extremely large traces, an XML type format is obviously unsuitable. 
> You want to minimize the overhead at trace creation time and have 
> something very compact. If you have improvements over LTTng's format 
> for very large traces, we could look and take the best of both!
> _______________________________________________
> dsdp-tcf-dev mailing list
> dsdp-tcf-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/dsdp-tcf-dev
> 
_______________________________________________
dsdp-tcf-dev mailing list
dsdp-tcf-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/dsdp-tcf-dev