Re: [tracecompass-dev] CTF2-PROP-1.0: Proposal for a major revision of t

[Matthew Khouzam <matthew.khouzam@xxxxxxxxxxxx>]

On 16-10-25 02:26 PM, Philippe Proulx wrote:
> Hello fellow trace format enthusiasts.
>
> This is a proposal for a major revision of CTF v1.8 (to CTF v2).
>
> I strongly suggest that you read the HTML version at:
>
>     http://diamon.org/ctf/files/CTF2-PROP-1.0.html
>
> since the text below is an AsciiDoc source. You should, however,
> inline-comment if you have something to say about a specific section of
> the document, but _please_ keep only a few lines of context (what's
> necessary) above and below your comment, because the text is about 4500
> lines long.
>
> Other emails will follow with other CTF 2 documents. We decided to
> decouple some features and optional parts of CTF 2 in different
> documents so that each one is really focused on its own subject. Then
> producers and consumers may comply with this or that document. For
> example, as long as a consumer can decode a CTF 2 trace (following the
> specification itself), it's not the end of the world if it doesn't know
> that a given integer field type prefers to be displayed in base 16.
>
> The other documents are:
>
> * http://diamon.org/ctf/files/CTF2-DOCID-1.0.html
> * http://diamon.org/ctf/files/CTF2-BASICATTRS-1.0.html
> * http://diamon.org/ctf/files/CTF2-PMETA-1.0.html
> * http://diamon.org/ctf/files/CTF2-FS-1.0.html
>
> Feel free to question this proposal!
>
> A few things that still annoy me:
>
> * Should a boolean field type inherit the properties of an integer field
>   type instead of a simple bit array field type? In other words, should
>   a boolean field type have a signedness property?
It is ok to have it but ignore it. I would suggest that boolean not have
a sign explicitly visible though.

Here's some quick questions wrt to bools and bit arrays:

A bool is false if and only if the data is all 0, so I would argue that
byte order is less interesting than signs.

Here's a bit array question:

You have an array of 8 bool in a byte, can we use formatters to have it
display the capabilities directly?

Something like:
PORTB = 0x11011000
can it be represented natively as
OUT7 | OUT6 | OUT4 | OUT3?




>
>   Since the interesting values of a boolean field are really _true_ and
>   _false_, in my opinion we should not care about any signedness here.
>   If you need this, you can use the new union field type and match a
>   boolean field type of size X with a (signed, for example) integer
>   field type of size X.
>
> * Do we really need to support other bases than base 10 in the constant
>   integer JSON object? AFAIK, other bases are not required to encode and
>   decode any integer value. They're only there to ease human reading of
>   the metadata stream... however it's pretty much the only place where
>   such a human-friendly entity is defined, so is it really needed?
Does this go towards

*5- A CTF 2 trace /should/ be as easy as possible to consume?*

*
I would argue reading some of the json that this will not be the hardest
thing to convert to human readable.
*

>   Keep in mind that keeping the support for bases 2, 8, and 16 requires
>   each single CTF 2 consumer to be able to convert those strings to
>   integers.
>
> * There's a clear relation between some field types that, the way it's
>   written now, have no common parent.
>
>   For example, a variable-length integer field type describes fields
>   that, once decoded, provide integer values, just like the integer
>   field type does. However, they have no relation. Even though they both
>   share a `signed` property, the variable-length integer field type does
>   not need a `size` property, which is inherited from the bit array
>   field type.
>
>   Same thing for the text array field type vs. the array field type
>   (former does not need an `element-field-type` property because it's
>   implicit).
>
>   Array field type and sequence field type could also be related by
>   their common `element-field-type` property, but they are not as of
>   this version.
>
>   Do you have any idea how to bring them into relation with one another
>   without making the text too heavy? I'm thinking about some kind of
>   property mixin (or trait?) which could be applied over a field type.
>   For example, the "integer field type mixin" could define a single
>   `signed` property and both the integer field type and the
>   variable-length integer field type could claim to "implement" this
>   mixin. "Mixin" is probably not the right term. This could simplify
>   some parts of the text where a field providing an integer value is
>   needed: the text could read something like "a field type with the
>   integer field type mixin applied is needed here".
This is similar to the clock and mapping of CTF 1.8, I don't think
there's a way around it.

>
> * I'm not impressed by the clock field tags, in that we define in an
>   upper layer an m-map which can be inserted _within_ an m-map which was
>   defined in a _lower_ layer of the specification.
>
>   However I believe it's important that all field tags which target a
>   specific scope field be in the same fragment that defines the type of
>   this scope field. For example, all field tags which target the event
>   record header scope should be part of the data stream class fragment
>   where this event record header field type is defined.
See above, it's a bit of a catch 22 unless you make the clock the parent
of the trace, you will be stuck like this, since a computer has
(normally) 1 clock and can make many traces.

This is the danger of having something 100% free-form, it will be
inherently unstructured.
>
>
One thing I was having a hard time understanding was this:
*
4. The size of any CTF 2 static field, that is, any field with a
non-dynamic type, /must/ always be the same in data streams to speed up
the validation in some situations.*

In other words, a field described by a fixed-size type, or by a compound
type containing only fixed-size types, /should/ always have the same
size, no matter its offset in the data stream.

**This guarantee allows, in certain situations, to greatly speed up the
validation process by having a great part of it done at the metadata
stream level.

Could you give an example please?