[tsf-dev] Storing TSF `Statement` information in a more structured way

[John Horsley <john.horsley@xxxxxxxxxxxxxxx>]

Having (attempted) to use TSF on a couple of projects recently I have 
some proposals, that it would be good to get some feedback on, about 
what information we should be storing in relation to TSF `Statements` 
and in my opinion it should be stored as close to the 'Statement' as 
possible i.e. in the same file. This inclusion of a 'tooling-managed' 
section brings the relevant tool generated information closer to the 
Statements (meaning reviewing diffs is easier as more of the context is 
visible) as well as keeping the separation of tooling vs human managed 
content visible.

The following proposal is not intended to define the implementation 
details, and should not be file format specific (there is no reason 
multiple different file formats e.g. YAML, TOML, etc could not be 
supported in the tooling).

The `Statement` file **should** include the following properties:

- statement (required)
  - this should be explicitly defined, rather than the 'left over text' 
as it is currently

- references (optional)
    - should be constrained to pre-defined classes, initial suggestion 
for available classes:
        - context: defined in a file (or inline text)
        - rebuttal (or fallacy?): defined in a file (or inline text)
        - misbehaviour: defined in a file (or inline text)
        - evidence:
            - for SME type validation: defined in a file (or inline text)
            - for 'automated' validation: defined location of the 
generated file.
                - a placeholder should be generated if the validator 
has not run
                - should contain (at least) the log output of the 
validator run

- assessment (required)
    - validation (required):
        - type (required): None or SME or `validator name`
        - configuration (optional): for automated validation only
    - confidence (optional): to align with current 'scoring' this 
should be the numeric confidence that the Statement is`true`, attributed 
to an SME (or SMEs)
        - for 'SME' type, stored in git based on the supplied reference 
content
        - for 'validator' type, generated at `validation` based on the 
validator tooling and output
    - (not stored in git) verdict:  True or False, generated with 
`validation` of the graph
        - if evidence type is `SME` the verdict should be `True` if the 
referenced evidence is available in the trudag context, `False` otherwise
        - if evidence type is `validator` the validator should provide this

- tooling-managed:
    - `state` of the associated content (allows tooling to assess what 
has changed), out of date content should cause an error in graph linting:
        - individually stored based on the defined content:
            - Statement
            - References:
                - pointer to the stored `state` for each reference 
(should be stored centrally as multiple statements might use the same 
reference)
                - combined `state` of all references (means that it is 
clear the references have changed when `diffing` the file)
            - Assessment config:
                - pointer to the stored `state` for validator (should 
be centrally stored as multiple statements might use the same validator)
                - combined `state` of the assessment `configuration` 
and the validator `state`
            - Child Links
            - Total:

                - combined state of 'Statement', 'References', 
'Assessment config' and 'Child Links'


The `Statement` file **should not** include the following properties, 
currently defined as YML `frontmatter` in the Markdown files:

- normative
    - I have seen 'non-normative' files used to preface a group of 
statements with a summary, if this is required functionality then there 
should be a dedicated approach for this rather than abusing the 
`doorstop` legacy 'non-normative' option.
- level & publish
    - better to have a 'nav' like approach that defines this at a graph 
level for the publishing
- evidence
    - amalgamated into `assessment` property
- score

    - amalgamated into `assessment` property



Regards,


John Horsley

Codethink Ltd