Re: [tsf-dev] TSF process feedback - part 2

[Paul Sherwood <paul.sherwood@xxxxxxxxxxxxxxx>]

Hi Derek
On 2026-05-05 00:59, Derek M Jones wrote:
> > > > It's worth thinking about what TSF is claims to do at a high-level. 
> > > > I think the optimal way to do this is evaluating TSF
>
> Who is the potential customer, i.e., the people paying with their 
> attention
> and work time?

That's quite unusual as a definition of "customer", but ok; I'm sticking 
with my previous answer...

> Beneficiaries of this will ideally include:
>
> - engineers/product teams (e.g. demonstrate "best-practices" without 
> need to absorb N proprietary standards)
> - safety assessors (e.g. reduce time spent locating and normalising 
> work-products)
> - security assessors
> - quality assessors
> - marketing (only if we gain more traction)

> The major established users are going to stay with the established 
> tools,
> those that they have always used.

Yes I expect they will, until they (the people and the tools) are 
superseded.

> These people have been trained to write safety cases.

In some cases, true. In others, I believe the activity has already been 
delegated to folks without that training, with or without the support of 
LLMs.

> Nobody writes a safety case because they want to.

I'm sure some folks do.

> The next tier down are starting to be asked about conformance to
> this that and the other.
> External stakeholders need to be satisfied.
> Regulators, legal department, major customers.

Agreed.

> There seem to be lots of tools already out there
> A survey of 10 tools, filtered down from 46 tools
> https://dl.acm.org/doi/fullHtml/10.1145/3342481

Thanks for this.

> Grok's responses to a few basic questions
> https://x.com/i/grok/share/33b5581aa03f493284b661326197609f

I'm not going there :)

> Most of these are academic tools, with no support past
> the end of the research grant that paid for them.

Quite.

> The target customer is an infrequent user.
> Deterred by the effort needed to build a case.
> Deterred by the cost of acquiring necessary skills.

This may be *a* target "customer" by your definition, but not the only 
one.

> Need to address a broad market and support an ecosystem.

True.

> Where are we?
>
> Different interpretations of a scoring function.
> Different methods of combining values to create a score.
>
> Defining one scoring function restricts the appeal of the tool.
>
> Why not allow users to calculate any score using any method?

I agree - and I expect we'll get to that.

> Provide builtin support for the common methods of
> combining values and scoring functions.

Yes, makes sense.

> Also support the calculation of multiple scores.

I think we already do. There's nothing to stop folks applying different 
algorithms (as I understand it some folks are already doing this).

> Writing conformance  statements is a skill that takes
> practice to learn and become good enough.
>
> LLMs are very good at analyzing sequences of words.
>
> Why not provide a skills assistant that:
>
>    o checks individual compliance statements
>
>    o checks a set of compliance statements
>
>    o checks a compliance document

This is an interesting idea - I just need to conquer my instinctive 
suspicion that LLMs are mostly snake-oil.

> A review of LLMs as judges
> https://arxiv.org/html/2511.02203v1
> More LLM-based tools at the end of this thread
> https://x.com/i/grok/share/33b5581aa03f493284b661326197609f
>
> A simplistic analysis of the contents of one of the TA-analysis 
> conformance
> files
> https://x.com/i/grok/share/f70c5d3b6e654c96a7f72d0b80f3b9bd

Simplistic indeed. The first point it makes is mostly incorrect, as far 
as I can tell. I could also argue with a lot of the criticisms, but 
frankly the thought of having to debate with any of the LLM services 
just makes me depressed.

> I used to spend a lot of time reading programming language standards,
> where "shall" and "must" have specific meanings.  The apparent random
> switching between the two, in these .md files certainly triggered me.

The mention of LLMs triggers me, but I'm trying to get over it :-)

> Also is the "we" set of people the same as the "our" set of people?

I use we to refer to the (nascent) community of folks taking an interest 
in TSF... which presently includes you :)

> These .md files need a lot of cleaning up.

Point taken.

> A plug for my hybrid talk at the British Computer Society
> tomorrow
> https://www.eventbrite.co.uk/e/building-models-of-software-process-behaviours-helped-by-llms-tickets-1987814433481

Is it actually hybrid? The location says "in person"

br
Paul