Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[jakartaee-platform-dev] Fair rules for "optional" TCK compliance tests (was: Inconsistency on the term "pruned")

> On May 22, 2020, at 1:20 PM, David Blevins <dblevins@xxxxxxxxxxxxx> wrote:
> Unless we fix that last bullet everything will stay in optional status and we'll create a situation that is in practice competitively unfair to GlassFish.  

I'd like to start a thread which does not need to complete now, but I think would be good to get going.

In our 'Inconsistency on the term "pruned"' thread I raised the point that our current TCK rules around optional are competitively unfair to GlassFish.

The rules we have now state that one implementation must implement all optional parts of a specification before that specification and TCK go final.  To understand why this is an incredible disadvantage let's talk about how optional features come into our lives and why they are so hard to remove.

It's very common that an implementation has an innovation that is very popular with their users and gains some limited traction with other implementations.  That feature often finds its way into a specification as a standard requirement.  Over time sometimes that idea never catches on with other user bases or loses favor to better techniques in the industry.  In that situation, however, the original user base tied to the implementation that founded the idea will still be there, only now dependent on the standard interfaces.

This puts us in a catch 22.

 - If we remove the feature, that means per the current rules it cannot be shipped by anyone.  This damages the user base of the implementation that innovated the idea basically saying to them, "The users of other vendors didn't find the feature you critically need to be as useful as you do, so now you can't use it anymore."

 - If we mark it optional, that means per current rules one implementation must implement it and all other optional ideas.  This damages that implementation as they're basically in the position to have to implement everyone's failed ideas.  Being the industry's technical debt collector is not competitively fair and only gets worse with time.

So what do we do?

Down one path we nuke and entire implementation and their oldest users.  Down the other path we bog and implementation down with technical debt they can't reasonably pay for as they don't have the users that need it.

Neither are good options and I suggest the only fair and just solution is we find a way to shift the debt to the implementation that not only can pay it, but needs it to survive.

Here are two ways we could solve that.  While not sexy, they are potentially better than the consequences above:

 - Require "a sufficient number of compatible implementations" to prove all optional tests.  Meaning, we may need more than one compatible implementation to be present for a specification to go final.  The Pizza specification requires Cheese and has options for Pepperoni, Onions and Pinneapple.  Joe's implementation has Onion and Peperoni.  Jane's implementation has Pepperoni and Pineapple.  Between the two of them all requirements are met and we reasonable know all parts of the TCK can be passed.  Until both show up, however, the specification and TCK cannot be released.

 - Split optional tests into their own spec/tck that can be implemented and released later.  We see over time Willy is the only implementation shipping Pineapple and is not keeping up.  We split out the Pineapple tests into its own specification/tck and it sits there till Willy is able to pass it.  From the Pizza Platform perspective it is not optional or removed, it simply has reverted back to being a standalone specification with no connection to the Platform.

I suggest there is no universal decision we need to make between A or B.  In some situations we may chose A, in others B.  We may in fact find ourselves in the patter where we try A for a while and revert to B if we see impact on our timelines.


David Blevins

Back to the top