Re: [jakartaee-tck-dev] [jakartaee-platform-dev] Discussion of enabling

[David Blevins <dblevins@xxxxxxxxxxxxx>]

On Apr 11, 2022, at 8:23 AM, Scott Marlow <smarlow@xxxxxxxxxx> wrote:

On 4/10/22 6:33 PM, Emily Jiang via jakartaee-tck-dev wrote:

As you may know, we have been discussing the issue of the standalone JSON-B, JSON-P executed in the SE container only. Last week, the issue was brought up and discussed among the Jakarta Specification committee. There was a suggestion to revert the changes of deleting JSON-B and JSON-P TCKs from the Jakarta Platform TCKs so that these TCKs can still be executed in the EE container as a short-term compromise for the Jakarta EE 10 release. We can discuss further at the platform call on the 12th. Any thoughts or comments welcome!

Regarding adding those tests back into the Platform TCK, it may help the conversation if someone gives an example of an EE container bug that might not of been revealed if we didn't add those tests back in. 

I know we frequently see integration issues with components that cause large numbers of tests to fail.  When I say that, people should not read that to mean the features of a spec that integrate with other specs.  I mean basic functionality of the component spec failing when run in a fully integrated container.

These are some of the scenarios I tend to see the most:

 - Any sort of pluggable interface in the component.  A good deal of components have an interface or three that allows for default implementations to be replaced.  Typically for creating instances, doing dependency injection, scanning for annotations, etc.  Basically if there is anything pluggable about the component implementation, you have to run the tests again once you've integrated it.

 - Ensuring default configuration/settings are compliant.  It was a rule in the Java EE days that your default settings had to be compliant.  The intent being what you ship is out-of-the-box in the state that passes the TCK.  This is not written down as a rule in Jakarta EE, but we still try to follow that principle in our server.  Sometimes tests fail because the component implementation has some setting which is not set as the default and we need to change it.  They are often system properties and you don't know they even exist or are important until you run the TCK tests on your server and see failures.

These are the biggest.  They're basically unsolvable without being able to run the tests again in the container.  From a purely specification and TCK perspective, no matter how trivial we think a spec is we can't really dictate to implementations they cannot offer pluggable parts, extensions or configuration that changes their behavior.

This is good behavior we want to enable with platform/profile TCKs.  Tighter integrations that avoid overhead or add features is how we enable platform/profile implementations to compete.

We do see other issues, such as conflicts with META-INF/services files, classloaders, NoClassDefFoundErrors from missing depndency jars you didn't think you needed, etc. but there's no point in going into detail on those as I think people would just give advice on how to never have these kinds of issues so the TCK didn't have to play a role in assuring they don't happen.  Though, I would likely argue that's one of the roles of a TCK.

-David