[jakartaee-tck-dev] Input on user requirements for specifying Specificat

[Scott Marlow <smarlow@xxxxxxxxxx>]

From yesterdays TCK call, we learned that the TCK test AssertionIDs add traceability to the TCK tests.  In the past, the TCK test development process started with determining a list of the most important SPEC/API requirements that should be tested (e.g. based on what is the most difficult to implement or likely to experience problems).  Picking the most important tests to add to the TCK, would preceed development of new TCK tests for an EE release. 

Some notes are jotted down in the Platform TCK wiki page that will eventually include the requirements for adding TCK test assertions.

As a TCK user, I am interested in reading the identified Specification/Javadoc text that TCK tests are written to validate.  I'm not especially interested in how the Jakarta EE development process deals with the specifics of mapping assertion IDs to the Specification/Javadoc sections, I would rather that each (newly written) TCK test include everything that I need to read in the test JavaDoc comments.  Another alternative could be for every TCK test failure to include context as to what the test is validating or something equivalent.

As someone who works on TCK testing, I do like the idea of picking the most important things to test from each Specification + API document but how should we handle doing that for new TCK tests?  Is that something that each SPEC API lead has in mind as they create new Standalone TCK tests?  I don't expect that they would add many tests that verify unimportant things. 

Thoughts?

Scott

Scott