[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [jakarta.ee-spec.committee] Agenda item for next call: how to maintain TCK exclude list?
|
On 11/11/20 4:39 PM, Jean-Louis Monteiro wrote:
Hi Paul and all,
For the next spec committee, I'd like to add an agenda item to discuss
how we plan to follow up and maintain the exclude list (ts.jtx).
With the current brand new release, we have an non empty exclude list.
Dunno if we have a process or not for that. So I'd like to discuss it.
We do have a TCK Process 1.0 document [1].
Here is my take on handling the non-empty excluded test list:
For existing Jakarta EE releases, we do not add/update tests, we will
only exclude challenged tests. The reason is that once an implementation
has passed the TCK for a specific SPEC version, we don't add/update
tests as that would change the rules of what implementations must do for
that particular SPEC version.
So far we have kept the initial exclude list that we started with in the
initial Jakarta EE 8 Platform TCK code drop.
I believe that for Java EE, the exclude list was truncated at the start
of development of new EE releases, so that the teams could rework the
excluded tests. I'm not sure of how that was handled exactly internally
from a process point of view.
IMO, when we start developing a new EE release, I think that the
relevant SPEC API team that deals with any excluded tests should either
attempt fixing the tests or delete them. The process of fixing the
excluded tests is a `best effort` attempt to improve the test by
ensuring it follows the relevant SPEC requirements. After we ship that
next EE release, we will hear of any TCK challenges for such improved
tests and get actual details of why the test is wrong and the relevant
SPEC team can try improving again for the next EE release or remove it
in the next EE release.
IMO, when starting a new EE release, the SPEC API teams could remedy
excluded tests if they have a clue as to why they were excluded. Or
they could guess as to how they should be improved/deleted or are fine
as they are.
An example of the current ts.jtx file for the Platform TCK is at [2].
There are also similar files for each Standalone TCK such as JPA [3].
Scott
[1] https://jakarta.ee/committees/specification/tckprocess/
[2]
https://github.com/eclipse-ee4j/jakartaee-tck/blob/master/install/jakartaee/bin/ts.jtx
[3]
https://github.com/eclipse-ee4j/jakartaee-tck/blob/master/install/jpa/bin/ts.jtx
Thanks
--
Jean-Louis Monteiro
http://twitter.com/jlouismonteiro <http://twitter.com/jlouismonteiro>
http://www.tomitribe.com <http://www.tomitribe.com/>
_______________________________________________
jakarta.ee-spec.committee mailing list
jakarta.ee-spec.committee@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee
