| Re: [jakarta.ee-spec] [External] : Re: Process for TCK service releases that include TCK updates for running signature tests on newer JDK versions... |
I would recommend this be brought to the Specification Committee for discussion and once everyone is satisfied, that a resolution be proposed to approve this as a new TCK process guide.
It would be nice to see a change-bar version of the document (I guess I can get that from the current source diff)
Under the section 'Process for Releasing a point revision' (the last section) -- My preference would be to include documentation to reference a compatible implementation that successfully passed the revised TCK. For exclude only updates, this should be easy if you can get one or more of the original compatible implementations to rerun their tests. For updates, that add new Java versions, this could be more difficult but, presumably we'd be releasing the update for the purposes of qualifying a particular implementation so, probably that version could be included (though I guess that might not be an open-source compatible implementation). In my opinion, we always want evidence that the TCK was run and an implementation successfully passed it. Referencing the certification request associated with that implementation would be the easiest way to capture this.
The particular implementation that created the TCK challenge
hasn't yet created their certification request yet as they are
blocked on waiting for the new TCK release to be published. They
may also be waiting for other TCK challenges to be processed
before creating their compatibility request. I do like the
suggestion but I'm not yet understanding how we can accomplish
it. At the very least, I would like the particular implementation
to download the (not yet released) TCK to verify it after it has
been built and communicate that the (not yet released) TCK is
working as expected.
We are also transitioning over to not having a reference
implementation to use for verifying not yet released TCKs. We can
ask the various compatible implementations to test the new TCK but
we cannot expect them to do that in a timely manner.
For reference, the referenced section currently contains:
The process for releasing a point revision of a TCK entails filing an issue in the jakartaee/specifications repository with the following details:
"
Scott
On 8/18/2021 9:16 AM, Scott Marlow wrote:
On 8/5/21 11:01 AM, Kevin Sutter wrote:
Hi Scott,
I think we should pursue an update to the TCK process to allow service releases to fix Signature tests related to newer versions of Java. Not sure if we have to be that specific, but we do need to allow for these type of updates. The alternative of ignoring certain tests gets tricky and nebulous since ignored tests may actually have an issue, but we wouldn't know as casual observers. It would be much better to be clearer and service releases would allow us to be clear. Thanks!
I just updated https://github.com/jakartaee/jakarta.ee/pull/1018 to be less specific about service releases to fix tests for newer versions of Java (could be signature test changes or dealing with removal of Java security manager or something else).
Does anyone disagree with updating the TCK Process version from `1.0` to `1.1`? For what reason/condition should we update the version to `2.0`?
Does anyone else need to review https://github.com/jakartaee/jakarta.ee/pull/1018 before it gets merged?
---------------------------------------------------
Kevin Sutter
STSM, Jakarta EE and MicroProfile architect @ IBM
e-mail: sutter@xxxxxxxxxx Twitter: @kwsutter
phone: tl-553-3620 (office), 507-253-3620 (office)
LinkedIn: https://www.linkedin.com/in/kevinwsutter
Part-time schedule: Tue, Wed, Thu (off on Mon and Fri)
From: "Scott Marlow" <smarlow@xxxxxxxxxx>
To: jakarta.ee-spec@xxxxxxxxxxx
Date: 08/05/2021 08:25
Subject: [EXTERNAL] [jakarta.ee-spec] Process for TCK service releases that include TCK updates for running signature tests on newer JDK versions...
Sent by: "jakarta.ee-spec" <jakarta.ee-spec-bounces@xxxxxxxxxxx>
For Jakarta EE Platform 9.1+ supports implementations running TCK compatibility certification tests on JDK versions Java SE 8, Java SE 11+. In support of running TCK tests on JDK versions greater than SE 11, we expect that some tests will need to be revised (e.g. see jaxb-tck/issues/57 [1] for updating signature tests related to need new signature tooling library and signature map files).
Last December, we started making changes to the `TCK Process 1.0` that includes the following text [2] which introduces an alternative to excluding (challenged) TCK tests:
`The specification project may approve (user) workarounds for an `accepted` TCK challenge (as alternative to excluding TCK tests).`
My question today is whether the above quoted text is enough to cover Jakarta EE 9.1 compatibility certification requests against Java SE 17 (which will include signature test failures due to jaxb-tck/issues/57 [1])? If the answer/vote is yes, certain signature test failures can be ignored on newer JDK versions, if and only if the signature test failure is caused by inadequate TCK signature support for the relevant Java SE (e.g. JDK 17) version. If the answer/vote is no, we will need an additional TCK process change to specifically allow a SPEC TCK service release that updates signature tests to resolve the signature test failure (e.g. allow jakarta-xml-binding-tck-3.0.2.zip [4] to be officially released by Spec team so that implementations can submit certification requests against jakarta-xml-binding-tck-3.0.2.zip).
Scott
[1] https://github.com/eclipse-ee4j/jaxb-tck/issues/57
[2] https://github.com/jakartaee/jakarta.ee/pull/1018/files#diff-1fe254a18287c0db31fd9cb0a6bca11b1efda926095c3a65b73ef2ae0c89360dR223
[3] https://jakarta.ee/committees/specification/tckprocess/
[4] https://download.eclipse.org/ee4j/jakartaee-tck/jakartaee9-eftl/promoted/jakarta-xml-binding-tck-3.0.2.zip_______________________________________________
jakarta.ee-spec mailing list
jakarta.ee-spec@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec
_______________________________________________ jakarta.ee-spec mailing list jakarta.ee-spec@xxxxxxxxxxx To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec
_______________________________________________ jakarta.ee-spec mailing list jakarta.ee-spec@xxxxxxxxxxx To unsubscribe from this list, visit https://urldefense.com/v3/__https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec__;!!ACWV5N9M2RV99hQ!awIHuBnS0vFrSdaLJF1CkeydaJ6HBDuZO-HU31C_-GlXemvLd-yxK7aypDQTQqQ$