Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [] [External] : Re: Process for TCK service releases that include TCK updates for running signature tests on newer JDK versions...

On 8/26/21 8:24 PM, Ed Bratt wrote:

I would recommend this be brought to the Specification Committee for discussion and once everyone is satisfied, that a resolution be proposed to approve this as a new TCK process guide.

It would be nice to see a change-bar version of the document (I guess I can get that from the current source diff)

Under the section 'Process for Releasing a point revision' (the last section) -- My preference would be to include documentation to reference a compatible implementation that successfully passed the revised TCK. For exclude only updates, this should be easy if you can get one or more of the original compatible implementations to rerun their tests. For updates, that add new Java versions, this could be more difficult but, presumably we'd be releasing the update for the purposes of qualifying a particular implementation so, probably that version could be included (though I guess that might not be an open-source compatible implementation). In my opinion, we always want evidence that the TCK was run and an implementation successfully passed it. Referencing the certification request associated with that implementation would be the easiest way to capture this.

The particular implementation that created the TCK challenge hasn't yet created their certification request yet as they are blocked on waiting for the new TCK release to be published.  They may also be waiting for other TCK challenges to be processed before creating their compatibility request.  I do like the suggestion but I'm not yet understanding how we can accomplish it.  At the very least, I would like the particular implementation to download the (not yet released) TCK to verify it after it has been built and communicate that the (not yet released) TCK is working as expected. 

We are also transitioning over to not having a reference implementation to use for verifying not yet released TCKs.  We can ask the various compatible implementations to test the new TCK but we cannot expect them to do that in a timely manner.

For reference, the referenced section currently contains:

Process for releasing a point revision of a TCK

The process for releasing a point revision of a TCK entails filing an issue in the jakartaee/specifications repository with the following details:

  • Link to the TCK release to be published.
  • Updated TCK links in the specification's file.
  • A high-level description of what tests were excluded from the TCK and why.



On 8/18/2021 9:16 AM, Scott Marlow wrote:

On 8/5/21 11:01 AM, Kevin Sutter wrote:
Hi Scott,
I think we should pursue an update to the TCK process to allow service releases to fix Signature tests related to newer versions of Java.  Not sure if we have to be that specific, but we do need to allow for these type of updates.  The alternative of ignoring certain tests gets tricky and nebulous since ignored tests may actually have an issue, but we wouldn't know as casual observers.  It would be much better to be clearer and service releases would allow us to be clear.  Thanks!

I just updated to be less specific about service releases to fix tests for newer versions of Java (could be signature test changes or dealing with removal of Java security manager or something else).

Does anyone disagree with updating the TCK Process version from `1.0` to `1.1`?  For what reason/condition should we update the version to `2.0`?

Does anyone else need to review before it gets merged?

Kevin Sutter
STSM, Jakarta EE and MicroProfile architect @ IBM
e-mail:  sutter@xxxxxxxxxx     Twitter:  @kwsutter
phone: tl-553-3620 (office), 507-253-3620 (office)    

Part-time schedule: Tue, Wed, Thu (off on Mon and Fri)

From:        "Scott Marlow" <smarlow@xxxxxxxxxx>
Date:        08/05/2021 08:25
Subject:        [EXTERNAL] [] Process for TCK service releases that include TCK updates for running signature tests on newer JDK versions...
Sent by:        "" <>

For Jakarta EE Platform 9.1+ supports implementations running TCK compatibility certification tests on JDK versions Java SE 8, Java SE 11+.  In support of running TCK tests on JDK versions greater than SE 11, we expect that some tests will need to be revised (e.g. see jaxb-tck/issues/57 [1] for updating signature tests related to need new signature tooling library and signature map files). 
Last December, we started making changes to the `TCK Process 1.0` that includes the following text [2] which introduces an alternative to excluding (challenged) TCK tests:
`The specification project may approve (user) workarounds for an `accepted` TCK challenge (as alternative to excluding TCK tests).`
My question today is whether the above quoted text is enough to cover Jakarta EE 9.1 compatibility certification requests against Java SE 17 (which will include signature test failures due to jaxb-tck/issues/57 [1])?  If the answer/vote is yes, certain signature test failures can be ignored on newer JDK versions, if and only if the signature test failure is caused by inadequate TCK signature support for the relevant  Java SE (e.g. JDK 17) version.  If the answer/vote is no, we will need an additional TCK process change to specifically allow a SPEC TCK service release that updates signature tests to resolve the signature test failure (e.g. allow [4] to be officially released by Spec team so that implementations can submit certification requests against
[4] mailing list
To unsubscribe from this list, visit

_______________________________________________ mailing list
To unsubscribe from this list, visit

_______________________________________________ mailing list
To unsubscribe from this list, visit;!!ACWV5N9M2RV99hQ!awIHuBnS0vFrSdaLJF1CkeydaJ6HBDuZO-HU31C_-GlXemvLd-yxK7aypDQTQqQ$ 

Back to the top