| Re: [cu-dev] [External] : Re: Mentor Review Feedback, Concurrency 3.1 |
Hi Kyle
There will only be one TCK tracked by the Spec. committee. Whatever that file is, should be the reference archive (docs + binaries + ancillary materials). If that reference archive contains artifacts that are used to run the TCKs, those subset archives (JARs) must have the same SHA sum of the files that are in the reference file tracked by the committee. You are confirming that it true. However the TCK ZIP, from which it is extracted isn't identical to the one listed in the _index.md. (e.g. concurrency-tck-3.1.0.zip has a SHA sum that is different from jakarta.enterprise.concurrent-tck-dist-3.1.0-dist.zip), Therefore, I have no way of knowing how these relate and our process has no way to track this other tck ZIP file. So, even though the embedded JARs are the same, the archive that contains it isn't going to match anything. Had concurrency-tck-3.1.0.zip simply been a rename of jakarta.enterprise.concurrent-tck-dist-3.1.0-dist.zip the SHA sums would have matched and we'd have been fine. In fact, since the TCK binary from within the larger archive is the same, the test results are valid. However, the TCK is defined as the binaries, ancillary files, and all their included documentation. Hence the larger, reference container archive has to ultimately be the one that we track. (I'm sorry if this is confusing.)
In short, I think, jakarta.enterprise.concurrent-tck-dist-3.1.0-dist.zip should be identical to concurrency-tck-3.1.0.zip. If there is another reason for these to differ, let me know and we can try to figure out how to resolve this. Ultimately, this file is the normative TCK and what should be referenced in all reports.
Once the TCK has the correct license, I'm sure this can all be squared away. I regret we didn't do a better job informing everyone of the new TCK and Spec. license tiles.
Regarding the number of tests -- All I want is the Spec committer team to confirm the number of tests. If that is done with these update, I'm satisfied.
Thank you,
-- Ed
On 5/1/2024 3:42 PM, Kyle Aure via cu-dev wrote:
Hey Ed,
Thanks for sending this along.
Here are responses to your concerns and some followup questions:
TCK SHA sums:
- Currently the TCK can be obtained from 3 different locations:
- jakarta.enterprise.concurrent-tck-3.1.0.jar - SHA 256: 9c16f858b19da7041125b268dd0f8c80105cd02dd3cca9c87b3abf8b81988a65
- jakarta.enterprise.concurrent-tck-dist-3.1.0-dist.zip - SHA 256: 8cb7b51ecb013c7880f0d6b2371c3e8a18cc8a1f1f73a5e3196ceee9922a7063
- Embedded JAR - SHA 256: 9c16f858b19da7041125b268dd0f8c80105cd02dd3cca9c87b3abf8b81988a65
- concurrency-tck-3.1.0.zip - SHA 256: 7b79bba4167530eb899fecb225e597346c3957f37b3b05bd825d7ab1d58512bd
- Embedded JAR - SHA 256: 9c16f858b19da7041125b268dd0f8c80105cd02dd3cca9c87b3abf8b81988a65
- So the question we (Open Liberty) has is which SHA should we be reporting?
- We reported the SHA for the zip downloaded from eclipse, but it seems we should have reported the SHA sum for the TCK jar itself.
- I have updated the certification template for concurrency to reflect this: https://github.com/jakartaee/concurrency/pull/485
TCK Test Counts:
- Pull request opened: https://github.com/jakartaee/concurrency/pull/484
- Our documentation listed the number of tests ran (268) and tests skipped (27)
- Whereas, the maven-surefire-plugin lists the number of tests total (295) and tests skipped (27)
- So the number of skipped tests is double counted and our documentation did not account for that.
Spec landing page (_index.md):Specification license text needs to be updated everywhere it appears:
- Pull request opened: https://github.com/jakartaee/concurrency/pull/483
FYI - Seeing as how I need to update the license we will need to re-build and re-stage the final release meaning we will need to re-run the TCK and post results.
Thank you,Kyle Jon Aure
On Wed, May 1, 2024 at 1:16 PM Ed Bratt via cu-dev <cu-dev@xxxxxxxxxxx> wrote:
_______________________________________________Hi there,
First off, I'm very grateful that you have delivered all the material needed for release review of this specification version.
Dmitry and I are going to be reviewing the materials you have put together for release review. As we have in the past, we will be using a couple of longer checklists to ensure that all the materials are ready to go and there aren't any SNAFUs during the ballot. I have pasted the checklist into the PR and I'll be following up if we find any issues.
Here is a short-list of issues I'd like to get your feedback on. My PR review also contains these details.
TCK
- Please revise the TCK license to EFTL v1.1. This refers explicitly to Eclipse Foundation AISBL
- License included in the TCK zip -- /LICENSE
- License in the TCK reference guide. -- since this just references by link, the only thing incorrect is that it says 'v 1.0' -- you might consider just dropping the version (though I wouldn't expect this to change again but who knows.)
- Note, I recommend this be addressed prior to the addressing the following point
SHA Sums for the TCK -- this seems to be a challenge for all of the specifications and I hope that we can simplify this in the future. The TCK that is to be referenced for release must be the exact TCK that will be posted with the final artifacts. The only SHA Sum we track is for the full distribution TCK (includes the tests, the documentation, and any ancillary artifacts). When TCKs provide subset JAR files (e.g. a binary TCK JAR), that must have the same SHA as the same JAR in the distribution. If this does not hold true, we have no way of accurately tracking that the vendor actually used the TCK that is referenced from the Specification Summary Page. I have noted the following SHA-256 Sums (note they all differ):
- TCK SHA sum referenced in Open Liberty CCR (concurrency-tck-3.1.0.zip) -- 7b79bba4167530eb899fecb225e597346c3957f37b3b05bd825d7ab1d58512bd
- TCK SHA sum referenced in Spec. PR (jakarta.enterprise.concurrent-tck-dist-3.1.0-dist.zip) -- 8cb7b51ecb013c7880f0d6b2371c3e8a18cc8a1f1f73a5e3196ceee9922a7063
- SHA Sum of contained file (jakarta.enterprise.concurrent-tck-3.1.0.jar) -- 9c16f858b19da7041125b268dd0f8c80105cd02dd3cca9c87b3abf8b81988a65
- TCK SHA sum in PR/Alternate (jakarta.enterprise.concurrent-tck-3.1.0.jar) -- 9c16f858b19da7041125b268dd0f8c80105cd02dd3cca9c87b3abf8b81988a65
The TCK artifacts in the PR seem consistent. However, the TCK used by OpenLiberty doesn't seem to match. Could you please investigate this with your contact from OpenLiberty and correct the record and/or the test target? While the Spec. committee would prefer to only track the main distribution TCK (in this case tck-dist-3.1.0), we will accept the sub-component SHA, so long as it matches the SHA in the distribution TCK.
It seems there is something different in the Staged TCK. Remember, even if you just rebuild the TCK, the SHA sums will differ.
- Please confirm the test count for OpenLiberty is as expected. The result lists skipped tests and the count total differs from the 'expected output' of the TCK User Guide (OpenLiberty reports 295 while the UG suggests 268. Both have the same number of skipped tests -- in an ideal world, the initial CCR and the UG wouldn't have skipped tests but that's not a requirement).
Spec landing page (_index.md):
- Please revise the landing page to reflect that OpenLiberty 24.0.0.6-beta is the initial CI. (the text suggests there might be another CI and I don't see another 3.1 CCR in the concurrency spec. issue list.)
- Please confirm that you are happy with the summary/change text content. To my read, it still has a bit of 'we could do this, or these bugs might be fixed). I'd recommend, for example, you pick a few issues that you think highlight the work accomplished. If you have a release tag, milestone or other change tracking document, you may refer to that as well (some document that lists all the changes).
Specification license text need to be updated everywhere it appears (in the Specifications and in JavaDocs) to reference Specification License 1.1 (this has explicit reference to Eclipse Foundation AISBL). Please revise each of the following:
- Specification PDF -- license text
- Specification HTML -- license text
- JavaDocs -- URL to license in Spec. git repository. You should update the license in the javadoc folder )y and leave the link in the JavaDocs alone or, you could revise the link in the Javadocs to point at the primary specification location (here).
Thank you!
-- Ed Bratt
cu-dev mailing list
cu-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/cu-dev
_______________________________________________ cu-dev mailing list cu-dev@xxxxxxxxxxx To unsubscribe from this list, visit https://urldefense.com/v3/__https://www.eclipse.org/mailman/listinfo/cu-dev__;!!ACWV5N9M2RV99hQ!ImR0ZAWNKMgEkghgehf4bSOg8eL1tIZ8Nm_a8ulHb2hl0vFMx52wf0Rc1L1wA1l4TBz9gVNy24EIobQ$