| Re: [jakarta.ee-spec] Compatibility Certification Requirements for EE 9.1 |
Updates:
These certification requests for stand-alone TCKs not produced by
the Platform TCK project that run on JDK 11 are already in place
I wrote Compatibility on the list below, but it should have been
Debugging. This TCK is run by the Platform TCK project but the TCK
is produced out of the jsp-api project. The TCK is unchanged and
the Platform certification job validates the TCK (on JDK 11 and 8)
--
Based on these finding, I believe that everything needed to produce this CCR is already in place.
So -- we just need to decide that the evidence (the CCRs) should
be produced (and projects still have the option of using their own
results if they wish). If we do take that decision, I can get
started on this right away.
Thanks,
-- Ed
In the Jakarta EE Specification committee, we had a lengthy discussion about compatibility certification requirements for Jakarta EE 9.1. This e-mail is my attempt to describe the problem and also describe a solution.
Jakarta EE 9.1 has explicitly set a requirement that it will support JDK 11. Jakarta EE 9, explicitly required JDK 8. The 9.1 platform specification now implies that all component specifications from 9.0 are also valid for JDK 11. The thought experiment I offered in the Specification Team meeting today was: "If a user were interested in running the stand-alone Servlet TCK, should they expect that TCK would work correctly on JDK 11?" I think the answer to this question is "yes." If there is agreement on this, expanding on that a bit -- one might then ask -- what evidence is provided to demonstrate that the JDK 11 requirement for each and every TCK is verified?
It is my position that each specification must have a compatibility certification request (CCR) that verifies their TCK runs correctly on JDK 11. (The good news, is we already have almost everything we need.)
During this project, we have been tracking the TCK updates and result status using the Jakarta EE Platform TCK (AKA CTS).
Stand-alone TCKs that are created by the Jakarta EE Platform TCK project have also been rebuilt -- changed or not. These are planned to be released as service releases. However, the SHA Sum values we referenced for their previous Jakarta EE 9 release CCR and the component specification pages will need to be updated and we cannot just replace the EE 9 entry since that was, and still would be an accurate TCK for Jakarta EE 9.
When Compatibility Certification Requests for Jakarta EE 9 were created, published, and approved, these were all verified against JDK 8.
If we proceed only with Platform compatible certification requests, we will be providing evidence that an EE Platform (likely multiple platforms, yay!) pass the Jakarta EE Platform TCK and that the platform implementation is compatible with the component specifications that the Platform Specification indicates it can be used to certify as compatible. However this does not demonstrate that the component specifications have verified their stand-alone TCKs.
Some background about the Platform TCK project: The stand-alone TCKs from the Jakarta EE Platform project are produced from common TCK test code that is used to create the Platform TCK and all the respective stand-alone TCKs. As part of the production process, to verify the stand-alone TCKs, GlassFish is used as a "known archive" of compatible implementations -- This is true only for the specific specifications that have stand-alone TCKs and are produced from this project. That list of specifications is identified in [1] below. There are 19 specifications in this list.
The stand-alone TCK verification uses compatible implementation from the component project as a quality measure before release. It has traditionally been up to the project teams to decide if they want to use the test results from the Platform TCK project, or if they want to provide separate implementation results. For example, Enterprise Security and the Soteria implementation have generally opted to take the results produced by the Platform TCK project. Eclipse Jersey and Jakarta REST generally opt to use their own test results even though the Jakarta EE Platform TCK project has these data available. In all these cases, the TCK can be verified and documented -- what was tested -- in what operating environment, and with what JDK in a CCR.
The component specifications are not changing. However, the TCKs produced by the Platform TCK project have been rebuilt. These new TCKs are being tested compatible against compatible implementations contained in Eclipse GlassFish, with JDK 11.
Without a new CCR, even the specifications with TCKs that are not changing will have produced no evidence that their TCK runs correctly with JDK 11. In cases where the previously used compatible implementation will pass on both JDKs -- great -- all that is needed is a verification re-run, and an update that denotes that fact.
If the previous statements are found to be accurate and agreed upon, the following can be documented with the tests that are already being run, today:
All specifications in lists [1] and [4] below can verify that their TCKs pass with JDK 11, using the results we are currently generating out of the Platform TCK project (25 specifications). Specification stand-alone TCKs from list [2] are explicitly included with each platform CCR (that brings us to 29 specifications). These results are also being produced in the Jenkins workspace for the Jakarta EE TCK project. The remainder, projects from list [3], would need to have their current stand-alone TCK and implementation status validated. I'm pretty confident they are in good shape and I don't anticipate much delay getting these data.
A recent TCK test run Blue Ocean "graph" is available here (recent results with Eclipse GF 6.1, JDK 11, Full profile). A recent stand-alone TCK test run, Blue Ocean "graph" is available here (recent results of all stand-alone TCKs that are produced from the Jakarta EE TCK Project, run on JDK 11).
I am willing to do most of the documentation work for all of this (see [5] below for a breakdown) and I think there is sufficient time to accomplish this before the Platform and Web Profile specifications are taken to ballot, presuming that the TCK results hold at their current level -- if any component projects want to produce their own CCR for JDK 11, they are welcome to do that. If we can get Spec. teams to proactively approve the CCRs, we can probably have them all completed before the platform ballot is started (but that's not certain). I am confident most, if not all, of the CCRs can be submitted before the platform ballot starts.
As I said in the Spec. meeting today -- I thought this was the plan all along but -- even if that wasn't well understood, I believe this is an evidence requirement that we must adhere to. I think this provides the evidence trail that we want to maintain for Jakarta EE and with luck, meeting this requirement won't impact any project schedule targets. In most, if not all cases, we're already doing the work. It's just a matter of doing the paper work to document that fact. I'm willing to do nearly all that "paperwork."
-- Ed
[1] -- Stand-alone specification TCKs that are produced by Jakarta EE Platform TCK project:
- Annotations
- Concurrency
- Connector
- _expression_ Language
- Authorization
- Authentication
- Rest
- Messaging
- Persistence
- Faces
- JSONB
- JSONP
- Server Pages
- Tag Library
- Transactions
- SOAP with Attachments
- Enterprise Security
- Servlet
- WebSocket
[2] Specifications that are explicitly called out by Platform specification and must be included in a platform compatibility certification request:
- DI
- CDI
- Bean Validation
- XML Binding (required for JDK 11 and higher)
[3] Specifications with stand-alone TCKs that will validate component compatibility for a given platform, but are not produced by the Platform TCK project:
- Activation
- Batch
- Compatibility
[4] Specifications that are certified compatible only with the Platform TCK (i.e. do not have stand-alone TCKs or are the platform):
- Platform
- Web Profile
- Enterprise Beans
- Enterprise Web Services
- Managed Beans
- Web Services Metadata
[5] Work needed to validate:
- Collect the current TCK result data (using the data generated in Jakarta EE TCK Project)
- Generate TCK result pages (under the Eclipse GlassFish web-site)
- File the necessary CCRs for each specification
- File PR(s) to add these TCK details to the Specification pages (Add the link to the 9.1 TCK; as needed, add the compatible implementation details.)
- File the component certification e-mail with the EMO.