| Re: [jakartaee-tck-dev] [jakartaee-platform-dev] Fwd: [jakarta.ee-spec.committee] [External] : Re: TCK archive format? (Consider allowing JAR or ZIP) |
We need a proposal to coalesce around, before we can contemplate
when we would want to make it a recommendation or requirement.
My guess is, we couldn't do anything more than make a
recommendation for EE 10. To do that, we'd need to relax the TCK
requirements we have now. Relaxing the requirements will require a
committee vote. If we cannot find a consensus, then we'd need all
Spec groups to produce a single ZIP file and anything available
via Maven would be for "convenience" -- not "normative."
This is likely only an issue for the handful of specifications
that have separated their TCKs out of Jakarta EE Platform TCK in
EE 10.
Long term, I believe we are all in agreement that we'd like to
solve this problem so multiple downloads and test runs are not
required. If a proposal doesn't gel quickly and/or the committee
isn't able to act quickly to change the process, we'll just have
to use the current requirements and work to solve this for a
subsequent release of EE. At this point, that does seem like the
probable outcome.
-- Ed
Are you proposing to update the TCK process for EE10 to accommodate this change? As far as I can determine, only the specification committee needs to approve an update the to TCK process.
On Feb 15, 2022 at 6:25:22 PM, Ed Bratt <ed.bratt@xxxxxxxxxx> wrote:
Forwarding a copy of the discussion previous -- related to packaging and delivery of artifacts for publication through Sonatype Maven ... (Sorry, I don't know how to host these cross-group discussion since the membership lists is not synchronized.)
We agreed this morning to discuss potential proposals for revised TCK delivery archives that the Platform team might want to recommend to the Spec. committee in it's call Wed. next week. We created an item to discuss this at the next Platform committer meeting (Tuesday a week from today).
The forwarded message thread is the conversation that, for the most part took place late last year. We weren't facing the final specification ballot reviews at that time and we never made any recommendations.
My personal goal is that we adapt the process requirements to allow users to pull the binary TCK from Maven Central such that this allows for a final certification to be completed and the results submitted without any need to acquire an additional download from an alternate location (e.g. from download.eclipse.org). Users may need to read and abide by textual details that are in the ancillary TCK documents, but the run-time test aspects can be completed in a valid way, without running the tests in any 'special' way, using an officially sanctioned artifact. As I understand the process, today's process requirements doesn't make that possible. In fact, none of the TCKs that previously came from the Jakarta EE TCK Project were 'officially' available from Maven Central. Other component specifications may have provided artifacts via Maven Central, but unless they are SHA-SUM identical to the TCK provided at download.eclipse.org, that would not be a technically valid TCK. (Even the TCKs that are provided under the Project License are not considered acceptable for certification since the sanctioned TCK archives have the EFTL license rather than the Project License).
I don't feel qualified to propose a layout for material that is to be delivered through Maven central. I will summarize the elements listed in the TCK Process here (from Jakarta EE TCK Process 1.2):
Required:
- The binary TCK, one with the project license, one with the EFTL license
- TCK is to be delivered as a final artifact and is not to be re-built by the certifying user/organization.
- User Guide (or equivalent)
- Software requirements
- Installation and configuration
- How to run the tests
- Where to file challenges
- TCK Rules not covered in the process guide
- Instructions describing how to run the compatible implementation(s) that are being used to validate the TCK
- Top-level README pointing to each of the preceding documents
Recommended:
- URLs for issue trackers for certification requests, bug reports, etc.
- Statement that 'certification of compatibility' process must be followed before a compatibility claim can be made
Optional content
- Test coverage document
- Test assertion document
The Jakarta EE TCK Process asserts that TCKs must be hosted from download.eclipse.org.
Additionally -- this document requires a SHA-256 hash to identify the TCK that was used to determine compatibility.
Some issues to consider
- If the Spec. committer group wishes, can they produce a single dual-licensed (project license + EFTL license) artifact (zip or jar)?
- Is a single artifact that contains all of the above a requirement? Otherwise if multiple artifacts are allowed (i.e. a run-time JAR, an expanded Source JAR, etc.), how would these to be tracked (SHA-1 vs. SHA-256, for example)? Multiple SHA sums (I'd presume).
Finally, if it wasn't ever stated to teams, I'd recommend all teams that are refactoring their TCK out of the Jakarta EE Platform TCK project, review the contents of that specification's 9.1 version TCK and use that, at least, as a starting point for their refactored TCK for EE 10. If anyone has questions (e.g. "where did this user guide come from?", "what is an assertion list?", etc.), questions can be answered here, and also in the Jakarta EE Platform TCK mailing list jakartaee-tck-dev@xxxxxxxxxxx. Prior to EE 10, none of the TCKs from the Jakarta EE TCK Project had been delivered via Maven Central. No doubt, we'll have some translation/porting issues to contend with as we make this transition.
Anyone that has a proposed layout and/or publication to Maven proposal, please feel free to make a recommendation or explain how their project might provide a suitable archive.
Thanks,
-- Ed
-------- Forwarded Message --------
Subject: Re: [jakarta.ee-spec.committee] [External] : Re: TCK archive format? (Consider allowing JAR or ZIP) Date: Fri, 11 Feb 2022 16:03:51 -0800 From: Ed Bratt <ed.bratt@xxxxxxxxxx> Reply-To: Jakarta specification committee <jakarta.ee-spec.committee@xxxxxxxxxxx> To: Jakarta specification committee <jakarta.ee-spec.committee@xxxxxxxxxxx>, Mike Milinkovich <mike.milinkovich@xxxxxxxxxxxxxxxxxxxxxx>
The time has arrived for us to settle on the questions here. I believe we need to make some choices about what we'll allow for EE 10.
The simplest but not most convenient choice is to simply state that TCKs will be archived and distributed as .zip files just as they have been in previous releases. The more complex choice is approve changes that will allow the final/normative TCKs to be referenced directly from Maven.
In my perspective, the User Guide documentation that is included can be quite lengthy -- and is more involved than a Readme type document. As Scott Stark pointed out -- to properly understand a TCK, one must review the documentation and know the details of the runtime requirements, configurations, extended instructions, etc. before using them. Though forcing them to be delivered in the download doesn't ensure that the user documentation is read or understood.
If we are going to allow Maven publication, I believe the following must be part of any recommendation we provide:
- The license must be dual license -- EFTL + EPL (EFTL is not a compatible open-source license with Sonatype's ToU). I believe this dual license should be available for Spec. teams to include (i.e. we ought not recommend Spec. teams concatenate these licenses together on their own)
- The source-code that comes from refactored TCKs from the Jakarta EE Platform TCK includes EPL + GPLv2 headers. I do not believe that we expect to alter these headers. However, the TCK use license, for certification must be EFTL. Since this has been vetted previously, I will presume, so long as the EFTL + EPL license is included in our approved manifest location, we can continue in this fashion. I don't know if anyone at Eclipse needs to review any of this. (This point is independent of using Maven as a publication platform)
- The User Guide (Reference Guide, Readme) must be available (either included in the run-time JAR, or perhaps included in the sources bundle) or we must include a reference that directs the user to obtain and understand it prior to submitting a certification request. These documents may be many pages and include images. In previous Jakarta releases these are provided in both HTML and PDF format.
- Since the User Guide (etc.) is part of the normative requirements for a specification, if we allow / recommend these be included in a separate archive, we will need to track SHA Sums of all the normative document/archives.
- Currently we require a SHA-256 sum. Sonatype provides SHA-1. When we approve use of Maven, should we allow SHA-1 instead?
We /could/ agree that the TCK runtime is published separately from the documentation. Further that we publish the runtime as a separate archive (presumably a JAR)-- but to both Maven and to the Specifications download (using a SHA-SUM to verify the two are identical) -- and simply refer vendors (perhaps via short read-me) to the separate downloadable TCK documentation bundle. This creates some problems while solving others.
This is somewhat urgent since I need to provide guidance to the Concurrency Utilities team regarding what we will allow/not allow with their current release proposal.
Thanks,
-- Ed
On 12/9/2021 8:57 AM, Ed Bratt wrote:
Mike,
I believe we have previously discussed and received assurance that dual-licensed TCKs (any artifact) are acceptable to Sonatype. So, these would require both the TCK license and the project license. Currently, the TCKs are produced under the project license while they are under development. When they are finalized, they are made available under the TCK license. I thought that we determined dual license was acceptable from the Working Group / Eclipse side but, if you are not sure, please do check if this is still allowable.
The certification process steps still require the Compatibility Certification Request with it's attestations. The Branding and Trademark processes is also still a requirement for those who choose those paths.
Subject to your assurance that this is an allowable delivery pattern, my goal with this discussion is to see if there is an opportunity for us to make the TCKs operate like any other Java element and can be built into the same developer delivery channel as other Java artifacts. I view this as a simplification and broadening of availability effort.
Emily and others, when a CCR is submitted, the submitter is required to include the SHA-256 hash of the TCK package that they used for their certification. Only finalized and approved TCKs and their corresponding SHA sums are acceptable. Therefore, only a single distribution is allowed -- under our current process.
Part of this discussion is to work out how to adjust our process, if we can find a proper solution that meets all procedural and mechanical requirements.
-- Ed
On 12/9/2021 5:59 AM, Mike Milinkovich wrote:
Beyond any technical considerations the licensing and patent IP flows in the JESP/EFSP are predicated on the use of the normative TCK binary made available under the TCK License. So there would be legal analysis required to enable this scenario.
On 2021-12-08 7:13 p.m., Scott Stark wrote:
I see what you are driving at, but this will definitely take some thinking about how to achieve. One could probably publish all of the relevant artifacts to a maven repo using qualifiers/naming conventions to cover user guide, exception list, etc.
On Dec 8, 2021 at 4:58:57 PM, Ed Bratt <ed.bratt@xxxxxxxxxx> wrote:
Maybe the problem is with how we are "packaging" the TCK content.
The TCK is the only Spec. artifact we create a SHA Sum for. The archive we use for the TCK includes the documentation which is generally not part of a typical Java binary JAR (i.e. at Maven Central, there's typically a Binary JAR, a Source JAR, a Java Doc JAR and maybe others -- but they're all separate).
We currently bundle all the TCK component elements into a single archive -- a ZIP archive. We treat that as a single deliverable. At the Spec. generation end, that makes production of that archive a multi-step process. Contrast that with the publication of the API, binary, and Java Doc JARs with the code-based resources can directly be published to, Maven central (or another repository service if the vendor liked). So, the normative code-based Spec. artifacts can be easily pulled into an Mvn script but the normative TCK elements cannot.
I'd like the TCKs to be accessible via maven because I think that improves the access flow for developer users. I understand that developers typically will need additional details to understand how to use these artifacts but, once they have that worked out -- I'd like to improve how these can be distributed more readily, using common build tools (I typically think mvn, but others also exist). I don't think using the TCK artifacts are all that much different from using a binary API jar file -- you have to know how to use the JAR before you pull it in.
In the context of the process we have today -- a binary (run-time) TCK JAR cannot be pushed to Maven central as a normative artifact because we track the zip bundle with the SHA SUM and Signature hash. To "properly" certify an implementation, even if we pushed the binary elements of the TCK to Maven, users would still need to pull down the normative copy, unpack and "install" those artifacts, then run/validate their implementation with that binary. (Or, just attest that is what they did. I'd assert that we don't want a process that promotes 'looking the other way.')
So, I thought, if perhaps we could just create a JAR file that had the same content as what is currently in the ZIP -- thinking that maybe we might simplify that process by one step -- but I'm now seeing that probably isn't going to 'just work.' So just changing the extension name from .zip to .jar won't cut it.
I want to make it possible for developers to pull normative TCK binaries down from Maven (or whatever their chosen repository happens to be). I want the artifacts that are pushed to the Jakarta EE Spec. download to exactly match what can be accessed from Maven -- or at least the run-time parts that are actually used at build-time.
Today, before the Spec. is finally approved, I can pre-test my implementation using the pre-final TCK .zip files. Once the Spec. is finalized, that zip is exactly copied to the Jakarta EE Specification Download folder and the SHA-SUM and Signature hash are generated (the SHA sum won't change because the binary is identical and the signature hash works correctly too). So, I don't need to re-run the TCK because it's unchanged (in any scenario where a change might have been needed, the SHA Sum would not be the same and I would need to re-verify the compatibility of my implementation -- even if the change was to a non-binary element of that archive.)
I was hoping we could figure this out for EE 10, but maybe it's just too late to consider making changes to allow for this that rapidly.
Does this make sense?
-- Ed
On 12/8/2021 11:36 AM, Scott Stark wrote:
There is no access to a user guide (required) and any example CI setup that is often included in a full TCK dist. Users certainly should have the ability to directly access the components directly via a repo, but I don't see how that can be the only mechanism when it comes to ratifying the associated spec. Am I missing the context in which you are asking about this?
On Dec 8, 2021 at 12:09:53 PM, Ed Bratt <ed.bratt@xxxxxxxxxx> wrote:
Thanks Scott.
This seems to suggest that users should NOT have access directly to the normative TCK artifacts -- That they should pull the comprehensive archive, unpack and digest it, THEN, utilize the contents. Is that a fair re-state?
We include GAVs for the API jars and JavaDocs. Why would we want to enforce force additional overhead on the TCKs (both on construction of the archive and on consumption)? (Or, what it is about TCK usage that I'm minimizing.)
-- Ed
On 12/8/2021 8:02 AM, Scott Stark wrote:
For the final submission I still think there needs to be a single zip that is comprehensive. The use of jar artifacts in a repo facilitates the ability to easily integrate TCK testing into CI environments, but it is a bad starting point to running the TCKs. Only after you have gone through the full zip distribution and associated user guide/example do you start wanting the individual artifacts.
On Dec 8, 2021 at 9:58:16 AM, Ed Bratt <ed.bratt@xxxxxxxxxx> wrote:
Hi,
As we move the TCK responsibilities to component projects and, as we
continue migrating away from the legacy TCK build tools, to MVN or other
build tooling, I wonder if we should hold on to the requirement that we
only accept .zip file bundles for normative TCK submissions.
Would the committee be interested in relaxing the formal requirements to
allow either <name>.zip OR <name>.jar? If we did this, projects would
not need to create multiple bundles and we could start to use Maven
Central or other types of distribution mechanisms more directly.
I believe all other requirements (SHA Sums, contents, etc.) can be
maintained just as directly with either file format.
Comments?
I don't know that we could adopt this before the close of the year, but
... perhaps this is simple enough to take up without extensive discussion.
Thanks,
-- Ed
_______________________________________________
jakarta.ee-spec.committee mailing list
jakarta.ee-spec.committee@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee
_______________________________________________ jakarta.ee-spec.committee mailing list jakarta.ee-spec.committee@xxxxxxxxxxx To unsubscribe from this list, visit https://urldefense.com/v3/__https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee__;!!ACWV5N9M2RV99hQ!a7DQDPqHeJq3Q0JbFqLFGnY9MKejqQtnohN_jFIYScnrOBo80YCr4N58Bxi9VNM$
_______________________________________________ jakarta.ee-spec.committee mailing list jakarta.ee-spec.committee@xxxxxxxxxxx To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee--
Mike Milinkovich
Executive Director | Eclipse Foundation AISBL
Twitter:@mmilinkov
_______________________________________________ jakarta.ee-spec.committee mailing list jakarta.ee-spec.committee@xxxxxxxxxxx To unsubscribe from this list, visit https://urldefense.com/v3/__https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee__;!!ACWV5N9M2RV99hQ!fHraA8um0eKgNng_5TBlhok7ptQ-hZo0sO2H1FIv2Xg8a05iTGZzOJcdydYSpfc$
_______________________________________________ jakarta.ee-spec.committee mailing list jakarta.ee-spec.committee@xxxxxxxxxxx To unsubscribe from this list, visit https://urldefense.com/v3/__https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee__;!!ACWV5N9M2RV99hQ!YGPZOkbYZcXimcG6-E7MjUL-WNkbh0OKAcfF-RDF85SNaGxkTijadkXaN0BgpNU$_______________________________________________
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
_______________________________________________ jakartaee-tck-dev mailing list jakartaee-tck-dev@xxxxxxxxxxx To unsubscribe from this list, visit https://urldefense.com/v3/__https://www.eclipse.org/mailman/listinfo/jakartaee-tck-dev__;!!ACWV5N9M2RV99hQ!ZkiRhK4Tn1Q-AVALCYT4RxlLfj0AM7-VYw6sQsm7Dg36ZMAKG7PWCthQRUdEiyg$