My opinion is: The GlassFish binary isn't what
needs to be preserved. That isn't being released. It is
just a mechanism for running the the TCK. The thing
being released is the API JAR. The tests will verify
that, when using a CI -- GlassFish 5.1, which is a
candidate compatible implementation now (verified by the
prospective Jakarta EE TCK in it's original form).
If the component teams had their own
customized TCK tasks, they could use those as well.
So -- you start with a CI -- in this case
Eclipse GlassFish 5.1.
Into that you generate a trial build, and
substitute the release candidate API JAR that you want
to validate for compatibility.
You run the TCK tests necessary to validate
that API JAR (either stand-alone TCK tests, or CTS TCK
if the component doesn't have a standalone TCK).
When you get a PASS result for the API JAR
from the TCK, you can make the compatible claim and you
have the test result to back it up.
For the cases of components with stand-alone
TCKs, the result is much more focused. With the CTS, you
will have a bunch of results that don't really matter,
to the component you are releasing, but that's just a
consequence of the test suite. If the end-result is
PASS, that component may be considered compatible.
Again, it is the API JAR that needs to be
preserved, along with the test results. That is done, by
virtue of staging it to OSSRH, then finally pushing it
to Maven Central. The trial build of Eclipse GlassFish
isn't released and need not be preserved.
Sure, other forms of testing might be more
robust, but we decided that we would allow for this kind
of "minimal" validation this time since we did not need,
nor did we want to go to the effort required to make
another release of Eclipse GlassFish. We could try to
preserve the has from GitHub so we could re-create the
build. We could preserve a hash of the build itself, but
I don't think this is required.
That's my take on this, anyway.
-- Ed
On 7/17/2019 11:25 AM, arjan
tijms wrote:
Hi,
The
issue I don’t see how to address is producing
a persistent version of the Glassfish binary
with persistent TCK results as we seem to be
requiring for spec release reviews.
As for the GlassFish binary, what
about providing the location GF was downloaded
from (and perhaps a hash of that binary?), and
if build from source for some reason, the git
revision?
For instance, I've been using the
following for the JASPIC TCK run:
And the hash (sha1)
"f63f451aa42a85e40a16766487809772ea547653"
Should we provide those details
for the results?
Kind regards,
Arjan
