James Roper wrote on 07/19/2018 12:31 AM:
Obviously we're not
interested in either extreme. We're looking for the right
cost/benefit tradeoff.
If we think that users are going to be running the TCK
against vendor's products because they don't believe the
product actually passed the TCK, we have a much bigger
problem.
The TCK is a very complex test suite (which
doesn't use Arquillian, by the way). Setting up and
running the TCK against someone else's product is a
difficult task, even given all the adapters that are
needed. It's far more likely that people will do it wrong
and make incorrect claims that the product doesn't pass
the TCK.
"The TCK" is actually not what we're talking about here.
Today, there may be only one TCK for Java/Jakarta EE, but in
future, and this document is specifying what should happen
in future, not the status quo that happens today (in fact
the intention is that this is an Eclipse wide specification
policy, not just Jakarta EE), the TCK should be split up,
and made to be a very simple test suite, to lower the
barrier of entry and encourage a more open platform that
anyone can contribute to. Now no one's suggesting that the
TCK be split up and rewritten tomorrow, as you said, it's
very complex, but I would hope over time that it heads in
this direction, that new iterations on specs would be
accompanied with pulling the TCKs for those specs out of the
main TCK, etc.
Splitting up the TCK, which is a worthy goal, is only part of the
problem. The setup necessary to run various parts of the TCK is
still complex. And anyone who needs to run the entire platform TCK
would probably prefer to be able to run "one" TCK instead of 37
individual TCKs.
I would love to see the community make significant improvements in
this area, but I doubt that it's ever going to be "simple".
And it's not correct to say that the TCK doesn't use
Arquillian - the CDI TCK, which is part of the TCK, does use
Arquillian. So we've already got the base DI mechanism for
Java EE that is heading in that direction.
Right, but the platform TCK, which is the bulk of the TCK, does not
use Aquillian.
The TCK adapter for a
proprietary product may make use of proprietary APIs that
the vendor does not want exposed.
All products must have a way for end users to deploy
things to a container that the vendor does expose, otherwise
the product is unusable. I don't see how a vendor can claim
that they don't want this exposed.
There's much more required than simply deploying an app. Maybe the
community will standardize everything that's required and we won't
need product-specific TCK adapters? That would be great, but that's
a lot of work.
Or the proprietary
product may come with a license that prohibits independent
testing of the product.
This is the sort of lack of transparency that the new
Jakarta EE should seek to prevent. I don't think it helps
Jakarta EE or the community.
It is a goal of Jakarta EE to allow independent implementations
under a license of their choice.
Or the product may
only be usable on hardware or in environments that most
users won't have access to.
That doesn't make sense - by definition, a user of the
software must have access to the hardware or environments
that it is usable on, otherwise they are not a user of it.
Right, but the user of the product may be the medical device
manufacturer that incorporates the Jakarta EE product into their
device. The Jakarta EE product may not be usable an a typical "PC".
I can imagine many
reasons a vendor might not want to make their TCK adapter
available to the public or available under an open source
license.
I can imagine many reasons a vendor might not want the
TCKs or specs themselves to be open source, but this is not
Java EE, this is Jakarta EE, it is an open project owned by
the Eclipse Foundation, a foundation that believes in open
collaboration and transparency.
Right, and what's done as a part of this project will be open and
transparent. I don't see that we need to limit the market to only
those vendors who agree that everything should be open and
transparent.
All that said, I'm
sure many vendors will make their TCK adapters
available, and will make their TCK results available, so
that others can see exactly how they've tested their
products. But I think there's good reasons that some
vendors may not want to do that, and I don't see the
benefit of requiring them to do it to be great
enough. The bar is already high enough to become a Java
EE / Jakarta EE vendor, we don't need to make it
unnecessarily higher.
Of course, if experience makes us suspect that vendors are
cheating, we can change the rules and require this. But
in 20 years of Java EE that hasn't been a problem.
We're
already making the bar orders of magnitude lower by making
the TCKs open, allowing vendors to run them themselves
without onerous legal argeements such as NDAs. TCKs can
(and in fact for new specifications do this already) be
run as part of a vendors CI, they get immediate feedback
the moment they break it. The bar is far, far lower than
it was before. But in lowering this bar, we've lost some
of the assurances that existed before, putting us in new
territory that Suns/Oracles 20 years of experience can't
speak for. Making the adaptors open is a simple way to get
that back.
I don't follow your logic for why this change in approach makes it
more likely that vendors will cheat. The official TCK will be
available under a special license, and that license can include many
of the assurances that have existed in the past.
Now perhaps there are existing Java
EE vendors that this is a problem for, and perhaps the
existing Java EE TCK is not practical for just any user to
run. But given that the scope of this document is not just
Jakarta EE, but all of Eclipse, I would rather that this
policy get it right, and then when applied to Jakarta EE,
the existing vendors be grandfathered in with an exception
that they don't have to implement this requirement, than to
have the Eclipse wide policy for TCKs not have this simple
mechanism for ensuring openness and transparency.
If you consider applying this policy to things beyond Jakarta EE,
such as IoT, I think it's even more important that vendors be given
the flexibility to choose the license that works for their business
model.
The TCK license should provide some of the assurances you're worried
about, but I don't see that it's necessary to require that it be
possible for any user to run the TCK against any vendor's product.
I think we should encourage that, and will obviously make it
true for any products developed under EE4J, but I don't think we
need to require it in all cases. And again, if this proves to be a
problem in practice, we can raise the bar later.
Anyway, it's time to hear from others. What do others on this list
think, especially other potential Jakarta EE vendors? Are you all
committed to providing your "TCK adapter" as open source software
and allowing users to run the TCK against your product? Do you
believe we should require every vendor to do so? Is there consensus
that this is necessary to achieve the goals of Jakarta EE?
|
