Dmitry, The one problem
with this approach is how do we keep the two test buckets in sync? Is
there any way that we could have a single source repository for the testcases
themselves? And, they get cloned into the respective test bucket
is attempting to test with them? I fully understand that the test
harnesses and frameworks are different, but does that preclude the use
of common testcase source? Or, maybe there's some other way to ensure
that these do not diverge? --------------------------------------------------- Kevin Sutter STSM, MicroProfile and Jakarta EE architect @ IBM e-mail: sutter@xxxxxxxxxx Twitter: @kwsutter phone: tl-553-3620 (office), 507-253-3620 (office) LinkedIn: https://www.linkedin.com/in/kevinwsutter
Kornilov <dmitry.kornilov@xxxxxxxxxx> To:
developer discussions <jakartaee-platform-dev@xxxxxxxxxxx> Cc:
Sutter <sutter@xxxxxxxxxx> Date:
Re: [jakartaee-platform-dev] TCK tests in the same repo as API and Spec
We should consider job Andy’s done as
a PoC. It should be merged to JSONB repository and Yasson team should adopt
it without giving up TCK testing using CTS. TCK team should review it and
recommend it (or not recommend) it to other project as a template for Jakarta
EE 10 work. I don’t want a situation that splitting is artificially blocked
because no-one from TCK team wants to do it.
Andy’s effort doesn’t affect Jakarta
EE 9 release. As I was saying before, JSONB TCKs in CTS repository will
be used for certification. TCKs in JSONB repository can be used for preliminary
compatibility testing. It’s much easier to use, it works much faster and
it’s easier to create Jenkins jobs with these tests.
I think it's fine for people to do experiments,
ideally in their own personal forks so there's no confusion as to its relationship
with the official version.
What I don't want is for (e.g.) Andy to get it to the point that it's good
enough for him, he decides he's done and merges it in to the official repos
for his projects, and the rest is left for the rest of us to figure out.
I'm fine for not everything to be done before some things are merged, but
I want to see a plan that the jakartaee-tck project committers agree
to for how we're going to get to the desired end point.
Among other things, we need to know that the plan will get us to a working
platform tck in time for a corresponding platform release.
And it would be best if the plan didn't have us running both the "old"
and "new" TCKs in parallel for releases for some non-trivial
This is a big project, so we're going to have to make some compromises,
but that doesn't mean giving up on any sense of planning. An agile
approach is good, but we need confidence that we'll be able to see the
P.S. I'm also fine with adding lots more committers to the jakartaee-tck
project if that helps us make progress more quickly in the short term.
There should be few impediments to spec projects updating their TCK
tests within the existing framework. They can start with PRs against
the jakartaee-tck repo and become committers as needed.
Kevin Sutter wrote on 2/3/20 5:40 AM: Jan, You bring up some excellent points. Having a well thought out plan
before jumping in with both feet would be nice. But, on the other
hand, we don't want to stifle the excited effort that Andy is demonstrating.
It's a tough line. Maybe we need to allow one or two teams
to experiment with the processes and see what it takes. And, then
we can take those experiences and develop a plan that works across all
The discussion on the various email lists is quite extensive, and I admit I do not follow every thread that is discussed. But I am waiting
for the plan from any of the committee we have since the transition to
Eclipse Foundation. And I am not aware about any clear statement about
what the final goal is. For start:
- Is the splitting really what is recommended?
- Should the Platform Jakarta EE TCK still be available in the future after the split?
- Should the interoperability between Specs tests (the tests that require multiple Specifications to cooperate as it is described in one
or the other specification, the tests that used to be part of the full
CTS only) still be part of the combined (?) Jakarta EE TCK?
- Is there a recommended framework to be used in the future with the TCKs (such as the existing JavaTest + perhaps Arquillian similarly to Microprofile + ?)?
Those should be goals presented by the Jakarta representatives before the TCK repo is started to be ripped apart. Not on email, but clearly on
the main Jakarta informational page (whatever that is now).