Andy’s effort doesn’t affect Jakarta EE 9 release. As I was saying
before, JSONB TCKs in CTS repository will be used for certification.
TCKs in JSONB repository can be used for preliminary compatibility
testing. It’s much easier to use, it works much faster and it’s easier
to create Jenkins jobs with these tests.
I worry that the TCKs will be out of sync. For an instance, someone adds tests easily to the local version of TCKs. He/she needs to remember to add TCKs to CTS version. I think this duplicate efforts. I think the splitting JSONB tcks needs to be done by the time of releasing JSON-B as part of Jakarta EE9.
@Jan,
How the concept of the deployment would work with
Andy's approach is not clear. Should the deployment be tested in the
future? Should it be left on the Spec to define the deployment
requirements/API or should it more common to every Spec? Currently, the
TCK framework does that and vendors implement the porting package that
takes care of the deployment. Every TCK uses the same mechanism. But
with having a different test framework in each TCK, the deployment needs
to be thought through.
I don't think the goal of breaking TCKs from the big bucket is to use a different test framework. I envisage the same test framework should be retained at least for the first release. I don't think using different test framework is something we should ban either. We should allow different specs to use a suitable test framework if there is strong benefit associated with the new test ramework.
If the tests should still
be part of the platform TCK, and the tests are in the Spec repository,
how to make sure the tests are run for full platform and not for the
standalone TCK?
There is a good use case for breaking up the TCKs into standalone and platform as some runtime might only be interested in some standalone specs instead of the full platform. This will help to push individually specs to be certified. For instance, the test group in Arquillian can be adopted to dvide the tests into different buckets.
Even the Microprofile has a unified Arquillian based test environment.
Using JUnit in one TCK, Arquillian in other, and JavaTest in the next
brings a challenge for the team that ought to run the TCKs. Are we cool
with having it different in every TCK?
In MicroProfile, you were right that Arquillian is a widely adopted technology. I think at the moment, Jakarta EE adopts different test frameworks and we can live with it. However, for any new specs, I think we should strongly recommend to adopt the mainstream test framework, such as Arquillian etc. However, I think this is orthorgno to the discussion of rehoming TCKs.
We should consider job Andy’s done as a PoC. It should be merged to JSONB repository and Yasson team should adopt it without giving up TCK testing using CTS. TCK team should review it and recommend it (or not recommend) it to other project as a template for Jakarta EE 10 work. I don’t want a situation that splitting is artificially blocked because no-one from TCK team wants to do it.
Andy’s effort doesn’t affect Jakarta EE 9 release. As I was saying before, JSONB TCKs in CTS repository will be used for certification. TCKs in JSONB repository can be used for preliminary compatibility testing. It’s much easier to use, it works much faster and it’s easier to create Jenkins jobs with these tests.
I think it's fine for people to do experiments, ideally in their own
personal forks so there's no confusion as to its relationship with
the official version.
What I don't want is for (e.g.) Andy to get it to the point that
it's good enough for him, he decides he's done and merges it in to
the official repos for his projects, and the rest is left for the
rest of us to figure out.
I'm fine for not everything to be done before some things are
merged, but I want to see a plan that the jakartaee-tck
project committers agree to for how we're going to get to the
desired end point.
Among other things, we need to know that the plan will get us to a
working platform tck in time for a corresponding platform release.
And it would be best if the plan didn't have us running both the
"old" and "new" TCKs in parallel for releases for some non-trivial
time.
This is a big project, so we're going to have to make some
compromises, but that doesn't mean giving up on any sense of
planning. An agile approach is good, but we need confidence that
we'll be able to see the endpoint.
P.S. I'm also fine with adding lots more committers to the
jakartaee-tck project if that helps us make progress more quickly in
the short term. There should be few impediments to spec projects
updating their TCK tests within the existing framework. They can
start with PRs against the jakartaee-tck repo and become committers
as needed.
Kevin Sutter wrote on 2/3/20 5:40 AM:
Jan, You bring up
some
excellent points. Having a well thought out plan before jumping
in
with both feet would be nice. But, on the other hand, we don't
want
to stifle the excited effort that Andy is demonstrating. It's a
tough
line. Maybe we need to allow one or two teams to experiment
with
the processes and see what it takes. And, then we can take
those
experiences and develop a plan that works across all the
projects.
---------------------------------------------------
Kevin Sutter
STSM, MicroProfile and Jakarta EE architect @ IBM
e-mail: sutter@xxxxxxxxxx Twitter: @kwsutter
phone: tl-553-3620 (office), 507-253-3620 (office)
LinkedIn: https://www.linkedin.com/in/kevinwsutter
The discussion on the various
email
lists is quite extensive, and I
admit I do not follow every thread that is discussed. But I am
waiting
for the plan from any of the committee we have since the
transition to
Eclipse Foundation. And I am not aware about any clear
statement about
what the final goal is. For start:
- Is the splitting really what is recommended?
- Should the Platform Jakarta EE TCK still be available in the
future
after the split?
- Should the interoperability between Specs tests (the tests
that
require multiple Specifications to cooperate as it is
described in one
or the other specification, the tests that used to be part of
the full
CTS only) still be part of the combined (?) Jakarta EE TCK?
- Is there a recommended framework to be used in the future
with the
TCKs (such as the existing JavaTest + perhaps Arquillian
similarly to
Microprofile + ?)?
Those should be goals presented by the Jakarta representatives
before
the TCK repo is started to be ripped apart. Not on email, but
clearly on
the main Jakarta informational page (whatever that is now).
-- Jan
On 31.01.2020 20:34, Lance Andersen wrote:
> I also agree with Bill. There has to be thought given
to CTS
> structure first IMHO so that there is consistency in the
test
> structure to make it easier to pull in the standalone
tests and
> where the platform specific tests are maintained for the
standalone
> technologies.
>
> This will take some time to flush this out. During the
Java
EE days,
> we did find it easier to work out of one master
workspace, but part
of
> that was due to the fact there was one team primarily
responsible
for
> the test development for all of the Java EE technologies.
>
>> On Jan 31, 2020, at 2:01 PM,
dmitry.kornilov@xxxxxxxxxx
>> <mailto:dmitry.kornilov@xxxxxxxxxx>
wrote:
>>
>> Bill, you are right. On the other hand the process of
splitting
has
>> to be started somehow. It will never start with an
assumption
that
>> some developer with TCK knowledge will take an
initiative and
start
>> working on it. There are not too many developers like
this and
no one
>> wants to take a risk and responsibility.
>> On the other hand I don’t think that it’s a right
time to do
it now
>> as part of Jakarta EE 9 release. I already proposed
thatwe will
not
>> move JSONB TCK tests from CTS yet. We will do it
after Jakarta
EE 9
>> is released. We will keep them in sync until that
time. Users
may use
>> TCK tests in JSONB repo as more convenient way of
checking
>> compatibility. But it cannot be used officially for
compliance
>> testing for now.
>> -- Dmitry
>> *From:*jakartaee-platform-dev-bounces@xxxxxxxxxxx
>> <mailto:jakartaee-platform-dev-bounces@xxxxxxxxxxx><jakartaee-platform-dev-bounces@xxxxxxxxxxx
>> <mailto:jakartaee-platform-dev-bounces@xxxxxxxxxxx>>*On
Behalf
>> Of*Bill Shannon
>> *Sent:*Friday, January 31, 2020 7:06 PM
>> *To:*jakartaee-platform developer discussions
>> <jakartaee-platform-dev@xxxxxxxxxxx
>> <mailto:jakartaee-platform-dev@xxxxxxxxxxx>>;
Andy Guibert
>> <andy.guibert@xxxxxxxxx <mailto:andy.guibert@xxxxxxxxx>>
>> *Subject:*Re: [jakartaee-platform-dev] TCK tests in
the same repo
as
>> API and Spec
>>
>> This has been discussed numerous times over the last
3 years.
No one
>> disagrees with the goal of splitting up the TCK
repo. Whether
the
>> individual spec TCKs are in the same repo as the API
classes or
in a
>> different repo is just a detail. The challenge is in
splitting
up
>> the existing TCK repo such that the resulting TCK for
each spec
is
>> functionally identical to the existing standalone
TCKs for the
specs,
>> and that the platform TCK is somehow created by
combining all
the
>> individual TCKs to produce a new platform TCK that is
functionally
>> identical to the existing platform TCK.
>>
>> Needless to say, this is not a small job.
>>
>> No one with deep knowledge of the existing TCKs has
come forward
with
>> a detailed plan for how to achieve the above.
Without that,
we're
>> all just wishing and hoping.
>>
>> And I strongly encourage you to*not*just start
hacking on the
TCK to
>> create what you want for your spec. We need to solve
the
larger
>> problem and a bunch of uncoordinated hacks to
individual TCKs
will
>> not get us there.
>>
>> Thanks.
>>
>> P.S. You should be able to find some of the previous
history
of this
>> discussion in the jakartaee-tck-dev mailing list.
>>
>> Andy Guibert wrote on 1/30/20 10:45 AM:
>>> Hi all,
>>> Currently all of the Jakarta EE TCK tests are
housed in one
big repo
>>> and they use a custom test framework from the
Java EE days:
>>> https://github.com/eclipse-ee4j/jakartaee-tck
>>> It is more convenient to have the TCK tests in
the same
repo as the
>>> API and spec docs because as the technologies
change over
time all 3
>>> parts (tck/api/spec) can be updated in the same
PR. Additionally,
>>> implementations can then consume the TCK tests as
Maven artifacts
>>> and run/verify that they pass the TCK. This is
what MicroProfile
has
>>> done and it works very well.
>>> As an example, I've started this off with the
JSON-B TCK test
here:
>>> jsonb-api pr: https://github.com/eclipse-ee4j/jsonb-api/pull/221
>>> yasson (impl) pr: https://github.com/eclipse-ee4j/yasson/pull/379
>>> Just wanted to share this with the wider dev
community and
encourage
>>> other specs to follow suit as time permits.
>>> Cheers, Andy
>>> -- IBM
>>>
>>>
>>> _______________________________________________
>>> jakartaee-platform-dev mailing list
>>> jakartaee-platform-dev@xxxxxxxxxxx <mailto:jakartaee-platform-dev@xxxxxxxxxxx>
>>> To change your delivery options, retrieve your
password, or
unsubscribe from this list, visit
>>> https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
>> _______________________________________________
>> jakartaee-platform-dev mailing list
>> jakartaee-platform-dev@xxxxxxxxxxx
>> <mailto:jakartaee-platform-dev@xxxxxxxxxxx>
>> To change your delivery options, retrieve your
password, or
>> unsubscribe from this list, visit
>> https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
>
> <http://oracle.com/us/design/oracle-email-sig-198324.gif>
> <http://oracle.com/us/design/oracle-email-sig-198324.gif><http://oracle.com/us/design/oracle-email-sig-198324.gif>
> <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance
> Andersen| Principal Member of Technical Staff |
+1.781.442.2037
> Oracle Java Engineering
> 1 Network Drive
> Burlington, MA 01803
> Lance.Andersen@xxxxxxxxxx <mailto:Lance.Andersen@xxxxxxxxxx>
>
>
>
>
> _______________________________________________
> jakartaee-platform-dev mailing list
> jakartaee-platform-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password,
or unsubscribe
from this list, visit
> https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
_______________________________________________
jakartaee-platform-dev mailing list jakartaee-platform-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe
from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
