Re: [jakartaee-platform-dev] [EXTERNAL] Re: TCK tests in the same repo a

[Bill Shannon <bill.shannon@xxxxxxxxxx>]

Andy Guibert wrote on 3/4/20 2:13 PM:
> Bill, I had a sit down meeting with the CTS lead at IBM and we discussed the
> pros and cons of the current way and the "externalized" way that I have
> proposed. Ultimately, he said that we already have both ways working and
> automated and it didn't make a big difference which way each spec goes.
I don't understand.  How do we have both ways working?  Do you mean the old TCK
and the new work you've done?

> I see two types of stakeholders with the TCK tests:
>  1) People who work on moving specs forward and writing new TCK tests,
> including getting tests passing for new versions of specs
>  2) People who run/maintain the TCK tests long term to ensure things don't regress
Aren't you forgetting the people who run the tests to check conformance of a
product?

> Based on the opinions I've gathered from people in role (2) in this thread and
> from direct discussions at IBM, they don't care one way or another.
> Based on the opinions I've gathered from people in role (1), they prefer the
> "externalized" way of testing. Furthermore, this decision of what type of
> testing to go with can be made at a per-spec level so it doesn't need to be a
> unanimous platform decision. For people in role (1) for JSON-B, it is
> unanimous that we want to migrate to the "externalized" TCK way, and we have
> already put in the work to do so.
>
> If anyone from other companies have concrete concerns with this approach
> (Oracle or others) I think it would be best to set up a Webex with the
> concerned parties so we can talk it over and report back to this thread.
My impression is that everyone wants "externalized".  Did you actually find
people who don't want that?  Or maybe I don't understand what you mean by
"externalized"?

I think we're just all waiting for more documentation that tells us how to get
from here to there, and what it will look like once we get there, especially for
the people who have to run the tests to test conformance of their product.

I would call this a "project plan", but maybe that's not a term you're familiar
with.