| Re: [jakarta.ee-spec.committee] Requirements for individual specification TCKs |
Hi Scott,
Since the JSON-B and JSON-P TCKs were in platform TCKs in the previous releases but they were deleted for EE10, could the deletion be reverted? I know this means the duplicated tests, but it is better than nothing. Before a new framework is provided, we might have to live with the duplication.
We already have the signature testing of JSON-B
+ JSON-P
SPEC APIs tested in the various EE containers which ensures that
the relevant EE containers are exposing the correct dependencies
to applications. Also the JSON-P/B SPEC API teams discussed this
before and stated that it wouldn't add any value to run the
JSON-B/P TCK tests from EE containers.
Scott
Thanks
Emily
================
Emily Jiang
Java Champion
STSM, Jakarta and MicroProfile Architect @IBMLiberty Cloud Native Architect & Advocate
IBM United Kingdom Limited
Registered in England and Wales with number 741598
Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU
From: jakarta.ee-spec.committee <jakarta.ee-spec.committee-bounces@xxxxxxxxxxx> on behalf of Scott Stark <sstark@xxxxxxxxxx>
Sent: 07 April 2022 14:37
To: Jakarta specification committee <jakarta.ee-spec.committee@xxxxxxxxxxx>
Subject: [EXTERNAL] Re: [jakarta.ee-spec.committee] Requirements for individual specification TCKsThis Message Is From an External SenderThis message came from outside your organization.It was discussed in platform dev calls and a special TCK call to determine how to handle the newly extracted TCKs.
It could potentially fall under the spec committee if it was an update to the TCK process doc that described requirements for moving TCKs out of the platform:https://jakarta.ee/committees/specification/tckprocess/
The guidelines are not actionable in my view because there needs to be a common framework to implement them. When you say an individual specification must provide a profile ready mechanism for running the TCK tests, that has to be defined sufficiently such that it is implemented consistently across specification and can be used in the profile TCKs.
Further, the assumption that individual specifications define profile level integration tests is debatable. It is not desirable from the perspective of introducing circular dependencies on downstream specification APIs. It is also not desirable from the perspective that profiles are free to override the individual specifications.
Red Hat initially wanted to block on the technical debt issue around TCKs for EE10, but it was seen as too big a problem to address given that no functional update had been provided to EE for years. Until this is seen as a priority and a blocking feature for a release, I don't see how it can be addressed.
On Apr 7, 2022 at 8:13:41 AM, Thomas Watson <tjwatson@xxxxxxxxxx> wrote:
Was this approach agreed upon by the specification committee? Is that even something the specification committee is responsible for deciding if it is the right approach? I admit that I may be missing some past context where a tactical decision was made to expediate an EE 10 release. Perhaps it is the best we can do for EE 10 with the timeline and the number of developers available to improve the situation.
Instead of focusing on what we can and cannot do for EE 10 I want to circle back to the potential proposal I laid out with this thread. Is there agreement that we have a gap in the current approach being taken? Would having requirements in place that I suggest help close the TCK gap to ensure that the EE platform and other profiles compatible implementations do indeed function properly for all aspects of the specification?
Tom
From: jakarta.ee-spec.committee <jakarta.ee-spec.committee-bounces@xxxxxxxxxxx> on behalf of Scott Stark <sstark@xxxxxxxxxx>
Sent: Wednesday, April 6, 2022 5:08 PM
To: Jakarta specification committee <jakarta.ee-spec.committee@xxxxxxxxxxx>
Subject: [EXTERNAL] Re: [jakarta.ee-spec.committee] Requirements for individual specification TCKsThis Message Is From an External SenderThis message came from outside your organization.What we are doing for EE10 is to duplicate tests that exercise the expected integration points that are most likely needed in the various EE containers. No one was willing to make test reuse a blocking requirement for EE10, so that is the best we can do.
On Wed, Apr 6, 2022 at 4:25 PM Emily Jiang <EMIJIANG@xxxxxxxxxx> wrote:
Hi Scott,
> 1. Run the standalone SE based TCK against the implementation jar. The JSONP TCK docs describe how to do this by setting the GAV maven coordinates of the vendor implementation.
In this case, the tests are SE only and they can't be executed in an EE container. How can we ensure to the customers that JSON-P behaves as specified without tests backed up in that context?
e.g. a snake swims well in cold water (e.g. SE). It might not function well in warm water (e.g. EE).
I totally agree with what David said: This is why we run the same test several times, once as a Servlet, again as a JSP, again as an EJB, etc. To go from that to the extreme of it not being testable in a container at all is too far of a departure IMO.
I think it is ok to run some tests in SE container, but these tests should also be executable in EE if the functionalities are applicable to EE.
Thanks
Emily
================
Emily Jiang
Java Champion
STSM, Jakarta and MicroProfile Architect @IBMLiberty Cloud Native Architect & Advocate
IBM United Kingdom Limited
Registered in England and Wales with number 741598
Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU
From: jakarta.ee-spec.committee <jakarta.ee-spec.committee-bounces@xxxxxxxxxxx> on behalf of Scott Stark <sstark@xxxxxxxxxx>
Sent: 06 April 2022 21:46
To: Jakarta specification committee <jakarta.ee-spec.committee@xxxxxxxxxxx>
Subject: [EXTERNAL] Re: [jakarta.ee-spec.committee] Requirements for individual specification TCKsThis Message Is From an External SenderThis message came from outside your organization.Taking JSONP as an example, for EE10 the discussion we had was that the requirements would be:
1. Run the standalone SE based TCK against the implementation jar. The JSONP TCK docs describe how to do this by setting the GAV maven coordinates of the vendor implementation.2. The platform TCK would keep the integration only tests to validate basic container level integration.
On Wed, Apr 6, 2022 at 2:10 PM Thomas Watson <tjwatson@xxxxxxxxxx> wrote:
For EE 10 we have several new individual TCKs. I would like to understand if we established any general policies for how such TCKs are executed. I'm asking for a general discussion about such a policy. But if it helps to have some examples I am referring to the TCKs for the individual specifications JSON Binding, JSON Processing and XML Binding. My understanding of these three individual TCKs is that they are only able to be run directly against Java SE.Tom
From: Scott Marlow <smarlow@xxxxxxxxxx>
Sent: Wednesday, April 6, 2022 12:23 PM
To: Jakarta specification committee <jakarta.ee-spec.committee@xxxxxxxxxxx>; Thomas Watson <tjwatson@xxxxxxxxxx>
Subject: [EXTERNAL] Re: [jakarta.ee-spec.committee] Requirements for individual specification TCKsThis Message Is From an External SenderThis message came from outside your organization.Is there a specific individual Jakarta EE specification TCK that you are referring to? Perhaps one of the new for EE 10 TCKs?
Scott
On 4/5/22 10:24 AM, Thomas Watson wrote:
Hi all,
It is not clear that the specification committee has discussed and decided on the requirements for individual specification TCK execution (aka standalone TCK). Recent discussions about individual specification TCKs have identified an exposure for a platform implementation to claim compliance for individual specifications that are included in the platform, web profile or core profile. A TCK must be executable within the context of the profile it is included in: platform, web profile or core profile. Otherwise, there is no confirmation that the compliant platform, web profile or core profile implementation functions properly with respect to the individual specification TCK.
Some individual specification TCK teams have indicated the desire to develop TCKs that are only executable outside of a profile implementation and only run directly on Java SE. It is acceptable for specification projects to develop a TCK that can run directly on Java SE. It is not acceptable for that complete TCK to only be executable directly on Java SE.
A TCK can have three types of tests:
- Tests that only apply when running outside of a platform (or profile), such as on Java SE, to test behavior defined by the specification for that mode of execution.
- Tests that only apply when running in a platform (or profile) to test behavior defined by the specification for that mode of execution.
- Tests that apply to the behavior defined by the specification for all modes of execution.
The tests of type 2 and 3 must be able to be executed in the context of the profile they are included in: platform, web profile or core profile.
We want to discuss proposing the following requirements for individual specification TCKs within the specification committee:
- The individual specification TCK must provide a profile-ready mechanism for running the required TCK tests (types 2 and 3) in the profiles they are included in: platform, web profile or core profile. Type 1 tests must be excluded and not required to be executed.
- The individual specification TCK must provide a mechanism for running the required TCK tests (types 1 and 3) outside the context of a platform or profile such as on Java SE. Type 2 tests must be excluded and not run.
- It is important that type 3 tests be run in compatible implementations of the profiles to validate the individual specification implementation functions properly in the profile implementation: platform, web profile or core profile.
Note that we avoided using the term "standalone TCK" because to some the term "standalone" means TCKs that run outside of any platform or profile container.
Thomas Watson
_______________________________________________ jakarta.ee-spec.committee mailing list jakarta.ee-spec.committee@xxxxxxxxxxx To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee
_______________________________________________ jakarta.ee-spec.committee mailing list jakarta.ee-spec.committee@xxxxxxxxxxx To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee