Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jakartaee-platform-dev] Individual specification TCK requirements for specs getting major/minor updates in EE 11


On 2/21/24 11:04, Scott Marlow wrote:


On 2/20/24 17:36, Brian Stansberry via jakartaee-platform-dev wrote:

There's been some discussion in the last week on a PR against the Jakarta EE TCK process guide[1] and on today's platform call[2] that needs discussion on this list.

The background is that last year there was a 'Requirements on standalone TCKs'
specification committee issue[3] that resulted in an update to the TCK process guide[4]. I think I can fairly correctly characterize the key motivation there as a (valid IMHO) desire to ensure that as new 'standalone' TCKs are created we don't lose the ability to test that EE component impls work correctly in an actual EE environment. 

We are coming up on release deadlines for various component specs, but I don't think the precise implications of the TCK process change are understood (for example, see the discussion at [5]). So my goal here is 1) to get more clarity, 2) to increase awareness of these requirements and 3) to work through any implications for completing the EE 11 release.

I'll frame all this in terms of questions. Some I'm pretty sure I know the answer, but maybe I'm wrong, or maybe others are less sure. Note that I'm going to refer a fair bit to things discussed in the links below, particularly the change included in [4].

1) What is a 'standalone TCK'?

This is a term for an individual specification's TCK that can be executed independent of the larger platform or profile TCK. This does not just mean a TCK for a 'standalone specification', i.e. a Jakarta spec like Jakarta MVC that is not part of any platform or profile.

I like Emily Jiang's use of 'individual specification TCK' in the [4] change, as it avoids the ambiguous meaning of 'standalone'.

2) Is this relevant to TCK service releases?

It's been clearly agreed that this is only relevant for TCKs that are producing updates for major or minor spec releases. Service releases are unaffected; service releases are not allowed to make these kinds of changes.

No.  I agree with the answers that TCK service releases do not need to be concerned with the TCK Process changes.


3) What are Type 1 tests?

I believe this refers to tests of things like 'CDI in Java SE'[6] -- that is, cases where specs have requirements that are only relevant when not running in an EE container. What matters is the type of spec requirement, not how a test has historically been executed.

This is a relatively small subset of all EE TCK tests.

4) How should Type 3 tests work?

My expectation is most tests are Type 3 tests.

The requirements in [4] state that an individual spec must provide a mechanism to run Type 3 tests within a platform/profile, but also must make it possible to run them outside of one. (I assume the latter is to avoid a circular dependency problem where a spec can't release because there is no platform/profile to test against, because the spec hasn't been released...)

In the past we have made exceptions to allow Components Specification to be released as final without testing on a platform/profile as type 3 tests create a chicken + egg problem of needing both a standalone implementation and platform/profile at the same time.  

If the Specification Committee needs to make an exception during the Component Specification review to ignore type 3 TCK tests since they cannot deploy on an EE 11 implementation yet, does that mean the Component Specification review cannot really happen until the last EE 11 wave?  The other question to answer is how to handle restarting the early EE 11 waves if any of the type 3 TCK tests are found to need code changes (note that we don't do service releases during an EE release for minor/major Component Specifications as per Specification Committee discussions in the past). 

FYI, https://www.eclipse.org/lists/el-dev/msg00190.html already indicates that it is easy to produce a type 3 _expression_ Language TCK since EL can just work in either just Java SE (no EE impl) or with an EE impl.  Note that as per https://jakarta.ee/specifications/platform/10/jakarta-platform-spec-10.0#a2161 _expression_ Language type 3 tests should be run from web (e.g. Servlet/Server Pages).  

Also note that https://jakarta.ee/specifications/_expression_-language/5.0/ mentions the implementation used as the initial _expression_ Language implementation (Eclipse Expressly 5.0.0-M2).  For _expression_ Language 6.0 what do people expect to appear under the "Compatible Implementations" section of the https://jakarta.ee/specifications/_expression_-language/6.0 page?  Just the _expression_ Language implementation?  Or details on the EE 11 Compatible Implementation as well?



Do we have any advice/best practices for how to do this? Anything we can extract from CDI, which handles this?

5) What about existing individual specification TCKs?

We've long had individual specification TCKs, and I'm sure some of what they include is Type 2 or 3 tests. Are we requiring that specs with those kinds of TCKs that are doing major/minor updates for EE 11 update those existing tests so they can run in a platform/profile impl?
I think one example is the Batch TCK if I recall correctly (I think the Batch TCK includes use of EJB).  I think Batch already meets the requirement.  But there are others that don't like the Persistence Spec (type 1) TCK which is currently in the Platform TCK.

From an individual specification point of view that doesn't have an EE implementation to test with, I'm not really sure of what the Specification Committee was thinking when they made the TCK Process change with regard to actual specification review.  Perhaps the individual specification team will hack together a milestone release of some slightly modified open source EE implementation that has some EE 11 specifications but I think this is a dangerous path to suddenly require individual specification teams to adapt to for the EE 11 release as the hacked together implementation will not really prove that the component specification can be used on EE 11 platform/profile implementations.


6) What new individual specification TCKs are we expecting?

I know we are refactoring the platform tck to try and move toward individual specification TCKs, but I haven't followed the details on where this is going.

From the TCK Refactor branch, we can look at the install folder and see the following individual TCKs:

For EE 11, not all of the TCK refactoring of the individual TCKs listed below need to be completed so I will note inline which TCK is needed for EE 11 to make the list easier to read:

1. https://github.com/jakartaee/platform-tck/tree/tckrefactor/install/caj Common Annotations

Annotations 3.0 (part of Wave 1 with review Feb 29)

2. https://github.com/jakartaee/platform-tck/tree/tckrefactor/install/connector Connector/JCA
3. https://github.com/jakartaee/platform-tck/tree/tckrefactor/install/el _expression_ Language

_expression_ Language 6.0 (part of Wave 2 with review Feb 29)

4. https://github.com/jakartaee/platform-tck/tree/tckrefactor/install/jacc Authorization
5. https://github.com/jakartaee/platform-tck/tree/tckrefactor/install/jaxws to be removed since jaxws is optional?
6. https://github.com/jakartaee/platform-tck/tree/tckrefactor/install/jms Messaging
7. https://github.com/jakartaee/platform-tck/tree/tckrefactor/install/jpa Persistence

Persistence 3.2 (part of Wave 5 with review March 29)

8. https://github.com/jakartaee/platform-tck/tree/tckrefactor/install/jsp Server Pages

Server Pages 4.0 (part of Wave 5 with review March 29)

9. https://github.com/jakartaee/platform-tck/tree/tckrefactor/install/jstl Standard Tag Library
10. https://github.com/jakartaee/platform-tck/tree/tckrefactor/install/jta Transactions
11. https://github.com/jakartaee/platform-tck/tree/tckrefactor/install/saaj Soap with Attachments
12. https://github.com/jakartaee/platform-tck/tree/tckrefactor/install/websocket WebSocket

WebSocket 2.2 (part of Wave 5 with review March 29)

Scott

Note that https://github.com/jakartaee/platform-tck/tree/tckrefactor/install/servlet is already migrated to the Servlet Spec.


7) Are the spec groups producing major/minor releases aware of these requirements? Where do things stand?

No for _expression_ Language as per https://www.eclipse.org/lists/el-dev/msg00190.html.  I'm not sure of why that is.



My guess is in most cases the answer to the first question is 'No'.

+1 



_______________________________________________
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev

Back to the top