Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jakartaee-tck-dev] Finalizing the TCKs 

On 27/07/20 8:51 pm, Ed Bratt wrote:
Due to how the Specification Committee has defined the process, any change to the TCK requires are re-release -- Docs. change, Exclude List change, anything else that would change content and/or generate a build forces a version number increment.
I think the procedures described below will allow us to stop builds, once a Specification has gone to ballot. So, that's my first concern. We just need to follow the ballot progress closely and not inadvertently create an update that wasn't asked for.
We can work on JDK11 based work, but I can't see how that could be accomplished without another Jakarta EE release cycle. The proposed plan with the highest likelihood (in my opinion, anyway) is to cycle the crank once again, just after Jakarta EE 9 is finished to re-ballot and release all the Specifications with whatever changes are needed for JDK11 modules etc.
I wonder if it would make sense to do the JDK11 work in a branch, then merge that all down after Jakarta EE 9 is finalized.
I suggest for any changes post JakartaEE 9 (including changes for JDK11 compatibility) be included in a new release with a micro version appended and following the same release cycle. For eg. If JSONP TCK require any change for jdk11 compatibility , we could follow the same procedure and release jakarta-jsonp-tck-2.0.1.zip. But can we not continue the jdk11 specific changes also in master branch? We don't need to wait till the Jakarta EE 9 is finalized for such changes.
Otherwise, for Jakarta EE 9 -- we need to get the TCK documentation files updated, we were behind on that last week and I've been encouraging the API teams to submit PRs for that work, but I don't know if we've caught up yet. Otherwise, as far as the tests themselves are concerned, I think they are running and passing using the GlassFish stand-alone test scenario. The API project teams need to confirm that the compatible implementation they are using is demonstrating that their implementation is also passing against the final TCK. (And, like above, in this case, the final TCK should be the final TCK built since the check-off item is the SHA-256 generated when the TCK is built.) 

Scott, did you have other concerns or issues that aren't yet covered?

-- Ed

On 7/27/2020 6:35 AM, Scott Marlow wrote:



On 7/27/20 7:22 AM, Alwin Joseph wrote:


On 27/07/20 4:39 pm, Lance Andersen wrote:
On Jul 26, 2020, at 9:05 AM, Scott Marlow <smarlow@xxxxxxxxxx <mailto:smarlow@xxxxxxxxxx>> wrote: 



On 7/24/20 3:18 PM, Ed Bratt wrote:
Just curious -- what is the plan for finalizing the stand-alone TCKs? I understand there is still some work going on with them -- documentation, maybe exclude lists, etc. But at some point, they need to be finalized. Certainly once a specification goes to Ballot, that TCK will need to be frozen -- even if other TCKs from this project still need work. Are the build systems set up to start curtailing the production of TCKs once the APIs go to ballot?
We have created a job https://ci.eclipse.org/jakartaee-tck/job/promote_jakartaee9_eftl_bundles/ to move the bundles from https://download.eclipse.org/ee4j/jakartaee-tck/jakartaee9-eftl/staged-900/ to https://download.eclipse.org/ee4j/jakartaee-tck/jakartaee9-eftl/promoted once we have finalized the TCKs. This job is manually triggered by selecting the required TCKs.
We could disable the TCKs from the job config once we promote a particular TCK. Could this suffice the requirement to freeze the TCK work.
Sure, we could rename each TCKs boolean build parameter after promotion, to ensure that we don't accidentally promote it again (if we do need to fix something after promotion, a https://ci.eclipse.org/jakartaee-tck/job/promote_jakartaee9_eftl_bundles change is needed to rename the build parameter back.
We should verify that a reference to a non-existinge Boolean build parameter defaults to false, not true. 

Scott
If we need to make further changes to the stand-along TCKs after an API has gone to ballot, what options do we have for further test changes? "Frozen" means no changes, so I think we could defer further test changes that could impact that (gone to ballot) API until the next TCK development cycle.  Are there other options?
I would expect if the testing finds issues that need to be fixed that these can be addressed as your only option is to exclude these tests.
For Java EE, we would address any late coming tests that we felt needed to fixed (sometimes due to configuration/platform issues), otherwise we would exclude.
Do we have any sense of the risk that a general problem might be found, as we close out Jakarta EE 9, that might force all the TCKs to be rebuilt after some of the ballots conclude?
We talked about keeping the TCK development process open for JDK11 test changes (e.g. for non-signature test failures that we might discover with other Jakarta EE 9 server implementations).  That is at risk.  A smaller risk, could be that fixing Platform TCK level failures could be impacted as well (assuming that some test failure need to be addressed in the Platform TCK, rather than working around said test failure in GlassFish 6.0).
Hmmm,  I would think until you stabilize the Jakarta EE 9 branch you would not allow updates outside of addressing  must fix issues.  Or am I mis-understanding the above?
I see that Alwin is still running tests this weekend (thank you Alwin again for covering for me this week!), lets see where we are tomorrow with the Stand-Alone TCKs and discuss further how to freeze each one to meet the below schedule.
If anyone has a different opinion on how we address the following schedule, please do speak up.  Personally, I see the point of following the below schedule so that we avoid delays that we might get, if we separately passed the TCKs after an API goes to ballot and the impact that has on other dependent APIs. 

Scott
As a reminder, these stand-alone TCKs are generated in this project (in wave sequence): 
 * Wave 0 (Any time)
     o Concurrency
     o Messaging
     o Persistence
     o (From the Platform TCK)
         + Web Services Metadata
 * Wave 1 (Any time)
     o Annotations
     o Expression Language
     o JSON Processing
     o Servlet
     o SOAP with Attachments
     o WebSocket
 * Wave 2 (Planned to start July 28)
     o Authentication
     o Authorization
     o JSON Binding
     o Server Pages
 * Wave 3 (Planned to start Aug 5)
     o XML Web Services
 * Wave 4 (Planned to start Aug 11)
     o RESTful Web Services
     o Transaction
 * Wave 5 (Planned to start Aug 18)
     o Connectors
     o Standard Tag Library
     o (From the Platform TCK)
         + Enterprise Beans
         + Enterprise Web Services
 * Wave 6 (Planned to start Aug 25
     o Security
     o Server Faces
 * Wave 7 (Planned to start Aug 31)
     o Jakarta EE Web Profile
     o Jakarta EE Platform
_______________________________________________
jakartaee-tck-dev mailing list
jakartaee-tck-dev@xxxxxxxxxxx <mailto:jakartaee-tck-dev@xxxxxxxxxxx>
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-tck-dev 

_______________________________________________
jakartaee-tck-dev mailing list
jakartaee-tck-dev@xxxxxxxxxxx <mailto:jakartaee-tck-dev@xxxxxxxxxxx>
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-tck-dev 

<http://oracle.com/us/design/oracle-email-sig-198324.gif>
<http://oracle.com/us/design/oracle-email-sig-198324.gif><http://oracle.com/us/design/oracle-email-sig-198324.gif> <http://oracle.com/us/design/oracle-email-sig-198324.gif>Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 
Oracle Java Engineering
1 Network Drive
Burlington, MA 01803
Lance.Andersen@xxxxxxxxxx <mailto:Lance.Andersen@xxxxxxxxxx>




_______________________________________________
jakartaee-tck-dev mailing list
jakartaee-tck-dev@xxxxxxxxxxx
To unsubscribe from this list, visithttps://www.eclipse.org/mailman/listinfo/jakartaee-tck-dev 

_______________________________________________
jakartaee-tck-dev mailing list
jakartaee-tck-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-tck-dev 



_______________________________________________
jakartaee-tck-dev mailing list
jakartaee-tck-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-tck-dev

Back to the top