| Re: [jakartaee-platform-dev] [External] : Re: Interceptors TCK -- what version of Platform TCK? |
On Thu, Sep 8, 2022 at 12:25 PM Scott Marlow <smarlow@xxxxxxxxxx> wrote:
During the Platform TCK call today, we talked about a few different options for refactoring what remains in the Platform TCK with different costs. Keeping in mind that the Platform TCK still has around 2.2 million lines of test code (based on `find -type f| xargs cat | wc -l` in src folder). Refactoring all of the Platform TCK tests has the highest risk that it could cause the EE 11 release to take longer than 1-2 years (IMO since we cannot validate any SPEC API Ballots until completing refactoring). If we refactor a small subset of Platform TCK tests, we would see a lower risk to the overall EE 11 release duration depending on how we do the refactoring.
Some options to discuss/consider:
- Completely refactor all remaining tests in the Platform TCK and take as much time as that takes. Cost is very high from an EE 11 release scheduling perspective as the Platform TCK project wouldn't be able to produce any standalone TCKs until the refactoring is complete, so the schedule impact is the first SPEC wave is blocked on starting the ballot until the Platform TCK refactoring is complete. Impact on EE11 release is that we have some initial overlap period where EE 11 SPECs are developed + TCK tests are refactored but the TCK refactoring does become a blocker for the EE 11 release starting the first SPEC ballot wave. Further cost is that we don't know how long the complete refactoring would take.
- Refactor a subset of each Platform TCK test bucket (e.g. specification being tested) but do not refactor all tests. The cost on the EE 11 implementations is that they still have to run both the old (Ant based Platform TCK tests) + new (refactored Platform TCK tests). To minimize the cost on EE 11 implementations, the Ant based Platform TCK tests would continue to be configured/run the same way as run with Platform TCK in EE 10. There is also a EE 11 implementors complexity cost of having some tests run via Ant and some tests run via Maven/Arquillian.
- Refactor a subset of each Platform TCK test bucket (e.g. specification being tested) but do not refactor all tests. The tests that are not refactored would be removed from the EE 11 release but could be refactored and added in future EE releases. The cost of reduced coverage varies depending on how much of each test bucket is refactored. The cost of risk to the EE 11 release also varies based on how much of each test bucket is refactored. The reduced test coverage could impact the quality of EE implementations.
4. Optimization of #2, Refactor a subset of each Platform TCK test bucket (e.g. specification being tested) but do not refactor all tests. The cost on the EE 11 implementations is that they still have to run both the old (Ant based Platform TCK tests) + new (refactored Platform TCK tests). To minimize the cost on EE 11 implementations, the tests not refactored would continue to be run via the Platform TCK.
I'm unclear on what #2 is then. My previous understanding was that it was to make the *Platform* TCK work like the Security, Authentication and Faces standalone TCKs do in EE 10. Basically two test executions, one using Arquillian-based tests, the other using the old Ant approach. That's not too bad for impls who already understand how to run the Ant platform TCK, as we'd just need to master the new way. Which presumably will be easy. ;)
#2 was a little ambiguous if `old` Platform tests are run with a
standalone TCK test running experience (but changed to work with
Platform TCK configuration settings + improved user doc) or the
actual current Platform TCK test running experience.
#4 clearly defines that the `old` TCK tests that aren't
refactored will continue to be run as part of the Platform TCK.
If instead it's about creating more individual spec TCKs that work like Security, Authentication and Faces, well, yuck. :) Getting porting kits set up for those 3 for EE 10 has been real challenge for the WildFly developers, who have a lot of expertise in this area. It would be really painful to have to do more like that, and I imagine it would be far worse for other impls with less expertise._______________________________________________
I think we all get the idea of how the first SPEC wave (and other SPEC API waves) are blocked on being able to run the relevant TCK. Perhaps we could be clever and come up with a solution for removing the SPEC API ballot bottleneck on the TCK with a contract that guarantees that any subsequent refactored TCK test changes must pass implementations that have submitted a compatibility certification request for the respective specification (e.g. think of a cooperative communications process that coordinates with the spec implementations that communicate their subsequent TCK test results back on a tracking Platform TCK issue). There could be other ways to improve our processes to allow SPEC API development to proceed forward in parallel while the TCK refactoring might still be happening throughout the EE 11 development process.
Scott
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
--
Brian StansberryPrincipal Architect, Red Hat JBoss EAPHe/Him/His
_______________________________________________ jakartaee-platform-dev mailing list jakartaee-platform-dev@xxxxxxxxxxx To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev