| Re: [jakartaee-platform-dev] : Re: Concurrency TCK without remote EJB, javax.rmi.RemoteException |
On 7/9/22 5:06 AM, Emily Jiang wrote:
Hi Scott,I don’t think your comments addressed my concerns. My concern is whether the skipped tests leaves a hole in the Concurrency spec as some speced behaviors might be left untested for web profile.
We do still test EJB with the Concurrency SPEC TCK with both the web profile + full platform. So web profile implementations must still integrate EJB + Concurrency technologies in general. In terms of the specifics of which Concurrency API is not currently covered by the Concurrency TCK when run with EJB + web profile (or full platform), we do not have that information currently but anyone in the community can grep through the TCK tests for each Concurrency SPEC API method to see if we missed testing any difficult to implement Concurrency SPEC API methods. Any such Concurrency SPEC TCK methods that we didn't yet cover with TCK testing, can be addressed by anyone in the community that wants to via a pull request.
There are many potential future goals for TCK tests that are not currently considered as required (at least not historically up to this point). One of the future TCK goals could be improving SPEC API testing code coverage.
There are many other possible future goals. The current focus is on refactoring Platform TCKs and we will continue that so that others can start contributing pull requests without needing to understand the many Platform TCK internals. The TCK refactoring will also help us to improve the overall development process for new EE releases.
Emily, thanks for asking hard questions, it is good to discuss these things. We all cannot agree on everything but it is great to have the discussion in the open!
Scott
Though EJB is no longer strategic, I think tests are still needed if the integration with EJB is speced unless we remove that from the spec.As for ear deployments, my question is whether the tests can still be relevant for war deployment. If yes, the fix should be converted to use war instead.
ThanksEmily
Sent from my iPhone
On 8 Jul 2022, at 19:26, Scott Marlow <smarlow@xxxxxxxxxx> wrote:
On 7/7/22 5:32 PM, Emily Jiang via jakartaee-platform-dev wrote:
What about the 21 EAR deployment tests? Do they just test the ear flavour or they happen to use .ear to test other functions? In this case, can these tests be updated to use .war? I am a bit concerned as there are 21 tests involved. By the way, it seems 49 less tests for web profile than platform for concurrency TCK based on Scott Stark's test result. From Scott Marlow's comment earlier, 30 tests (21+9) should be affected. Why are the other 19 tests missing? Are they just Platform only?
The jakarta.ejb.Remote class will no longer be used in the Concurrency TCK but jakarta.ejb.Remote is still referenced in around 260 Platform TCK test sources, so no loss.
Scott
ThanksEmily
On Thu, Jul 7, 2022 at 10:08 PM arjan tijms <arjan.tijms@xxxxxxxxx> wrote:
_______________________________________________Hi,
On Thu, Jul 7, 2022 at 7:13 PM Tracy Burroughs <tkb@xxxxxxxxxx> wrote:
Note that local EJBs are part of Web Profile, so if the concurrency tests are really intending to test with EJBs in general, then excluding the tests is leaving a gap for something that should work in Web Profile.
Although it's perhaps another discussion, if there were such a gap it might not be such a big problem, IMHO at least. The Web Profile specifically (and Concurrency in general) should probably deemphase EJB and instead emphasize CDI.
Kind regards,Arjan Tijms
_______________________________________________On the other hand, if there are tests for both local and remote EJB, then it would be appropriate to exclude just the remote ones.
Tracy Burroughs (tkb@xxxxxxxxxx)
WebSphere Application Server Development
IBM Rochester, Dept AAW, Bldg H315/050-2
2800 37th Street NW, Rochester MN 55901-4441
From: jakartaee-platform-dev <jakartaee-platform-dev-bounces@xxxxxxxxxxx> On Behalf Of Emily Jiang via jakartaee-platform-dev
Sent: Thursday, July 7, 2022 9:35 AM
To: jakartaee-platform developer discussions <jakartaee-platform-dev@xxxxxxxxxxx>
Cc: Emily Jiang <emijiang6@xxxxxxxxxxxxxx>
Subject: Re: [jakartaee-platform-dev] [External] : Re: Concurrency TCK without remote EJB, javax.rmi.RemoteException
Personally I think before the Platform and Web Profile release, we should have more freedom on what to put on service release. After the platform and web profile release, we should be very careful with the service releases. In this PR for concurrency,
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
ZjQcmQRYFpfptBannerEnd
Personally I think before the Platform and Web Profile release, we should have more freedom on what to put on service release. After the platform and web profile release, we should be very careful with the service releases.
In this PR for concurrency, it looks like Scott just excluded the tests for Web Profile. My question is whether the tests excluded test any functions that should work in a web profile. With the deletion, do we have a hole in the specification verification for the web profile (e.g. some functions untested)?
Thanks
Emily
On Thu, Jul 7, 2022 at 6:24 AM Scott Stark <starksm64@xxxxxxxxx> wrote:
The only certification request for concurrency to date is Open Liberty, and Brian said this was done with the full platform version:
There has been no web profile implementation pass the tck. Given that state, a 3.0.1 release with the last PR that GlassFish passes with both platform and web profile could be used for any full profile that wishes to use it, and all web profile implementations. If Open Liberty wants to keep certification of their full profile at 3.0.0, that is fine.
On Jul 6, 2022 at 12:10:58 PM, Ed Bratt <ed.bratt@xxxxxxxxxx> wrote:
I believe this case isn't covered in our process documentation (at least that I am aware of). In the Java EE days, a Fix would only be allowed by introduction of an alternate test. The rationale being that, some vendor might have passed the TCK using the original test. By following the 'alternate' path, if a vendor had passed the original test (even if they had not submitted results yet), they could continue doing so without impact to their previous work. New (as well as existing) vendors could select from the original and alternates at their discretion.
If I recall, we decided to simply forego the discussion and inclusion of the process describing alternate tests until a later date since that wasn't needed, when the process was originally codified under the JESP. Now might be a good time to consider adding that to the process text.
The key requirement, in my opinion being that we do not know where other vendors are in their certification process and we do not want to invalidate work done using a previously released artifact. Even if it's just an 'implementation detail' I don't know that we can be sure that such a change would not impose rework on a vendor product.
From what I have read on this, I would recommend this be added as an alternate test. The original and alternate tests should be resolved in a later feature release of the TCK/Specification.
I will discuss the issue of alternate tests with the Specification Committee separately.
Scott, if alternate was necessary, do you know how you would go about that? (I'd think a property in the TCK config/setup would be easiest but I'm not the one writing the code.)
-- Ed
On 7/6/2022 9:37 AM, Scott Stark wrote:
We discussed the current situation in today's platform TCK call with the proposed TCK PR that both removes all usage of jakarta.ejb.Remote and EARs and how that is problematic because it introduces failures in the full platform run. It was pointed out that perhaps simply excluding tests that won't run on the web profile could work. I had thought this could potentially exclude all tests, so I just looked at adding a test group to any test using an EAR deployment, and turned all remote ejb interfaces into simple interfaces. This does allow the tck to run and pass on GlassFish full platform and web profile:
Full platform:
===============================================
jakarta-concurrency
Total tests run: 149, Passes: 149, Failures: 0, Skips: 0
===============================================
[INFO] Tests run: 149, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 288.889 s - in TestSuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 149, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
Web profile with eefull group excluded:
===============================================
jakarta-concurrency
Total tests run: 100, Passes: 100, Failures: 0, Skips: 0
===============================================
[INFO] Tests run: 100, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 211.601 s - in TestSuite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 100, Failures: 0, Errors: 0, Skipped: 0
While a non-trivial number of tests are excluded, it is not even the majority of tests, so perhaps that is an acceptable alternative for EE10. There is a PR for review on this set of changes to the concurrency TCK here:
https://github.com/jakartaee/concurrency/pull/250
On Jun 29, 2022 at 11:59:53 AM, Steve Millidge (Payara) <steve.millidge@xxxxxxxxxxx> wrote:
Thanks Scott.
Personally I think you should be able to fix tests, where the test is buggy, as long as the post-conditions aren’t tightened. At the end of the day if someone is affected by a change and they fail a TCK service release which they previously passed they can ultimately raise another challenge on the service release and get the test excluded.
Steve
From: jakartaee-platform-dev <jakartaee-platform-dev-bounces@xxxxxxxxxxx> On Behalf Of Scott Stark
Sent: 29 June 2022 17:52
To: jakartaee-platform developer discussions <jakartaee-platform-dev@xxxxxxxxxxx>
Subject: Re: [jakartaee-platform-dev] Concurrency TCK without remote EJB, javax.rmi.RemoteException
Ok, we discussed this during the spec committee call and it is not so clear, so I'll use this PR as a test case for what is allowed. It is being raised to the spec committee via email.
On Jun 29, 2022 at 5:26:07 AM, Steve Millidge (Payara) <steve.millidge@xxxxxxxxxxx> wrote:
OK I’m not that familiar with the TCK challenge process and service releases. I thought we could only exclude or workaround challenged tests in a service release. I didn’t realise we could fix them.
If we can that’s great.
Steve
From: jakartaee-platform-dev <jakartaee-platform-dev-bounces@xxxxxxxxxxx> On Behalf Of Scott Stark
Sent: 29 June 2022 03:00
To: jakartaee-platform developer discussions <jakartaee-platform-dev@xxxxxxxxxxx>
Subject: Re: [jakartaee-platform-dev] Concurrency TCK without remote EJB, javax.rmi.RemoteException
Nothing in these changes violates the definition of a service release to address a tck challenge in the tck process as far as I see. They are not new tests and the existing remote interface usage is an implementation detail of the test. The only time a minor release is mentioned in the tck process is for the case of adding new tests.
On Jun 28, 2022 at 1:00:45 PM, Steve Millidge (Payara) <steve.millidge@xxxxxxxxxxx> wrote:
Wouldn’t incorporating a change of that magnitude into TCK require a new concurrency TCK minor release and ballot?
From: jakartaee-platform-dev <jakartaee-platform-dev-bounces@xxxxxxxxxxx> On Behalf Of Scott Stark
Sent: 28 June 2022 18:43
To: jakartaee-platform developer discussions <jakartaee-platform-dev@xxxxxxxxxxx>
Subject: [jakartaee-platform-dev] Concurrency TCK without remote EJB, javax.rmi.RemoteException
I have create a fork of the concurrency project and update the tck to only use local EJBs and even removed use of javax.rmi.RemoteException:
I have build the current glassfish repo and run the appserver/tests/tck/concurrency project against a build of this version of the concurrency TCK. It is showing no errors when run with SE 17. The PR for this update is here:
I'm not sure how to test this against a web profile configuration of glassfish. Can that be done from the appserver/tests/tck/concurrency project?
_______________________________________________
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev_______________________________________________
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev_______________________________________________
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
_______________________________________________jakartaee-platform-dev mailing listjakartaee-platform-dev@xxxxxxxxxxxTo unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev_______________________________________________
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
--Thanks
Emily
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
jakartaee-platform-dev mailing list
jakartaee-platform-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev
--
Thanks
Emily
_______________________________________________ jakartaee-platform-dev mailing list jakartaee-platform-dev@xxxxxxxxxxx To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-platform-dev