Re: [microprofile-wg] MP Telemetry TCK Challenge Discussion

[Emily Jiang <emijiang6@xxxxxxxxxxxxxx>]

Hi Jason,

Thank you for explaining your scenario! I think we talked about TCK Challenges from different angles. TCK Challenges are related to certifications. Therefore, you don't need to `Challenge` optional tests to get your certification. You can go ahead with your certification as these tests are optional. Having that said, you can file an issue as you have done if you spotted any issues with optional tests and these issues will be discussed.

As per your notes, since the instruction you quoted caused some confusion, I think that should be addressed with a PR.

In MicroProfile specification TCKs, there are some optional tests relying on Jakarta EE specifications outside of Jakarta EE Core Profile. For example, MP JWT has also some optional tests relying on Jakarta Security, not part of Core Profile either. MP Open API has some optional tests proving it works with Jakarta Validation. These optional tests do have business to demonstrate MicroProfile working with other Jakarta EE specifications and form a great ecosystem. 

If the optional category is not clear enough, we should discuss this at the MP Technical call on Tuesday 21st November on how best to document or restructure these kinds of tests. It will be great if you can join us there. You can see the meeting calendar here. We also have weekly calls on Mondays to discuss MP Telemetry as per Jan's notes. It will be great to have you there as well.

HTH

Thanks,

Emily

On Wed, Nov 8, 2023 at 9:05 PM Jason Lee via microprofile-wg <microprofile-wg@xxxxxxxxxxx> wrote:

Yesterday, I (with my red fedora firmly in place) opened a challenge against one set of tests in the MP Telemetry TCK: https://github.com/eclipse/microprofile-telemetry/issues/137

The initial challenge was filed because the test itself is testing functionality that depends on Jakarta EE specs (Concurrency and Servlet) that are not listed as required by the platform spec (https://download.eclipse.org/microprofile/microprofile-6.1/microprofile-spec-6.1.html#microprofile6.1). It is our understanding that sub specs can not expand that list of required specs, so the dependency of these tests (and the functionality their presence implies) is impermissible. After more discussion, we realized -- unless we've missed something -- that the tests also cover functionality that is specified in neither the MP Telemetry nor the OpenTelemetry specifications, so they're testing non-specified functionality, making them doubly inappropriate. We contend, then, that these tests should be removed from the TCK.

Emily responded that these tests are optional, so we can simply ignore the tests and still certify, and that's certainly our plan. However, optional or not, these tests -- as a matter of "legality" -- ought not be there, optional or not, for implementers and integrators to have to deal with. Furthermore, the assertion that the tests are optional is also somewhat suspect, based on the language in the TCK itself (https://github.com/eclipse/microprofile-telemetry/blob/main/tracing/tck/README.adoc#declaring-the-tests-to-run):

Although support for JAX-RS server async programming models is not optional, these tests depend on Jakarta Concurrency because they use ManagedExecutorService.

If you are testing in an environment which does not provide Jakarta Concurrency, you should exclude the optional-jaxrs-tests group.

Since WildFly does provide Jakarta Concurrency, it would seem that these test are not, in fact, optional. (Our OpenTelemetry integration is done using a shared library -- smallrye-opentelemtry -- that can not add the Jakarta dependencies need to properly implement support for the functionality under test. Addressing that would require a fair bit of re-work that we'd prefer to avoid given the optional nature of the requirement).

What we, the WildFly team, would like from this group is some clarity on these tests, specifically whether or not their presence, even if optional, is permissible given the concerns above.

Thanks!

--
Jason Lee (Red Hat WildFly/EAP Team)
OKC JUG President
https://jasondl.ee
https://twitter.com/jasondlee
http://linkedin.com/in/jasondlee

_______________________________________________
microprofile-wg mailing list
microprofile-wg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/microprofile-wg

--

Thanks
Emily