Re: [jakartaee-tck-dev] [External] : Re: standard recipe to follow for r

[Gurunandan Rao <gurunandan.rao@xxxxxxxxxx>]

IMO, Platform TCK refactoring should be targeted on SPEC basis for a particular Jakarta EE release, refactoring all of platform TCK is a huge effort and not feasible for a single Jakarta EE release.

Common TCK framework needs to be implemented for the refactored TCK's, some of the common functionality are as follows:

1) Signature test Driver and recording of new Signature.

2) Logging framework for TCK.

3) Test grouping to implement common grouping like SE/EE.

4) Test exclusion.

5) Framework for seamless run of refactored standalone tests in a container for Platform runs.

6) Test execution report.

We should also explore on feasible of a common framework to encapsulate from vendors the test tech Junit/TestNG/Arquillian.

---

From: jakartaee-tck-dev <jakartaee-tck-dev-bounces@xxxxxxxxxxx> on behalf of arjan tijms <arjan.tijms@xxxxxxxxx>
Sent: 08 October 2022 16:00
To: jakartaee-tck developer discussions <jakartaee-tck-dev@xxxxxxxxxxx>
Subject: [External] : Re: [jakartaee-tck-dev] standard recipe to follow for refactoring the Platform TCK tests...

 

Hi,

On Saturday, October 8, 2022, Emily Jiang via jakartaee-tck-dev <jakartaee-tck-dev@xxxxxxxxxxx> wrote:

One of the requirements is that the refactored tests must be runnable by runtimes such as Open Liberty so that the whole integration can be tested.

Yes, that specifically applies to the json and _expression_ language tcks, which run outside runtimes such as open liberty now.

Kind regards,

Arjan Tijms

 

Thanks 

Emily 

Sent from my iPhone

On 7 Oct 2022, at 17:50, arjan tijms <arjan.tijms@xxxxxxxxx> wrote:



Hi,

Not yet jumping towards a definite answer, but there are a couple of options to keep in mind.

There are now two main variants in use to base the tests on: Junit and TestNG. As far as I can tell, this choice is completely arbitrary and neither has any real advantage over the other (please correct me if I'm wrong).

There are two phases to use for the tests with maven. The unit test phase (using surefire) and the integration test phase (using failsafe). Sometimes the choice is arbitrary, sometimes there's an actual reason; integration tests allow one to use the full output produced by a maven module, e.g. a .war.

There are also two main ways how tests are practically conducted using Arquillian. There is the so called "testable" mode (also called "magic mode") where a test class that looks like a client test is magically (hence the name) transported to the server and run from there. Arquillian automagically using a protocol transfers test results to the client (junit or testng). The other mode is "non-testable" (also called "run-on-client"), where the unit or IT test explicitly runs on the client, and where explicit HTTP requests are submitted to the server under test. There's a few sub-variants here again. Some TCKs have e.g. a Servlet where they return some result and that result is then inspected by the client test. E.g. assert response contains "user logged-in". Others have a Servlet that manually inspects some results server-side, and then only return "succeeded" or "failed" to the client test. In some cases there's even a kind of junit/testng emulation like layer implemented and manually invoked at the server side.

Finally, but it largely follows from the choices above, there are TCKs that produce a "single test jar" that contain all the tests, which therefore requires a "runner pom" that imports this test jar via a special directive. Other TCKs are a "multi-module maven project", where each piece of test code is its own maven module and can be used and deployed as a single standalone application (and therefore also debugged as such).

We can probably create a table to categorize the existing TCKs into the above options.

Kind regards,

Arjan Tijms

On Fri, Oct 7, 2022 at 5:50 PM Scott Marlow <smarlow@xxxxxxxxxx> wrote:

Hi,

It would be good to have a standard recipe to follow for refactoring the Platform TCK tests as we discussed on the TCK call [1] this week.

What are the initial steps that we followed for EE 10 to refactor Platform TCK tests to junit?  For the Faces TCK refactoring I recall we discussed a lot on Faces issues (e.g. like how to deal with vendor deployment descriptors).  Does anyone feel ready to respond with what the steps might be? 

Or should we explore those further together (pick a test bucket and start refactoring at least one test in the Platform TCK)? 

Scott

[1] https://docs.google.com/document/d/1V1dDLJkd14EDRMPeuI0VzPtU4Lbli8FFBd1pLDLlOrY/edit#

_______________________________________________
jakartaee-tck-dev mailing list
jakartaee-tck-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-tck-dev

_______________________________________________
jakartaee-tck-dev mailing list
jakartaee-tck-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartaee-tck-dev