Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [servlet-dev] Moving Servlet TCK tests to the Servlet Specification project...


On Tue, Aug 29, 2023 at 5:39 PM Scott Marlow <smarlow@xxxxxxxxxx> wrote:


On Mon, Aug 28, 2023, 7:13 PM Olivier Lamy via servlet-dev <servlet-dev@xxxxxxxxxxx> wrote:
Hi
Listening here, I had a sort of feeling using the same repo was the preferred solution. Anyway if not it will be very easy to extract this as it is very separated Maven modules sources tree.
I have created a PR with this option: https://github.com/jakartaee/servlet/pull/540
Currently still a draft, it's just an import without any modification of sources from https://github.com/jakartaee/platform-tck/ (branch tckrefactor) once we agree on merging here I will create a PR to remove the code.
My idea was to have a first import marked as an initial import then make some changes:
- changing all the naming (packages and classes). currently every test classes are named URLCllient or Client (which generate almost non really human readable JUnit report example on Jetty CI build [1]) and packages use a format such: com.sun.ts.tests.servlet.api.jakarta_servlet.scinitializer.addfilter.URLClient. I was thinking about changing to something such jakarta.servlet.tck.api.scinitializer.AddFilter.

Only use Jakarta package for Spec API as per 
Specification committee.  They specifically didn't want tck tests to be created in the Jakarta package.

servlet.tck.api.scinitializer.AddFilter or tck.api.scinitializer.AddFilter or something else.

Thanks.
servlet.tck sounds good to me. 
 


I don't really have strong opinion here (but I have the feeling as usually we spend a lot of time on finding a good naming :))
- cleaning some part such not using anymore commons-httpclient but simple the JDK httpclient
- removing the (maybe not needed) static usage as the current code is using a lot of static methods which makes it impossible to run the tests in parallel via [2] Currently with Jetty 12 it takes around 15 minutes so there is probably some room to reduce that. 

Question: Do we want some extra Maven modules to run the Servlet TC against different implementations?

Toughts?

cheers 
Olivier



On Mon, Aug 14, 2023 at 10:03 AM Stuart Douglas via servlet-dev <servlet-dev@xxxxxxxxxxx> wrote:
The main advantage I can see for a single repo is that you can submit the spec changes and TCK updates in a single PR. I don't think different release cycles are that big a deal, there are plenty of examples out there of monorepos with different cycles, generally they just use a prefix on the tag to signify what is being released.

That said I don't feel super strongly about this either way.

Stuart

On Mon, 14 Aug 2023 at 06:36, Arjan Tijms via servlet-dev <servlet-dev@xxxxxxxxxxx> wrote:
Hi,

Most projects have everything in one repo, but a few have separate repos. And from those few, some have random other combinations. E.g. spec text and api in one repo, but tck in another. Or the TCK and spec text in one and the api in another etc. I guess there's an example to be found for every possible permutation, which is statistically what you would get indeed without any rules or guidance.

Of course, now someone really wants separate repos, and then with the next project someone really wants a single repo, etc.

Essentially I think there's no best outcome. The current situation of single, partial separate and fully separate just depends on whoever created those repos and whoever happened to respond to any discussion about it.

Kind regards,
Arjan Tijms

On Fri, 11 Aug 2023 at 18:35, Emily Jiang via servlet-dev <servlet-dev@xxxxxxxxxxx> wrote:
Since Jakarta TCKs have different release cycle due to challenges etc, it is much better to put tck under a new repo such as servlet-tck. CDI has spec/api repo and tck repo. TCK has done 10 release after 4.0 but the latest api/spec release is 4.0.1.

With this, I think I agree with Greg and suggest having a separate repo for the tck.

Thanks
Emily

On Fri, Aug 11, 2023 at 1:27 AM Olivier Lamy via servlet-dev <servlet-dev@xxxxxxxxxxx> wrote:
Hi,
As far as I understand, I understand this discussion as a consensus for this move.
I'm happy to propose a PR to move the code. 
This might take a bit of time as the current code in the TCK repo is based on some multi-modules but anyway, I will figure out something.
Cheers,
Olivier

On Fri, Jul 14, 2023 at 9:25 AM Stuart Douglas via servlet-dev <servlet-dev@xxxxxxxxxxx> wrote:


On Fri, 14 Jul 2023 at 05:00, Scott Marlow <smarlow@xxxxxxxxxx> wrote:


On Wed, Jul 12, 2023 at 8:34 PM Stuart Douglas <sdouglas@xxxxxxxxxx> wrote:
One question around this that will mostly affect full EE vendors. With the old way of running the TCK most of the tests were handled via a single adaptor/test harness (with a few exceptions such as CDI). Once you had your TCK setup you could for the most part run any subset of the tests using the same configuration.

The current underlying problem is that the Java Test Harness + Apache Ant + Platform TCK are not that well understood, so the number of contributors adding/updating TCK tests is too low.  I wouldn't expect that TCKs will use JUinit + Arquillian forever.  Hopefully there will be less complexity with the new TCKs which should make it easier to later switch to the latest build + unit test libraries.  
 

If all the TCKs are going to be split up my concern is that over time the requirements of the test harness for each project may diverge, and we end up with vendors needing to maintain multiple versions of kinda similar but subtly different test adaptors for each TCK.

Platform requirements should still be validated in a Platform/profile TCK.  The Platform TCK could extend the Servlet TCK tests with some (platform/profile level) cost to make changes for each release as needed.  The big payback on the effort is that TCK tests will be easier to maintain which improves the overall Jakarta EE development process.  

In WildFly, we did go through some pain setting up new EE 10 TCKs via https://github.com/wildfly/wildfly-tck-runners but I think that worked out fine.  I agree with Arjan's response earlier regarding other TCK adapters as well.
 

Do you think this will be a real concern?

The EE 8 - 9 upgrades were pretty smooth sailing but I recall there being work involved with earlier EE releases with a lot of TCK configuration changes and resolving several TCK failures as well.  I vaguely recall the first TCK releases as well that required "make" tools from four separate vendors (at least they did on Microsoft Windows).  The make scripts were rewritten to use Apache Ant and that has lasted a really long time.  

I was concerned as well when I first read the https://github.com/jakartaee/platform-tck/issues/51 issue about converting the Platform TCK into multi-module Maven projects but I think it is the best approach to take until we have new requirements to do something else in a later release.
 
I have not looked in depth at how this is being implemented so I am not really sure. In an ideal world IMHO as much as possible of the test infrastructure code should still be shared but I am not sure if it is being done that way.

I think it is a healthy thing to eliminate technical debt while rewriting a huge code base such that we are only left with needed features and will be better prepared for changes in the future.  We wanted to do this for EE 10 but decided to wait for EE 11 and here we are. :-)


Ok, it sounds like my concerns are not really an issue (or at least the lesser of two evils), so I would be ok with moving this into the Servlet repo.

Stuart
 

Scott
 

Stuart

On Tue, 11 Jul 2023 at 22:53, Scott Marlow via servlet-dev <servlet-dev@xxxxxxxxxxx> wrote:
Hi,

Does the Servlet Specification team want to move the Servlet TCK tests over to the Servlet Specification repo?  Olivier Lamy has been working on refactoring the Servlet TCK tests to work with JUnit/Arquillian and I think he would like to continue that effort with the Servlet Specification team, however that can best happen. 

Thanks,
Scott
_______________________________________________
servlet-dev mailing list
servlet-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/servlet-dev
_______________________________________________
servlet-dev mailing list
servlet-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/servlet-dev


--
Olivier
_______________________________________________
servlet-dev mailing list
servlet-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/servlet-dev


--
Thanks
Emily

_______________________________________________
servlet-dev mailing list
servlet-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/servlet-dev
_______________________________________________
servlet-dev mailing list
servlet-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/servlet-dev
_______________________________________________
servlet-dev mailing list
servlet-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/servlet-dev


--
Olivier
_______________________________________________
servlet-dev mailing list
servlet-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/servlet-dev


--
Olivier

Back to the top