Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jakartaee-platform-dev] TCK tests in the same repo as API andSpec

Andy Guibert wrote on 2/20/20 9:13 AM:
Bill and I discussed this topic on the jakartaee-tck PR for a while [1], but I believe it's time to try and loop the discussion back to the mailing list to try and get a final decision.
This has been discussed in the platform project team meeting recently.  The 2/18/2020 meeting minutes should appear here shortly.

Overall it seems that the platform isn't going to force specs to do TCKs a certain way, and that specs have the freedom to choose.
I don't think that's the case.

We need a way to create the overall platform TCK, e.g., by combining pieces from the standalone spec TCKs.  To make that work, there's going to need to be some agreement on how the pieces fit together.  For example, we'll likely need to agree on a testing framework, or small set of frameworks, and some packaging rules, that allow the integration.

Within the constraints of that framework(s), spec projects will be free to develop their tests however they want.  Hopefully some best practices will emerge and we'll learn from each other and come out with something even better.

Among JSON-B committers we are in agreement that we want to migrate to do TCKs this way, and JSON-P would also like to follow suit [2].

I'm happy to use JSON-B as a guinea pig to see how it works and react to feedback. 
At this point I don't see any remaining blockers for moving forward with the initial concept, but I will wait for a few days to let people chime in.
I think it's fine to run some experiments such as this.  They may help us learn some of the things we'll need to know to do the much bigger project.  We don't want to end up with two TCKs that you need to run for each spec, so all of this needs to come back together at some point.

Note however that these two cases are some of the simpler cases.  They're effectively just standalone libraries with minimal dependencies and minimal run time requirements.  When we get to things like Jakarta Mail, which depends on an external mail server, and JAX-RS, which is a client/server API, the framework and integration requirements will be more important.

On Thu, Feb 6, 2020 at 8:12 PM Bill Shannon <bill.shannon@xxxxxxxxxx> wrote:
Andy Guibert wrote on 2/6/20 5:54 PM:
> Right now we have 2 types of TCKs:
> A) CDI and BeanVal use Arquillian+TestNG, and produce their TCKs as maven
> dependencies
> B) All other specs use JavaTest
> I'm proposing to make a 3rd type of TCK (very similar to A) that uses
> Arquillian+JUnit.
Which would only be used by completely new TCKs?  You don't plan to convert any
existing TCKs to this style?

> However, I don't really care what test framework gets used to run the TCKs,
> the main items I care about as a spec developer are:
> 1) API/spec/TCK all in one repo
> 2) Implementations (standalone lib or app server) can _easily_ pull in and run
> the TCKs as a Maven Dependency and doing `mvn test` or `mvn verify`
This looks like another tabs vs. spaces issue.  :-)

All I care about is that the API, spec, and TCK are each in their own repo.  But
we decided not to force either style on projects; projects can structure this in
repos however they feel is best.

Most of our tests aren't produced as Maven artifacts and aren't runnable from
Maven so that's going to take a lot of work.  One of the challenges is that the
"official" signed versions of the TCKs are produced as standalone zip files.

Also, configuring some of these TCKs is significantly non-trivial.  It's now
clear that you'll ever be able to run them with just "mvn test".

> This is what MicroProfile has done and the model works very well. 
> Also, I'm not suggesting we mandate _all_ specs migrate to this way (or
> require all specs to do anything). Rather, I'm suggesting that we tolerate
> specs to migrate to the way CDI/BeanVal does it (except using JUnit instead of
> TestNG preferably).
I don't believe there's any restriction preventing new specs from doing so.

jakartaee-platform-dev mailing list
To change your delivery options, retrieve your password, or unsubscribe from this list, visit;!!GqivPVa7Brio!J0unhEB7hcmAaObJ3iqozFubPqFo-sRe16MNV7Bvd7shJ8UeSW__9wRQHBiVKz4cmA$ 

Back to the top