Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jakarta.ee-spec.committee] Notes on TCK maintenance
At one point we were discussing a scenario where the TCK version was advancing without a corresponding update in the spec, one example of which could possibly be fixing a faulty TCK test that had no corresponding spec issue as raised by Bill S. In that same conversation I thought we were also saying that an update of the spec/TCK would reset the minimum version of the TCK that one needed to pass. This was to address someone sitting on the TCK version that had less stringent test requirements.

I don't recall a point at which we unanimously switched to a view that the spec and TCK would always be released in lock step. This needs to be hammered out in the draft discussion.



On Wed, Jul 18, 2018 at 10:20 AM David Blevins <dblevins@xxxxxxxxxxxxx> wrote:
Pushing some rough notes on what we need to draft up or I'll forget the details:

- TCK major version is released.  Say Jakarta EE 9
   - All vendors work against this "golden master"
   - Test challenges happen naturally and result in a growing exclude list
   - "Passing" means you complete the TCK, minus the tests in the exclude list

- At the moment the Specification project would like to unexclude, update or add new tests, a maintenance version is released, such as Jakarta EE 9.0.1
   - TCK and Specification are versioned/patched in lock-step form
   - These maintenance allow the exclude list to be reset to zero and new tests to be added
   - Maintenance versions are subject to the same Specification Committee approval processes as major versions
   - A cadence for the when or how often maintenance versions must be release is not mandated.  In practice it may be quarterly (this is for our notes)
   - A vendor may pass any of the versions, 9.0.x, and claim compatibility
   - A compatibility page will be maintained to document all passing implementations, sorting the most up-to-date implementations at the top
   - Maintenance versions, 9.0.x, may not change the goals or add features to the specification
   - Test challenges happen naturally and result in a growing exclude list to the maintenance version and the process repeats.
   - "Passing" means you complete the TCK, minus the tests in the exclude list

- Test challenge process and exclude list management
   - Specification projects are responsible for the handling of test challenges
   - Challenges are handled and discussed openly on the projects xyz list (could be dev or we make a dedicated list)
   - Additions to the exclude list must pass a super majority vote by the specification project
   - Tests may not be altered, fixed, patched or added via an exclude list.  This requires a maintenance version as detailed above

As a side note commentary, if we did release a maintenance version it would be one way to show "speed" and that we are faster than before.  Thus, I don't see them as a problem to be avoided, but an opportunity that should be encouraged.  I'd recommend framing things up with quarterly as a goal and our timings and approval processes to ensure that remains realistic.

_______________________________________________
jakarta.ee-spec.committee mailing list
jakarta.ee-spec.committee@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee

Back to the top