| [jakartaee-spec-project-leads] Spec. Leads, Please take action -- Clarification of Test Challenge Process |
Greetings Jakarta EE Specification Teams
The Jakarta EE Specification Committee is attempting to address
an ambiguity regarding how each Jakarta EE Specification team
expects to handle TCK Challenge issues.
The Specification Committee is requesting that each Jakarta EE
Specification project clearly state their process for managing and
resolving TCK Challenges. Most commonly these are described in
your specification's TCK User Guide, under the section 'Test
Appeals Steps.' General guidance is also given in the Jakarta
EE TCK Process guide. You may want to review the goals and
requirements.
There are two primary paths for resolving test challenges. 1)
Active resolution, which requires an explicit approval action from
someone on the specification committer team or 2) Lazy Consensus
resolution, whereby a test challenge issue may become accepted if
no action has been taken for some length of time.
Each project must add text describing their specific procedures and resolution requirements in their TCK documentation (For many specifications, this will most naturally fit in the TCK Users Guide under the heading 'Test Appeals Process'), perhaps as a new list item 2 or 3. Additionally, projects must state their selected resolution process in their top-level project information file.
Specifically, each project must clarify if lazy consensus is
allowed, and if so, how ...
Before the next release of your TCK (major, minor, or micro), please revise the Test Appeals (or equivalent) documentation in your TCK User Guide to reflect the choices your specification intends to follow. The Spec. committee mentors will add a check-list item to ensure these updates are included in your next balloted release.
Here are some sample process descriptions your specification
might consider (Please customize these as necessary for your
specification project details)
Active Resolution user guide text
"TCK Challenge and resolution process for <your specification name>
This specification requires that all test challenges follow the "Active Resolution" process as described in the Jakarta EE TCK Process Guide. Please file your challenge in conformance with "Filing a Challenge" requirements in the TCK Process Guide. This team will attempt to take timely action in response to your challenge.
We try to track issues as they are filed but feel free to reach out to us via <your-specification-dev-list@xxxxxxxxxxx>. If that fails, you may escalate your issue to the Jakarta EE Platform Committer list jakartaee-platform-dev@xxxxxxxxxxx. If it is necessary to escalate further, you may request action following the Grievance process as described in the Eclipse Development Process, Handbook under Grievance Handling. "
Lazy Consensus example user guide text
"TCK Challenge and resolution process for <your specification process name>
This specification allows for lazy consensus as a resolution path for TCK test challenges as described in the Jakarta EE TCK Process Guide. For this specification, test challenges will be deemed accepted, <indicate your time-frame> [days or weeks] from the date the test challenge issue is [filed | last commented on by a specification team committer]. Please file your challenge in conformance with "Filing a Challenge" requirements in the TCK Process Guide. If any specification team member determines that the challenge requires review it should be marked with the tag <challenge-review> and it will be resolved under the 'Active Resolution' process as described in the Jakarta EE TCK Process guide. Generally, once a conversation occurs with a member of the committer team, the filer should consider that the challenge will follow the Active Resolution process.
In the event a challenge is approved under lazy consensus, any vendor implementation Compatibility Certification Request (CCR) that uses this challenge must include the issue ID along with the TCK details and test reporting statistics for this specification, regardless if the CCR is filed to this specification project issue tracker, or it is filed for any other specification project issue tracker. Failure to include this issue reference could result in CCR approval delay.
Once a TCK revision, that includes the resolution of this challenge is released, it is no longer necessary to include any additional reference in your CCR.
This specification team tries to track issues as they are filed but feel free to reach out to us via <your-specification-dev-list@xxxxxxxxxxx>. If that fails, you may escalate your issue to the Jakarta EE Platform Committer list jakartaee-platform-dev@xxxxxxxxxxx. If it is necessary to escalate further, you may request action following the Grievance process as described in the Eclipse Development Process, Handbook under Grievance Handling. "
Please also include this clarification in your GitHub project readme file. Simplified text may be used. For example, in the case of Active Resolution: "All test challenges for <your-specification-name> must be resolved by Active Resolution. Please see the TCK Users guide for more details." An example for Lazy consensus: "Test challenges for <your-specification-name> may be resolved through Lazy Consensus. Please see the TCK Users Guide for <your-specification-name> for more details."
For lazy consensus approval, the Specification Committee
recommends a minimum of two weeks (14 days), and a maximum of four
weeks (28 days) for the Lazy Consensus resolution time-frame.
The Specification Team has established an initial target of Friday May 16, 2025 for committer teams to review their process and select their process type. There is no need to generate a new release of your specification just for this update. However, we would appreciate your team making the appropriate documentation changes in their repository as soon as they have reached a decision.
If a specification team does not specify the Specification
Committee will select the Lazy Consensus option with a 14 day
approval time-frame. The committee will take action to make the
necessary changes to the TCK User's guide and to the project
readme. These will be submitted as PRs for your committer team to
review and approve.
As always, we encourage specification teams to be responsive to
vendor inquiries and issues.
The Specification Committee team is available to assist you or
your team if you have any questions.
Thank you
-- Ed Bratt, for the Jakarta EE Specification Committee