Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[jakarta.ee-spec.committee] Clarification for test challenge process

Committee members,

Below is the long awaited proposed text to specification committee members requesting them to update their TCK User's guide to clarify their use of Active or Lazy Consensus resolution.

Please review and we can discuss at our next committee meeting.

Some ideas:

  • I've suggested April 1 as an initial deadline. What do we think about that?
  • Should we file issues against each specification project to track this work?
  • Should we expand the To: list?

Happy to take feedback in reply as well as in discussion at the meeting.

Thanks,

-- Ed

To: jakartaee-spec-project-leads@xxxxxxxxxxx

Subject: Please choose your Test Challenge resolution process

Greetings Jakarta EE Specification Teams

The Jakarta EE Specification Committee is attempting to address an ambiguity regarding how each Jakarta EE Specification team expects to handle TCK Challenge issues.

The Specification Committee is requesting that each Jakarta EE Specification project clearly state their required process for managing and resolving TCK Challenge issues. Most commonly these are described in your TCK User Guide, under the section 'Test Appeals Steps.' This is also described in the Jakarta EE TCK Process guide if you want to review the goals and requirements.

There are two primary paths for resolving test challenges. 1) Active resolution, which requires an explicit approval action from someone on the specification committer team or 2) Lazy Consensus resolution, whereby a test challenge issue may become accepted if no action has been taken for some length of time.

Action Required

Each project must add text describing their specific procedures and resolution requirements in their TCK documentation (For many specifications, this will most naturally fit in the TCK Users Guide under the heading 'Test Appeals Process', perhaps as a new list item 2 or 3. Additionally, projects must state their selected resolution process in their top-level project information file.

Specifically, each project must clarify if lazy consensus is allow, and if so, how ...

  • Is lazy consensus resolution allowed?
  • If allowed
    • What is the time-frame for lazy consensus resolution.
    • What starts this time period (can it be reset?)
    • How to raise attention of the specification project committers, and how to escalate
    • What reporting is required for a certification that relies on an issue covered under  lazy consensus resolution

Before the next release of your TCK, please revise the Test Appeals (or equivalent) documentation in your TCK User Guide to reflect the choices your project has selected. The Spec. committee mentors will add a check-list item to ensure these updates are included in your next balloted release.

Examples

Here are some sample process descriptions your specification might consider (Please customize these as necessary for your specification project details)

Active Resolution user guide text

"TCK Challenge and resolution process for <your specification name>

This specification requires that all test challenges follow the "Active Resolution" process as described in the Jakarta EE TCK Process Guide. Please file your challenge in conformance with "Filing a Challenge" requirements in the TCK Process Guide. This team will attempt to take timely action but remember, we are volunteers. 

We try to track issues as they are filed but feel free to reach out to us via <your-specification-dev-list@xxxxxxxxxxx>.  If that fails, you may escalate your issue to the Jakarta EE Platform Committer list jakartaee-platform-dev@xxxxxxxxxxx. If it is necessary to escalate further, you may request action following the Grievance process as described in the Eclipse Development Process, Handbook under Grievance Handling. "

Lazy Consensus example user guide text

"TCK Challenge and resolution process for <your specification process name>

This specification allows for lazy consensus as a resolution path for TCK test challenges as described in the Jakarta EE TCK Process Guide. For this specification, test challenges will be deemed accepted, <indicate your time-frame> [days or weeks] from the date the test challenge issue is [filed | last commented on by a specification team committer]. Please file your challenge in conformance with "Filing a Challenge" requirements in the TCK Process Guide. If any specification team member determines that the challenge requires review it should be marked with the tag <challenge-review> and it will be resolved under the 'Active Resolution' process as described in the Jakarta EE TCK Process guide. Generally, once a conversation occurs with a member of the committer team, the filer should consider that the challenge will follow the Active Resolution process.

In the event a challenge is approved under lazy consensus, any vendor implementation Compatibility Certification Request (CCR) that uses this challenge must include the issue ID along with the TCK details and test reporting statistics for this specification, regardless if the CCR is filed to this specification project issue tracker, or it is filed for any other specification project issue tracker. Failure to include this issue reference could result in CCR  approval delay.

Once a TCK revision, that includes the resolution of this challenge is released, it is no longer necessary to include any additional reference in your CCR.

This specification team tries to track issues as they are filed but feel free to reach out to us via <your-specification-dev-list@xxxxxxxxxxx>.  If that fails, you may escalate your issue to the Jakarta EE Platform Committer list jakartaee-platform-dev@xxxxxxxxxxx. If it is necessary to escalate further, you may request action following the Grievance process as described in the Eclipse Development Process, Handbook under Grievance Handling. "

For the project readme, simplified text may be used. For example, in the case of Active Resolution only: "All test challenges for <your-specification-name> must be resolved by Active Resolution. Please see the TCK Users guide for more details." An example for Lazy consensus: "Test challenges for <your-specification-name> may be resolved through Lazy Consensus. Please see the TCK Users Guide for <your-specification-name> for more details."

For lazy consensus approval, the Specification Committee recommends a minimum of two weeks (14 days), and a maximum of four weeks (28 days) for the Lazy Consensus resolution time-frame.

The Specification Team has established an initial target of April 1 for committer teams to review their process and select their process type. There is no need to generate a new release of your specification just for this update. However, we would appreciate your team making the appropriate documentation changes in their repository as soon as they have reached a decision.

If your team does not respond

If a specification team does not specify the Specification committee will select the Lazy Consensus option with a 14 day approval time-frame. The committee will take action to make the necessary changes to the TCK User's guide and to the project readme.

As always, we encourage specification teams to be responsive to vendor inquiries and issues.

The Specification Committee team is available to assist you or your team if you have any questions.

Thank you

-- Ed Bratt, for the Jakarta EE Specification Committee


Back to the top