[jakarta.ee-spec] Proposed text for TCK Process Clarification (Issue 74)

[Ed Bratt <ed.bratt@xxxxxxxxxx>]

Specification Committee Members:

Below, please find some starter text to further our effort to resolve the ambiguity in TCK process when a committer team is not responsive to a TCK Challenge Issue. I believe that the stated resolution goal of the committee is to ensure that each Specification project explicitly states its intentions with respect to any automatic actions that might be associated with a TCK Challenge.

This is in intended to address Spec. Committee Issue 74.

Please review and let's try to finalize this at our next meeting. Ideally, we can make progress via e-mail. If there is lots of feedback, it may become necessary to move this text into a Google document (or equivalent). I'm hoping this isn't that controversial.

One question I have is: do we want to establish a date for completing this task? I've suggested Dec. 2. I am presuming that we will monitor and nag groups that don't take action but I didn't feel compelled to assert this, initially.

I look forward to your feedback.

Thank you,

-- Ed Bratt

----

To: jakartaee-spec-project-leads@xxxxxxxxxxx

Subject: TCK Challenge Clarification Action Needed

Hello Specification Committer Team members

It has come to the Specification Committee's attention that there is potential confusion about how TCK Challenges are to be handled. Particularly in the event that a committer team is not immediately responsive to the challenge issue. This message requests your committer team take specific actions to remove this ambiguity from the test challenge process.

The following text (or something close to it) appears under the heading of "TCK Test Appeals Steps" in most TCK User Guides:

Challenges can be resolved by a specification project lead, or a project challenge triage team, after a consensus of the specification project committers is reached or attempts to gain consensus fails. Specification projects may exercise lazy consensus, voting or any practice that follows the principles of Eclipse Foundation Development Process. The expected timeframe for a response is two weeks or less. If consensus cannot be reached by the specification project for a prolonged period of time, the default recommendation is to exclude the tests and address the dispute in a future revision of the specification.

(Emphasis added) This has caused confusion by some parties when filing TCK Challenges.

To resolve this potential confusion, the specification committee is requesting that each Specification Committer team clarify their policy regarding any automatic decision making policies for TCK Challenges. Specifically, we request that teams clarify if any type of automatic acceptance, or Lazy Consensus, or default actions will take place.

Please take the following actions:

1. Update your project's top-level read me and describe any automatic policy that the team will follow for any specification TCK challenge issue filed

2. Update your TCK User Guide text to match the choices made in (1).

Since accepting a challenge obligates the TCK committer team to make a new release of the TCK, we recommend that committer teams decide this issue carefully.

Here is some suggested text for your top-level readme:

TCK Test Challenge Process

This specification committer team [does|does not] allow automatic acceptance (Lazy Consensus) of TCK Challenges. More details about TCK Challenges are included in the TCK User Guide under 'TCK Test Appeals Steps'. You may also read about Challenges in Jakarta EE TCK Process. After filing a TCK Challenge Issue, a vendor is allowed to pursue escalation following the Eclipse Development Process Grievance Handling procedure.

If your specification project allows automatic acceptance, please include a description of how this is intended to work. For example:

For [project-name] TCK, a TCK Challenge will be deemed 'Accepted' if the issue is not resolved and there has been no update to the TCK Challenge issue for a period of [XX] days. If a challenge is automatically accepted, the vendor may submit a CCR with an exception statement, referencing the TCK Challenge Issue ID. Vendors are not permitted to alter the TCK except where specifically allowed in the TCK User Guide. 

Edit this to suit your team's process. If your specification does not offer automatic acceptance, leave the second paragraph out. Of course, change any of the text if it not compatible with your User Guide. Be sure to fill in the duration your team chooses if you will allow automatic acceptance. We recommend this be sufficient to allow for potential member absence or vacation time periods.

The specification committee encourages your project to follow the guidance offered in Jakarta EE TCK Process and that you strive for active and swift resolution for all TCK Challenges. If your team wants to include text about response time goals (e.g. This team strives to respond to all TCK Challenges within [NN] days) that would be encouraged. You might also consider adding something like: If you wish to escalate to the developer team, please send e-mail to <your-project-dev e-mail list>.

The committee recommends that all decisions be documented in the TCK Challenge Issue; that Accepted issues include reference to the specific change implemented (generally to the PR), followed by a reference to the released TCK update (maybe a repository tag, or download link).

The Specification Committee team is available to assist you or your team if you have any questions.

The Specification Team has established a target of Monday Dec. 2, to complete these updates.

Thank you

-- Ed Bratt, for the Jakarta EE Specification Committee