[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
RE: [wtp-dev] Re-Inventing Hot List
|
Here are my responses to your points-
1. There is a gate. We review the hot
bugs and demote them if they are not really that hot. We do that in bugzilla
and the weekly status calls. That requires all interested parties to be
engaged.
2. In general, people are not assigning
severity accurately. There is a clear definition of what the severities
mean. It is not just a matter of how important the reporter thinks the
bug is. Let's keep the standard meanings.
3. They are marked in a comment appended
to the bug, but adding the [hotbug] prefix is a good idea. There are also
listed on the Web site and there is a query to list them, and several variants,
e.g. unresolved, resolved, all, and the blocker/critical minus the hot
bugs , e.g. [1]
4. The PMC arbitrates if no consensus
is reached. We discuss them in the status calls. If an inadequate justification
is given, we demote them.
5. The adopter pool is open. Any vendor
or project that has plans to build on WTP is welcome. Do you know of any
vendor or project that has been excluded? Let's use the Bugzilla severities
so we have a common vocabulary. End users don't generally report Blockers.
They are really for developers since they block development or testing
of WTP. For a problem to be truly a Blocker, if would not be a rare edge
case. You can work around edge cases. So if an end user did report a real
Blocker then we would certainly address it. Even if an adopter reported
a Critical problem, we'd still look at it. If it was an edge case we might
well defer it.
[1] http://www.eclipse.org/webtools/plans/1.0/adopter-hotlist-report.html
Arthur Ryman,
IBM Software Group, Rational Division
blog: http://ryman.eclipsedevelopersjournal.com/
phone: +1-905-413-3077, TL 969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL 969-4920
mobile: +1-416-939-5063, text: 4169395063@xxxxxxx
"Konstantin Komissarchik"
<kosta@xxxxxxx>
Sent by: wtp-dev-bounces@xxxxxxxxxxx
02/22/2006 12:04 PM
Please respond to
"General discussion of project-wide or architectural issues." |
|
To
| "General discussion
of project-wide or architectural issues." <wtp-dev@xxxxxxxxxxx>
|
cc
|
|
Subject
| RE: [wtp-dev] Re-Inventing
Hot List |
|
Here are the problems that
I see with the process as it currently exists:
1. There is no gate to the
hot list. Nothing to control what gets added. I’ve seen issues that have
minimal impact and documented workarounds be tracked on the hot list.
2. We are under-utilizing the
priority and severity fields by focusing exclusively on the hot bugs at
the status meetings.
3. The hot bugs are not marked
in bugzilla, making it very difficult to construct queries around them.
4. There is no one to arbitrate
disagreements over the categorization.
5. And finally, the hot list
creates at least an appearance of an “exclusive club” that’s fundamentally
against the ethos of open source. An S1 coming from a WTP end user should
(in my opinion) be resolved with a higher urgency than an S4 coming from
a major adopter.
I realize that my proposal
uses the severity field in a novel way, but think about it this way. Assignment
of the initial severity is in the hands of the bug originator. It’s a
way for them to rank how much an impact this is having on them. It might
be a spelling issue, but if the impact of that from the originator’s perspective
is that they will not ship until the problem is fixed, then that issue
ranks as a high severity. Alternative, the spelling problem might be buried
deep in some dialog, so the originator might assign a lower severity to
that. If there is a disagreement the triage is there to arbitrate. What
about that million character URL example? As the likelihood of anyone running
into this is rather low, it’s a low severity issue despite the fact that
it causes a crash. Let’s take another example. Suppose an adopter is developing
a tool on top of WTP and requires a hook of some sort in order to get their
scenario working properly. Unless a workaround is found or the hook is
added, their scenario is dead in the water. That deserves to be an S1.
The problem with the bugzilla definition of the severity levels is that
they are too restrictive. Technically, asking for an api hook under the
current system would force the bug to be marked as an enhancement request.
There is no way for the originator to indicate the _actual_ impact
of the problem.
My proposal would establish
a concrete way to categorize the actual impact of the bug on the originator
that we can all talk about in a meaningful way. It establishes a triage
committee to arbitrate disagreements. And, finally, it re-emphasizes that
if a bug is targeted to a certain release, one can have a good degree of
certainty that it will be fixed in that release.
- Konstantin
From: wtp-dev-bounces@xxxxxxxxxxx
[mailto:wtp-dev-bounces@xxxxxxxxxxx] On Behalf Of Arthur Ryman
Sent: Wednesday, February 22, 2006 7:13 AM
To: General discussion of project-wide or architectural issues.
Subject: Re: [wtp-dev] Re-Inventing Hot List
Kosta,
Thx for the suggestions. Let's take a step back for a minute and review
why we have a Hot Bug process.
WTP has two main goals:
1) Provide tools for Web application developers.
2) Provide a platform for tool developers.
Both of these are very important for the success of WTP. The Hot Bug process
is aimed at goal #2.
There are many more application developers than tool developers so the
bugs that are important to tools developers can easily get lost in the
large bugzilla backlog. To put things in perspective, there have
been a total of 6700 bugs opened for WTP to date, but only 200 of them
are Hot Bugs. That's around 3% of the total.
We need to treat tool vendors, aka adopters, differently. WTP is the first
Eclipse project, AFAIK, to treat adopters specially. The current process
needs to be refined over time. I'm sure we don't have it right yet.
However, I don't think we should rely on severity since I'd like us to
use severity correctly. Sometimes an adopter problem does not meet the
criteria for Blocking or Critical but is still very important. A Major
problem could be enough for an adopter to not use WTP. The meaning of Major
is that some major function is unusable. That major function might be exactly
what the adopter needs.
This means we need to use another field. Priority is orthogonal to severity.
Priority is our way of saying what we are going to work on. If we mark
a bug as P1 then it is release-defining, i.e. we should delay the release
if a P1 function is broken. That translates into how we allocate our resources.
We need to plan our work so that we fix all the P1s first, then the P2s,
etc.
To make this more concrete, suppose there was a bug in our HTML editor
that caused Eclipse to crash whenever the HTML editor opened a document
that contained a URL that was longer that 1,000,000 characters. That would
be a Critical bug because it caused a crash, but I would assign it a very
low priority because it is a very extreme edge case and would never affect
a user.
In contrast, suppose our server tools listed a server adapter for IMB WebSphere.
I'd immediately open a Hot Bug to get that corrected. From my point of
view that would be a P1, but the severity would be trivial.
Arthur Ryman,
IBM Software Group, Rational Division
blog: http://ryman.eclipsedevelopersjournal.com/
phone: +1-905-413-3077, TL 969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL 969-4920
mobile: +1-416-939-5063, text: 4169395063@xxxxxxx
"Konstantin Komissarchik"
<kosta@xxxxxxx>
Sent by: wtp-dev-bounces@xxxxxxxxxxx
02/21/2006 09:36 PM
Please respond to
"General discussion of project-wide or architectural issues." |
|
To
| "General discussion
of project-wide or architectural issues." <wtp-dev@xxxxxxxxxxx>
|
cc
|
|
Subject
| [wtp-dev] Re-Inventing Hot
List |
|
I’d like to start the conversation about how we can improve the Adopter
Hot List process going forward. Here is a proposal to get us started. Feel
free to poke holes in it...
This proposal places more emphasis on the severity field and establishes
a strict release exit criteria for high severity issues. A triage committee
(composed of PMC members or their designated replacements) would be established
in order to resolve disagreements over the severity and other issues.
- In order to exit a release all S1 (Blocking
Severity) and S2 (Critical Severity) bugs targeted to that release have
to be resolved. Additionally, there should not be any untargeted/uninvestigated
S1/S2 bugs. The triage committee can push bugs of lower severity that are
targeted to that release to the next release if there isn’t sufficient
time remaining in order to resolve them.
- All S1/S2 bugs will be reviewed at the weekly
status meetings. The severity as set by the originator can be challenged.
The triage committee will arbitrate and have the power to assign a lower
severity if necessary.
- Anyone can submit a bug of a lower priority
to the weekly status meeting agenda in order to get the status or to request
a targeting decision.
- A bug is only targeted to a particular release
when a dev has made a commitment to work on it for that release.
- Once a bug is targeted, that setting cannot
be changed without the approval of the originator or the triage committee.
This applies to all bugs regardless of the severity.
I believe that if the above process is implemented and abided by than we
no longer need the Adopter Hot List. Thoughts?
- Konstantin
_______________________________________________________________________
Notice: This email message, together with any attachments, may contain
information of BEA Systems, Inc., its subsidiaries
and affiliated
entities, that may be confidential, proprietary, copyrighted
and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.
_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev
_______________________________________________________________________
Notice: This email message, together with any attachments, may contain
information of BEA Systems, Inc., its subsidiaries
and affiliated
entities, that may be confidential, proprietary, copyrighted
and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.
_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev