I appreciate the
energetic discussion on the wtp-dev list, but we need to ensure that we’re
achieving the project’s aims with this critical support mechanism. I describe a
possible approach below; let’s review where we are on this discussion on the
Tuesday call.
I believe we need to
retain control over what goes on the hotlist, although mastering the data in
Bugzilla, automating generation of the hot list on the WTP site, and getting
“please add …” noise off wtp-dev are all worthwhile
goals.
Here’s one process that
addresses these concerns without wholesale revisiting of the hotlist
concept:
- Create a bug to represent the “Hot
List Suggestion Box” (per release). Anyone can add a comment requesting a new
addition to the hot list, discuss existing requests, indicate a suggestion has
been accepted, record a decline, etc.
- One person is designated
“promoter”; when a decision is reached to promote a suggestion to the hot
list, the promoter adds a comment to the defect with a standard
syntax.
- The WTP adopter page hotlist
report searches the comments in this bug and uses these special entries to
produce the hot list report. We can augment the current report with additional
columns for requesting adopter, etc. as we see fit by augmenting the form of
the entries and the report generation logic. Note that only the promoter’s
comments are used, so there is no possibility of an “accidental” promotion by
others.
The only downside I see
with the above is that a standard Bugzilla query cannot be used; if we want to
achieve that without giving up control over promotion, then we can use a proxy
defect / hotlist entry in lieu of comments and a text
converter:
- (Alternative approach) When a
decision is reached to promote a bug to the hotlist, the promoter creates a
proxy defect that depends on the original bug, titling it “[hotlist]
<original title>”.
- To view the hotlist bugs, you
search for [hotlist] in the title *and* the promoter as submitter; this
search can be done in Bugzilla and no special postprocessing is required to
view the results.
From:
wtp-dev-bounces@xxxxxxxxxxx [mailto:wtp-dev-bounces@xxxxxxxxxxx] On Behalf Of David M Williams
Sent: Wednesday, February 22, 2006 10:50
AM
To: General discussion of project-wide or architectural
issues.
Subject: RE: [wtp-dev] Re-Inventing Hot
List
Sounds like we
have a ways to go to resolve these issues.
I,
for one, would like to see this moved off the wtp-dev mailing list, and
utilitize bugzilla better.
As it it, being on
the dev list has effect of either wasting time, or watering down the importance,
since
when someone says "please add
xxxx to hot list" ... EVERYONE has to click on it to see if
something they should pay more
attention to. Or .. else, ignore it, and wait for someone else
to tell them what is on their
top priority list.
It even seems the "seperate
list" queries are often out of date, so, hard to trust them.
|
"Konstantin
Komissarchik" <kosta@xxxxxxx>
Sent by:
wtp-dev-bounces@xxxxxxxxxxx
02/22/2006 12:04
PM
|
Please respond
to
"General discussion of project-wide
or architectural issues."
<wtp-dev@xxxxxxxxxxx> |
|
|
To |
"General discussion of project-wide or architectural
issues." <wtp-dev@xxxxxxxxxxx>
|
|
cc |
|
|
Subject |
RE: [wtp-dev]
Re-Inventing Hot
List |
|
Here are
the problems that I see with the process as it currently exists:
1. There is no gate to
the hot list. Nothing to control what gets added. I’ve seen issues that have
minimal impact and documented workarounds be tracked on the hot
list.
2. We are
under-utilizing the priority and severity fields by focusing exclusively on the
hot bugs at the status meetings.
3. The hot
bugs are not marked in bugzilla, making it very difficult to construct queries
around them.
4. There is no one to
arbitrate disagreements over the categorization.
5. And finally, the hot
list creates at least an appearance of an “exclusive club” that’s fundamentally
against the ethos of open source. An S1 coming from a WTP end user should (in my
opinion) be resolved with a higher urgency than an S4 coming from a major
adopter.
I realize that my
proposal uses the severity field in a novel way, but think about it this way.
Assignment of the initial severity is in the hands of the bug originator. It’s a
way for them to rank how much an impact this is having on them. It might be a
spelling issue, but if the impact of that from the originator’s perspective is
that they will not ship until the problem is fixed, then that issue ranks as a
high severity. Alternative, the spelling problem might be buried deep in some
dialog, so the originator might assign a lower severity to that. If there is a
disagreement the triage is there to arbitrate. What about that million character
URL example? As the likelihood of anyone running into this is rather low, it’s a
low severity issue despite the fact that it causes a crash. Let’s take another
example. Suppose an adopter is developing a tool on top of WTP and requires a
hook of some sort in order to get their scenario working properly. Unless a
workaround is found or the hook is added, their scenario is dead in the water.
That deserves to be an S1. The problem with the bugzilla definition of the
severity levels is that they are too restrictive. Technically, asking for an api
hook under the current system would force the bug to be marked as an enhancement
request. There is no way for the originator to indicate the _actual_ impact of the
problem.
My proposal would
establish a concrete way to categorize the actual impact of the bug on the
originator that we can all talk about in a meaningful way. It establishes a
triage committee to arbitrate disagreements. And, finally, it re-emphasizes that
if a bug is targeted to a certain release, one can have a good degree of
certainty that it will be fixed in that release.
-
Konstantin
From:
wtp-dev-bounces@xxxxxxxxxxx [mailto:wtp-dev-bounces@xxxxxxxxxxx] On Behalf Of Arthur Ryman
Sent: Wednesday, February 22, 2006 7:13
AM
To: General discussion of project-wide or architectural
issues.
Subject: Re: [wtp-dev] Re-Inventing Hot
List
Kosta,
Thx for the suggestions.
Let's take a step back for a minute and review why we have a Hot Bug
process.
WTP has two main goals:
1) Provide tools for Web application developers.
2) Provide a platform for tool developers.
Both of these are very
important for the success of WTP. The Hot Bug process is aimed at goal #2.
There are many more
application developers than tool developers so the bugs that are important to
tools developers can easily get lost in the large bugzilla backlog. To put
things in perspective, there have been a total of 6700 bugs opened for WTP
to date, but only 200 of them are Hot Bugs. That's around 3% of the
total.
We need to treat tool
vendors, aka adopters, differently. WTP is the first Eclipse project, AFAIK, to
treat adopters specially. The current process needs to be refined over time. I'm
sure we don't have it right yet.
However, I
don't think we should rely on severity since I'd like us to use severity
correctly. Sometimes an adopter problem does not meet the criteria for Blocking
or Critical but is still very important. A Major problem could be enough for an
adopter to not use WTP. The meaning of Major is that some major function is
unusable. That major function might be exactly what the adopter
needs.
This means we need to use
another field. Priority is orthogonal to severity. Priority is our way of saying
what we are going to work on. If we mark a bug as P1 then it is
release-defining, i.e. we should delay the release if a P1 function is broken.
That translates into how we allocate our resources. We need to plan our work so
that we fix all the P1s first, then the P2s, etc.
To make this more concrete,
suppose there was a bug in our HTML editor that caused Eclipse to crash whenever
the HTML editor opened a document that contained a URL that was longer that
1,000,000 characters. That would be a Critical bug because it caused a crash,
but I would assign it a very low priority because it is a very extreme edge case
and would never affect a user.
In contrast,
suppose our server tools listed a server adapter for IMB WebSphere. I'd
immediately open a Hot Bug to get that corrected. From my point of view that
would be a P1, but the severity would be trivial.
Arthur Ryman,
IBM
Software Group, Rational Division
blog:
http://ryman.eclipsedevelopersjournal.com/
phone: +1-905-413-3077, TL
969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL
969-4920
mobile: +1-416-939-5063, text: 4169395063@xxxxxxx
|
"Konstantin
Komissarchik" <kosta@xxxxxxx>
Sent
by: wtp-dev-bounces@xxxxxxxxxxx
02/21/2006 09:36
PM
|
Please respond
to
"General discussion of project-wide
or architectural
issues." |
|
|
To |
"General discussion of project-wide or architectural
issues." <wtp-dev@xxxxxxxxxxx>
|
|
cc |
|
|
Subject |
[wtp-dev]
Re-Inventing Hot
List |
|
I’d like to start the
conversation about how we can improve the Adopter Hot List process going
forward. Here is a proposal to get us started. Feel free to poke holes in
it...
This
proposal places more emphasis on the severity field and establishes a strict
release exit criteria for high severity issues. A triage committee (composed of
PMC members or their designated replacements) would be established in order to
resolve disagreements over the severity and other issues.
- In order
to exit a release all S1 (Blocking Severity) and S2 (Critical Severity) bugs
targeted to that release have to be resolved. Additionally, there should not
be any untargeted/uninvestigated S1/S2 bugs. The triage committee can push
bugs of lower severity that are targeted to that release to the next release
if there isn’t sufficient time remaining in order to resolve
them.
- All S1/S2
bugs will be reviewed at the weekly status meetings. The severity as set by
the originator can be challenged. The triage committee will arbitrate and have
the power to assign a lower severity if necessary.
- Anyone can
submit a bug of a lower priority to the weekly status meeting agenda in order
to get the status or to request a targeting decision.
- A bug is
only targeted to a particular release when a dev has made a commitment to work
on it for that release.
- Once a bug
is targeted, that setting cannot be changed without the approval of the
originator or the triage committee. This applies to all bugs regardless of the
severity.
I
believe that if the above process is implemented and abided by than we no longer
need the Adopter Hot List. Thoughts?
-
Konstantin
_______________________________________________________________________
Notice:
This email message, together with any attachments, may
contain
information of BEA Systems, Inc., its
subsidiaries and affiliated
entities, that may be
confidential, proprietary, copyrighted and/or
legally
privileged, and is intended solely for the use of the individual
or entity
named in this message. If you are not the intended recipient,
and have
received this message in error, please immediately return this
by email and
then delete it.
_______________________________________________
wtp-dev
mailing
list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev
_______________________________________________________________________
Notice: This email message, together with any
attachments, may contain
information
of BEA Systems, Inc., its subsidiaries and
affiliated
entities,
that may be confidential, proprietary, copyrighted
and/or
legally privileged, and
is intended solely for the use of the individual
or entity named in this message. If you are not the intended
recipient,
and have received this
message in error, please immediately return this
by email and then delete
it.
_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev
