I appreciate the energetic discussion on
the wtp-dev list, but we need to ensure that we’re achieving the project’s
aims with this critical support mechanism. I describe a possible approach
below; let’s review where we are on this discussion on the Tuesday call.
I believe we need to retain control over
what goes on the hotlist, although mastering the data in Bugzilla, automating
generation of the hot list on the WTP site, and getting “please add …”
noise off wtp-dev are all worthwhile goals.
Here’s one process that addresses
these concerns without wholesale revisiting of the hotlist concept:
- Create
a bug to represent the “Hot List Suggestion Box” (per release).
Anyone can add a comment requesting a new addition to the hot list, discuss
existing requests, indicate a suggestion has been accepted, record a
decline, etc.
- One
person is designated “promoter”; when a decision is reached to
promote a suggestion to the hot list, the promoter adds a comment to the
defect with a standard syntax.
- The
WTP adopter page hotlist report searches the comments in this bug and uses
these special entries to produce the hot list report. We can augment the current
report with additional columns for requesting adopter, etc. as we see fit
by augmenting the form of the entries and the report generation logic. Note
that only the promoter’s comments are used, so there is no
possibility of an “accidental” promotion by others.
The only downside I see with the above is
that a standard Bugzilla query cannot be used; if we want to achieve that
without giving up control over promotion, then we can use a proxy defect / hotlist
entry in lieu of comments and a text converter:
- (Alternative
approach) When a decision is reached to promote a bug to the hotlist, the
promoter creates a proxy defect that depends on the original bug, titling
it “[hotlist] <original title>”.
- To
view the hotlist bugs, you search for [hotlist] in the title *and* the promoter as submitter; this
search can be done in Bugzilla and no special postprocessing is required
to view the results.
From:
wtp-dev-bounces@xxxxxxxxxxx [mailto:wtp-dev-bounces@xxxxxxxxxxx] On Behalf Of David M Williams
Sent: Wednesday, February 22, 2006
10:50 AM
To: General
discussion of project-wide or architectural issues.
Subject: RE: [wtp-dev]
Re-Inventing Hot List
Sounds like we have a ways to go to resolve these
issues.
I,
for one, would like to see this moved off the wtp-dev mailing list, and
utilitize bugzilla better.
As
it it, being on the dev list has effect of either wasting time, or watering
down the importance, since
when
someone says "please add xxxx to hot list" ... EVERYONE has to click
on it to see if
something
they should pay more attention to. Or .. else, ignore it, and wait for someone
else
to
tell them what is on their top priority list.
It
even seems the "seperate list" queries are often out of date, so,
hard to trust them.
|
"Konstantin
Komissarchik" <kosta@xxxxxxx>
Sent
by: wtp-dev-bounces@xxxxxxxxxxx
02/22/2006 12:04 PM
|
Please
respond to
"General discussion of project-wide or
architectural issues." <wtp-dev@xxxxxxxxxxx>
|
|
|
To
|
"General
discussion of project-wide or architectural issues."
<wtp-dev@xxxxxxxxxxx>
|
|
cc
|
|
|
Subject
|
RE: [wtp-dev] Re-Inventing Hot List
|
|
Here are the problems that I see with the process
as it currently exists:
1. There is no gate to the hot list. Nothing to control what
gets added. I’ve seen issues that have minimal impact and documented
workarounds be tracked on the hot list.
2. We are under-utilizing the priority and severity fields by
focusing exclusively on the hot bugs at the status meetings.
3. The hot bugs are not marked in bugzilla, making it very
difficult to construct queries around them.
4. There is no one to arbitrate disagreements over the
categorization.
5. And finally, the hot list creates at least an appearance
of an “exclusive club” that’s fundamentally against the ethos
of open source. An S1 coming from a WTP end user should (in my opinion) be
resolved with a higher urgency than an S4 coming from a major adopter.
I realize that my proposal uses the severity field in a novel
way, but think about it this way. Assignment of the initial severity is in the
hands of the bug originator. It’s a way for them to rank how much an
impact this is having on them. It might be a spelling issue, but if the impact
of that from the originator’s perspective is that they will not ship
until the problem is fixed, then that issue ranks as a high severity.
Alternative, the spelling problem might be buried deep in some dialog, so the
originator might assign a lower severity to that. If there is a disagreement
the triage is there to arbitrate. What about that million character URL
example? As the likelihood of anyone running into this is rather low,
it’s a low severity issue despite the fact that it causes a crash.
Let’s take another example. Suppose an adopter is developing a tool on
top of WTP and requires a hook of some sort in order to get their scenario
working properly. Unless a workaround is found or the hook is added, their
scenario is dead in the water. That deserves to be an S1. The problem with the
bugzilla definition of the severity levels is that they are too restrictive.
Technically, asking for an api hook under the current system would force the
bug to be marked as an enhancement request. There is no way for the originator
to indicate the _actual_ impact
of the problem.
My proposal would establish a concrete way to categorize the
actual impact of the bug on the originator that we can all talk about in a
meaningful way. It establishes a triage committee to arbitrate disagreements.
And, finally, it re-emphasizes that if a bug is targeted to a certain release,
one can have a good degree of certainty that it will be fixed in that release.
- Konstantin
From:
wtp-dev-bounces@xxxxxxxxxxx [mailto:wtp-dev-bounces@xxxxxxxxxxx] On Behalf Of Arthur Ryman
Sent: Wednesday, February 22, 2006 7:13 AM
To: General discussion of project-wide or
architectural issues.
Subject: Re: [wtp-dev] Re-Inventing Hot List
Kosta,
Thx for the suggestions. Let's take a step back for a minute and review why we
have a Hot Bug process.
WTP has two main goals:
1) Provide tools for Web application developers.
2) Provide a platform for tool developers.
Both of these are very important for the success of WTP. The Hot Bug process is
aimed at goal #2.
There are many more application developers than tool developers so the bugs
that are important to tools developers can easily get lost in the large
bugzilla backlog. To put things in perspective, there have been a total
of 6700 bugs opened for WTP to date, but only 200 of them are Hot Bugs. That's
around 3% of the total.
We need to treat tool vendors, aka adopters, differently. WTP is the first
Eclipse project, AFAIK, to treat adopters specially. The current process needs
to be refined over time. I'm sure we don't have it right yet.
However, I don't think we should rely on severity since I'd like us to use
severity correctly. Sometimes an adopter problem does not meet the criteria for
Blocking or Critical but is still very important. A Major problem could be
enough for an adopter to not use WTP. The meaning of Major is that some major
function is unusable. That major function might be exactly what the adopter
needs.
This means we need to use another field. Priority is orthogonal to severity.
Priority is our way of saying what we are going to work on. If we mark a bug as
P1 then it is release-defining, i.e. we should delay the release if a P1
function is broken. That translates into how we allocate our resources. We need
to plan our work so that we fix all the P1s first, then the P2s, etc.
To make this more concrete, suppose there was a bug in our HTML editor that
caused Eclipse to crash whenever the HTML editor opened a document that
contained a URL that was longer that 1,000,000 characters. That would be a
Critical bug because it caused a crash, but I would assign it a very low
priority because it is a very extreme edge case and would never affect a user.
In contrast, suppose our server tools listed a server adapter for IMB
WebSphere. I'd immediately open a Hot Bug to get that corrected. From my point
of view that would be a P1, but the severity would be trivial.
Arthur Ryman,
IBM Software Group, Rational Division
blog: http://ryman.eclipsedevelopersjournal.com/
phone: +1-905-413-3077, TL 969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL 969-4920
mobile: +1-416-939-5063, text: 4169395063@xxxxxxx
|
"Konstantin
Komissarchik" <kosta@xxxxxxx>
Sent by: wtp-dev-bounces@xxxxxxxxxxx
02/21/2006 09:36 PM
|
Please
respond to
"General discussion of project-wide or
architectural issues."
|
|
|
To
|
"General
discussion of project-wide or architectural issues."
<wtp-dev@xxxxxxxxxxx>
|
|
cc
|
|
|
Subject
|
[wtp-dev] Re-Inventing Hot List
|
|
I’d like to start the conversation about how we can improve the Adopter Hot
List process going forward. Here is a proposal to get us started. Feel free to
poke holes in it...
This proposal places more emphasis on the severity field and establishes a
strict release exit criteria for high severity issues. A triage committee (composed
of PMC members or their designated replacements) would be established in order
to resolve disagreements over the severity and other issues.
- In order to exit a release all S1 (Blocking
Severity) and S2 (Critical Severity) bugs targeted to that release have to
be resolved. Additionally, there should not be any
untargeted/uninvestigated S1/S2 bugs. The triage committee can push bugs
of lower severity that are targeted to that release to the next release if
there isn’t sufficient time remaining in order to resolve them.
- All S1/S2 bugs will be reviewed at the weekly
status meetings. The severity as set by the originator can be challenged.
The triage committee will arbitrate and have the power to assign a lower
severity if necessary.
- Anyone can submit a bug of a lower priority to
the weekly status meeting agenda in order to get the status or to request
a targeting decision.
- A bug is only targeted to a particular release
when a dev has made a commitment to work on it for that release.
- Once a bug is targeted, that setting cannot be
changed without the approval of the originator or the triage committee.
This applies to all bugs regardless of the severity.
I believe that if the above process is implemented and abided by than we no
longer need the Adopter Hot List. Thoughts?
- Konstantin
_______________________________________________________________________
Notice: This email message, together with any attachments, may contain
information of BEA Systems, Inc., its subsidiaries
and affiliated
entities, that may be confidential, proprietary, copyrighted
and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.
_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev
_______________________________________________________________________
Notice: This email message, together with
any attachments, may contain
information of BEA Systems,
Inc., its subsidiaries and affiliated
entities, that may be confidential, proprietary,
copyrighted and/or
legally privileged, and is intended solely for the
use of the individual
or entity named in this message. If you are not
the intended recipient,
and have received this message in error, please
immediately return this
by email and then delete it.
_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev
