| FW: [dsdp-pmc] The Eclipse Project Dashboard |
|
Bjorn is not subscribed to the PMC list. Here
are his comments. We should use dash-dev for future comments, I think. From: Bjorn Freeman-Benson
[mailto:bjorn.freeman-benson@xxxxxxxxxxx] Martin, Doug, (and the DSDP PMC and dash-dev) 1. The project dashboard has been totally
changed recently, without any notice to committers. I liked the old dashboard,
and I had put a link to it on the dsdp/tm homepage. So my #1 feedback to Bjorn
is: if you are planning any changes to publicly visible URLs, let committer know what you're up to.
Don't we want to have open processes? I hate broken links like http://www.eclipse.org/projects/dashboard/dashboard_detail.php?project=dsdp.tm that
used to work fine. The project dashboard was a prototype (that's what the
big red box warning on the page was about) and was being misused by (among
other people) the press. The data was not valid. Rather than continue to tell
people that, I just removed it. It was a public beta. I waffle between two
positions: providing public betas and then changing them, or not providing any
information at all. We look forward to your input via Bugzilla and your help in
writing the dashboard code (it's all in the Project Dash CVS). 2. Projects should not define the metrics themselves
if they are publicly visible. The dashboard gets unusable if it's not totally
clear what's visible. Projects could be enabled to use the DASH databases to
make their own computations and publish them on their own homepages. But the
common dashboard should work the same for all projects, or it gets totally
confusing. 3. I agree that different metrics would be useful in
different phases of the project, but there should be
ONE common definition of what is visible. I think the best solution is to come up with a
community-defined single formula. The original formula was a pseudo-random
invention by me. Far better would be to have the community experiment with the
raw data and come up with a consensus about the formula or formulas. The old
prototype dashboard provided raw data for such experiments, but there was no
conversation about the formula other than to say "it's wrong". I'm
not sure what to do here. 4. Yes, being explicit about the formulas
used is important. I don't think that SQL statements are sufficient. There should
be some plaintext explanation of what's visible on a report. There was a whole page about how the old dashboard was
computed, and there were links from the dashboard pages, and yet people didn't
seem to find it. That page explained the formula, the raw data, everything. The
new project-defined or community-defined formulas can do the same. 5. Regarding the new metrics: 5a) dsdp.tm project is missing totally. 5b) I liked the metrics on mailinglist, newsgroup
and bugzilla activity, I'm missing those. I'm not sure that commits only is a
good indicator. We are working hard to
recreate the code that extracts that data. We would be happy to have your
assistance. We agree that commits alone is not a good measure. |