I agree that the manual testing process has served us quite well and I
am glad Pawel took the time to migrate it to the
new debug components of the CDT. I would like to make a minor
suggestion to the "manual test coordination page".
Our old process would simply require someone to indicate which part of
the testing he/she was volunteering for
in a release; however, there was no indication if the test was
I would like to suggest that we add some indicator for completion and
maybe even result summary.
For example, if I volunteer to test the
"CDT/cdt-debug-dsf-gdb#Launching" part, I would immediately add this
to the relevant signup page.
When I actually did the testing I would update to something like
- CDT/cdt-debug-dsf-gdb#Launching [complete: no bugs]
- CDT/cdt-debug-dsf-gdb#Launching [complete: bug 123456, bug 123457]
or, if individual completion is too tedious for some
- Marc [complete]
I think this would help in knowing the status of the testing effort,
without being much of a burden.
Opinions welcome, of course.
[mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Pawel Piech
Sent: Tuesday, January 06, 2009 5:25 PM
To: CDT General developers list.; Device Debugging developer
Subject: [cdt-dev] Manual Testing Process in CDT
In the DD project, the we have been using a process to manually
test each released milestone. It has been a very useful method for
assessing the quality of a release in areas where automated tests are
not available. It is also a volunteer-based effort where the real goal
is to document what is covered by testers, and thus creating more value
from the testing that is already being done.
After migrating DSF and DSF-GDB into CDT, I would like to
continue having this test cycle process at least for the new components.
To this end I have migrated the relevant wiki pages from DSDP/DD into
I started this by migrating relevant pages from DSDP/DD into CDT
wiki over the last couple of days, and I tried to do this in a way that
should allow any other components to incrementally participate in this
I first created place holders for a wiki page for each
component, which are at the bottom of the main CDT wiki page
<http://wiki.eclipse.org/CDT> . I then created the pages for the
incoming cdt-debug-dsf <http://wiki.eclipse.org/CDT/cdt-debug-dsf> and
components. On the component pages, I created two sections relevant for
testing: "Features" and "Test Procedures". The Features section simply
lists, under separate headings, the features present in the component.
Each of these sub-sections may contain a description of the feature, a
detailed listing of its components, links to N&N or user manual. In the
"Test Procedures" section I copied the actual step-by-step test
procedures that could be used to test various features of the component.
I also created a main manual test coordination page
<http://wiki.eclipse.org/CDT/Manual_Testing> . This page mainly
contains a template and links to test pages for each milestone and
release, which is to be tested. On the individual test pages, there is
a master-list of all the features in all the components
, from which each tester can choose features to test.
To increase the coverage (and value) of this testing effort, all
that is needed is a list of features in each component. Specific test
procedures are also useful, but IMO, they are not necessary. Given the
fact that most CDT contributors use CDT in commercial products, which
have their own test procedures and test cycles, it would be great if the
information from these product's test cycles found it's way back into
cdt-dev mailing list