Meeting:
EclipseLink test strategy discussion
Date:
2007/12/06
Attendees: David
Twelves, Mike Norman, James Sutherland, Chris Delahunt, Blaise Doughan, Gordon
Yorke, Mike Keith
Minutes
This meeting was
aimed at identifying the main goals that should drive creation of an overall
test strategy for EclipseLink.
Current
situation
- Existing test
frameworks (loosely based on components) are not always consistent across
the following areas:-
- Test packaging
hierarchy
- Set of
properties files that need to be modified before running test
suites
- Method of
executing test suites
- Test frameworks
on which tests are currently written
- The above are
largely a result of each component having different testing needs (eg. MOXy
currently has no requirement to run server or database
tests)
Goals for an
overall EclipseLink test strategy
- A consistent user
experience should be provided for contributors wishing to write and exercise
tests across EclipseLink components.
- A contributor
needs to be able to easily add and run tests when submitting a patch.
We need to give them the power to easily be able to do
that.
- To achieve the
above, components should align with a standardized EclipseLink test
strategy.
- Location of ant
scripts and other utilities should be standardized in SVN.
- Standard
configuration of tests through property files, where
possible.
- Standard method
of executing component test suites via high-level ant
tasks.
- SRG/LRG test
suites should exist for each component, which can be rolled up to a
high level SRG/LRG for the product as a whole.
- Test output
format should ideally be standardized
- It is acknowledged that migration
of all the existing test frameworks to newer platforms (eg.
JUnit4) would be a significant amount of work. This is particularly
true for some of the ORM tests in the foundation component.
However, this is thought to be acceptable given that
- It is likely
that individual contributors will typically focus on one component
- Test
creation guidelines can be documented for each component. For
example, contributors will be encouraged to create JUnit tests using JPA
API, rather than creating new tests in the older test framework in the
Foundation component.
Next
steps
- New backlog
item: One committer to develop a proposal for a consistent test strategy
across components, addressing items 1-5 above. This is currently
unstaffed.
- This proposal
should also include the developer test process to be followed for writing
and running tests.
- Once this is in
place, each component area needs to examine what work is required in each
area to conform to this proposal. Each component should also document
any differences from the standard test process. This will include
capturing required work items and detailed estimates.
