Hi Eric,
My vote is for the static approach. If I want to test eclipselink.jar
then I run the "test-against-eclipselink-jar" target, then I'm
confident about eclipselink.jar. Otherwise to test a specific set of
binaries I need to examine my environment before the test run (as I
understand it you would need to delete eclipselink.jar in order to test
the bundles, then delete the bundles to test the classes?).
I see the unit tests playing several roles:
- Unit testing the code
For this particular goal multiple approaches are valid.
- Verifying the installer bits (i.e. eclipselink.jar)
Based on my experience with OXM, JAXB, and SDO tests my position is
that we should use static testing. Before we specifically tested
against the eclipselink.jar for these tests there was an instance where
we failed to detect that the eclipselink.jar was being built without
some of the required classes. As our bleeding edge customers regularly
uptake nightly builds we need to ensure the jars are valid.
- Verifying the appropriate dependencies
We need to avoid any kitchen sink approaches to testing. At
one point the OXM layer became dependent on the JPA jars, this wasn't
caught because at the time the test frameworks included too many
unnecessary dependencies. The test frameworks should be set up with
minimal class paths.
-Blaise
Eric Gwin wrote:
Two
idealogical items were brought up this week regarding the test builds
(I've specifically added an assumed third (disallowing classes)):
1. What to test against:
- Most tests used to throw the kitchen sink into the classpath to
get them to work (eclipselink.jar, bundles, and classes
as well as all possible dependencies)
- SDO and MOXy always test nightly against the eclipselink.jar only
- To get the rest of the tests working, without a recompile of the
product, tests have been added to verify the existence
of eclipselink, the bundles, and the classes. Then one set is
used in the classpath (in order of preference):
eclipselink.jar, the bundles, or classes
- The same target is executed regardless of classpath.
- I included the bundle set in anticipation of testing against OSGi
platforms, and because we seem to be promoting the
targeted use of eclipselink as an added configuration (therefore we
will need the capability of certifying the bundles)
- Given the new directive for testing to not recompile the product it
seems, it can easily be extended to include
"tests should only be run against product jars" (eclipselink, or
bundles) when using ant (eclipse testing
remain unaffected).
2. Dynamic vs Static Testing
Rather than running a test target against any available set of
product classes (eclipselink.jar, the bundles, or class dirs), a
preference has been stated to use static targets for a configuration
set.
- The reason is to ensure that you are testing against what you
think you are. For example "test-srg-using-jar" would only
and always run the SRG against the product jar,
"test-srg-using-bundles" and "test-srg-using-classes" would only test
against the bundle jars and classes respectively.
- The down-side is that scripts would need to be focused for a
specific methodology, and couldn't be reused for multiple purposes.
To limit impact on testing and development I had been approaching the
reorg design utilizing a dynamic approach. I can see the benefit for
going to a static approach, but I need your input first.
I can also see a benefit to disallowing class testing, but the rigidity
may cause problems later. Certainly product-only defaults can be setup.
Irregardless of the outcome, I plan on continuing to test availability
first and report a missing dependency in the effort to increase
usability and robustness, as well as reduce debug time.
Anyone have strong preferences?
-Should the tests dynamically target the product using a single target,
or should configuration-specific static targets be used?
-Should testing against jars be enforced? If so does this include
testing jars (vs allowing test classes) (in some cases this is
already necessary, in others it is not)?
-Eric
_______________________________________________
eclipselink-dev mailing list
eclipselink-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/eclipselink-dev
|
