| Re: [cdt-dev] New DSF test structure |
|
This is really great!
The wiki to update is here: https://wiki.eclipse.org/CDT/cdt-debug-dsf-gdb#Automated_testing I don't expect Elena to do it :) but I won't have time until after M6. If someone else can do it, that would be appreciated. Thanks again Elena! From: laskava@xxxxxxxxx [laskava@xxxxxxxxx] on behalf of Alena Laskavaia [elaskavaia.cdt@xxxxxxxxx]
Sent: March 18, 2016 3:53 PM To: Marc Khouzam Cc: CDT General developers list. Subject: Re: [cdt-dev] New DSF test structure Thanks Marc.
If you not dealing with gdb tests you can skip the rest
Since injected version is a field variable I cannot use global overrides which was used before. So all version testing has to be done in test (of @Before)
not in @BeforeClass methods. Overhead of skipping a test is 20ms which is reasonable considered that running test takes at minium 0.5 second.
I can do a bit more re-structuring later to optimize more.
TODO: fail test if running in UI thread right away
2) You can run either individual test or any suite provided you did #1. By default parameters are "gdb,gdbserver", so if you don't change anything it willLike -Dcdt.tests.dsf.gdb.versions=gdb.7.7,gdbserver.7.7,gdb.7.11 in "VM Arguments" section of launch config on Arguments tab. So this is command separated list of gdb versions (without spaces) with "gdb" prefix for gdb local and gdbserver prefix for remote (note if you want to run all just use AllSuites instead) Developing new test cases
- If you adding test case in existing ones just make sure you use assume for the right gdb
If some functionality only available (or not) in certain gdb's you can use junit4 assume statement. Parent class provides that following functions for convenience assumeGdbVersionAtLeast("7.4") - test will only be run for gdb 7.4 or higher and ignored for the rest assumeGdbVersionLowerThen("6.9") - test will be run only for gdb which is stricly lower than 6.9 (which is 6.8 and lower) assumeGdbVersionNot("6.8") - skip the test for a specific gdb. You can also combine them if need be You can also assume for remote if you test only remote (or visa versa), i.e Assume.assumeTrue("Skipping non-remote", remote); If you need to change some setup function based on version there are some version comparision utils you can use. I only found 2 method then needed if version for setup - If you adding new test case make sure it extends BaseParametrizedTestCase and use Parametrized running (see others as example)
- And new test case into SuiteGdb (remote or local)
Implementation details:
if you looking for implementation details interesting classes are BaseParametrizedTestCase which defines getVersion which is what it injected as parameters,
AllSuite - how you can override parameters from a class (cannot do it if using Suite annotations)
TODO:
- I should put this test on the wiki somewhere
- Lots of tests are using sleeps which is bad for performance and random failures
- Lots of tests use similar harness which can be implemented as junit4 Rules
- We can make a smart default run which instead of running specific version of gdb will run all versions but only tests which won't be run in any higher version, i.e. all tests will be run once if most approprate gdb version
P.S. Thanks Marc for putting up my re-formatting (although it was not reformatted but white space removing tool but it did something awful in some cases). On Fri, Mar 18, 2016 at 11:00 AM, Marc Khouzam
<marc.khouzam@xxxxxxxxxxxx> wrote:
|