[platform-dev] Disabling test cases

[Rolf Theunissen <rolf.theunissen@xxxxxxxxx>]

Hi,

It is common practice to disable test because they are not stable. I don't want to discuss if that is good or bad practice. However, I think we can improve a bit on how we disable test cases.

It is common practice to comment test-cases out to disable them. Or, to add if statements such that test cases are skipped. As a result it is not obvious how many tests are currently disabled. In the first case (commenting out) the test is just not counted, in the latter case (if statement) the test-case always passes.

Many of the tests are now migrated to JUnit4. JUnit4 has support to ignore test-cases. When a test case or suite should be ignored it can be annotated with @Ignore. When a test-case should be ignored on one OS only, assume statements can be used.

https://stackoverflow.com/questions/1689242/conditionally-ignoring-tests-in-junit-4

The benefit of using @Ignore and assume is that test results get labeled as 'ignored'. This makes it obvious that many test are currently not run for a piece of code.

Best Regards,

Rolf