[tycho-user] Testing plug-in's without using single test-bundles

[Paolo Bazzi <paolo@xxxxxxxxx>]

I'm looking for possibilities to test a set of eclipse plugins without 
having to use a single bundle for each plugin which is tested. Currently 
I have a PDE build running for an Eclipse product (about 70 plugins, 
features and a product). All the unit tests for all plugins are 
contained together in a single plain-java project, having an 
Eclipse-project reference to all plugins in order to be able to 
instantiate the classes and run the tests. This setup is no longer 
working as I converted the PDE build to maven tycho, since the plain 
java project misses all target platform projects. I'm not executing real 
OSGI plugin tests, but some tests require having the core eclipse 
classes like IProgressMonitor contained in the classpath, since I use 
those eclipse runtime interfaces also in own method signatures.

After setting up the new maven tycho build successfully I tried a few 
possibilities to get the tests running again:

1) Converting the plain-java test project into a plugin test project
Disadvantages:
- In order to be able to test classes in internal packages, I have to 
export every single package with x-friend:<my.test.plugin> notation and 
have to repeat this procedure for each new tested package

2) Adding a second source folder in each plugin and move the tests into 
the corresponding plugin
Disadvantages:
- Tycho seems to use build.properties to include the necessary source 
folders for the compile step. Since both the src/main/java and 
src/test/java needs to be registered as source folder, the real classes 
and the test classes are mixed up in the target/classes output folder 
and finally contained in the plugin's JAR file. I did not find a way to 
configure tycho to use src/main/java as sourceDirectory and 
src/test/java as testSourceDirectory.
- Tycho executes the unit tests only, if the package type is 
"eclipse-test-plugin"
- Sonar seems not to recognize tests executed in this way (I did not 
spend a lot of time trying to solve this issue, maybe there is an easy 
solution for this point)

3) Add the necessary eclipse target platform plugins as plain-maven 
dependency to the plain-java test project
Disadvantages:
- The target platform information is duplicated, once in the target 
platform for the tycho build, and once in the maven dependency list of 
the test project (executed with plain maven-surefire)
- The target platform bundles are deployed twice in the artifactory, 
once as target platform p2 archive, and once as maven dependencies 
(plugin+POM)

4) Adding a test fragment for each plugin (this seems to be the usually 
chosen possibility)
Disadvantages:
- Needs a huge effort (> 70 plugins, >4500 unit tests), so I would need 
to add about 70 new fragments and split up all tests.


At the moment, possibility 3) seems the most reasonable for me... any 
suggestions? otherideas?

Regards,
Paolo