|Change buildsystem to Gradle or alternatives [message #1769769]
||Thu, 03 August 2017 09:21
| David Graeff
Registered: April 2016
The pure maven based buildsystem seems to be too slow for the existing codebase. On my system a linux kernel with about 800k lines of code builds in 7:30min while ESH takes 20min on my Core i7 system with a DSL internet connection speed.
A very successful consumer of gradle in the java environment in my opinion is android. It is super easy to integrate unit tests as well as device tests. Gradle works with the common IDEs like Eclipse and IntelliJ IDEA.
Downloading dependencies is a different build step then building and testing whilst with maven it is all mixed together. With gradle I can download dependencies and then develop on the go, while mavens "mvn -o" never worked reliably for me.
There exists osgi gradle plugins according to my research and gradle allows maven dependencies.
Gradle builds a lot faster than maven, it appears to be smarter to decide what to build. Follow up incremental builds are possible in contrast to maven: If i just change a line in a test and want to rebuild, maven takes again 15min, while gradle would probably do this in seconds.
The gradle DSL is much easier to explain in the documentation than the pom.xml.
A gradle task could be used to create a new binding + test from a skeleton, instead of using the cmd/sh files that hide that super long maven command line.
Less useless output while building, maven can be very verbose.
Building a single binding + test instead of building everything if the build command is issued within a binding directory. This is what I would expect from a proper build-system actually.
Maven was published to the public 2004, gradle reached 1.0 in 2012 and maven feels really aged nowadays.
Feedback would be nice.
|Re: Change buildsystem to Gradle or alternatives [message #1769935 is a reply to message #1769793]
||Sun, 06 August 2017 14:33
| David Graeff
Registered: April 2016
The OSGi world is moving, too. Now there are "subsystem"s in the spec and the karaf specific feature file is not the only way to provision bundles. I understand, that the ESH core might stick to the maven solution for a while, although I recon it will be more difficult in the future to find people to contribute or maintain that solution.|
For ESH extensions I propose to change the build system though with the following goals in mind:
- Have the tests together with the extension.
- Independent if necessary. The extension directory builds without being embedded in the ESH git checkout.
- Works offline after initial dependency resolution
- Extensions can have own dependencies (and don't need to distribute embedded jars)
- Incremental builds
- Build process creates the feature file for karaf or other provisioning files (like a subsystem file)
1. Tests together with the extension
The current system is a little off and most tools that I know for creating test stub classes expecting the tests to be in src/test/java.
2. Independent extension directory
This would allow to develop an extension in an own repository, and easily setup a continuous integration or continuous delivery. ESH could split the main repository, so that extensions have their own issue area on github.
3. Works offline
This works awesome with gradle. I issue a build and dependencies are downloaded to the .gradle cache directory (and shared with following builds) and then it builds the jar. If I rebuild, the existing dependencies are used.
I'd like a build system with the same experience.
4. Extensions with own dependencies
OSGi allows this awesome modularity and at the same time the build system is centered around this monolithic target platform. A proper build system should allow an extension to declare it's own dependencies.
Right now: A lot of the target platform deps are in there because of extensions. Other extensions embed their dependencies, which is far from optimal as well, because people will forget to update those. Newer versions of libraries (like apache commons or guava) can't be introduced progressively, one extension at a time.
5. Incremental builds
At least the bndtools built a bundle immediately after saving a file and it is even deployed to the OSGi instance in a run/debug situation.
6. Build process creates the feature file for karaf
If I write a binding for ESH or OH2, I need to edit two additional files outside of my binding directory within the repository. One is the pom for maven, the other is the features file for karaf. Why is a feature file or subsystem file not generated and later collected by the build system.
I have seen Peter Kriens porting effort. I thing it is harder to accomplish the above goals with sticking to the manifest first approach. The bnd guys actually say the manifest file was never meant to be written by humans. Dependencies can be expressed in gradle or the bdn configuration directly.
This is only the user perspective, of course.
|Re: Change buildsystem to Gradle or alternatives [message #1770026 is a reply to message #1769793]
||Mon, 07 August 2017 15:07
| Kai Kreuzer
Registered: December 2011
> @Kai: Can you recall where I can find the discussion thread?|
There is no thread, it was endless coding and skype sessions...
Some outcomes were:
- Bndtools requires a workspace to be a flat list of projects from a single repo. There is no way to structure it in sub-components or across multiple repos. A setup like the openHAB IDE (which combines projects from ESH, openHAB Core, Addons1, Addons2, maybe Z-Wave and other repos) will not be possible at all. That's for me the biggest blocker.
- ESH should be first refactored to have proper API bundles that can be compiled against and which should be the only dependency when e.g. starting a binding development in a separate repo. This will be a rather huge task to accomplish.
- We would still want to get dependencies from Maven repos and publish our artifacts to Maven repos. Bnd support for that was not ready last year, it has been worked on meanwhile.
- Launching the runtime from within the IDE wasn't as easily possible as with the current setup - this is a main feature for daily development, though.
- The grass always looks greener on the other side, but the devil is in the detail. From my experience, it will be a huge effort to switch the build system and it will involve almost all developers. Looking at what things there are to do around ESH with what little manpower, it felt not to justify the effort, at least for now.
Regarding your points:
1. Unit tests already can be within the bundle itself. And I don't see it as a huge pain to have plugin tests as separate fragments.
2. We once already discussed that the extensions should be moved to a separate repository to allow independent versioning and releases. But this is unrelated from the build system as the example of openhab2-addons shows (which is a separate repo that builds against ESH artifacts).
3. We should investigate ways to get that done with Tycho, afaik this should be possible.
4. This is something we actually try to avoid. As ESH is not an enterprise application, but targeting embedded devices, size matters. Therefore we try to keep the number of dependencies as small as possible and do the best to make everyone using the same libs for certain purposes. The current setup actually helps us doing so.
5. Well, if developing in the IDE, PDE directly takes changed classfiles from the workspace - no need to repackage anything. Agreed, for remote debugging, the auto-deploy mechanism of bndtools is pretty nice.
6. I have no idea how well Karaf feature creation fits in bnd. For the moment, I don't think it is too much pain the way it is, especially as ESH is supposed to be "pure OSGi" and we only maintain Karaf feature information as a side project. I haven't heard anyone asking for subsystem artifacts so far and the overall relevance of subsystems seems to be limited in the OSGi community...
Powered by FUDForum
. Page generated in 0.02520 seconds