Building IoT applications with OSGi and Eclipse Concierge

Early years

When the OSGi Alliance released their first specifications in 2000, they were clearly targeting embedded devices. The motivation for OSGi can be best summarized with two key observations:

1. software for embedded systems is dynamic (components can come and go);
2. since these devices have long lifetimes, software needs to be maintained efficiently without constantly disrupting user operations.

The latter has been implemented through bundles, modular units of Java code that can be loaded, unloaded, and updated individually while only affecting other bundles that directly depend on them. The former lead to the introduction of services: lightweight entities that separate interface from implementation, are maintained in a central service registry, and thereby reduce the coupling between modules. Services are complemented by an event system to notify consuming bundles about changes in service availability.

The popular devices at that time included, television set-top boxes, gateways for the “connected home”, personal digital assistants (PDAs), or the devices that people, unsuspecting of what the future would bring, were calling "smart phones". In 2006, still in the days of Sharp Zaurus’ and Nokia Communicators, I became increasingly frustrated with the growing footprint and the sedate runtime behavior of the popular OSGi framework implementations and started the work on Concierge as part of my master’s thesis. It was my attempt to implement the OSGi R3 specifications in as little code as possible, ultimately manifesting itself in an 86 kiB JAR file. At the same time, I reduced the complexity of the code and exploited optimizations that made the framework run significantly better on the JVMs that were commonly used on mobile and embedded devices. These often lacked full JIT compiler support. I wanted the benefits of OSGi for embedded software, without the compromises.

Where we are today

Fifteen years later, the average CPU in an embedded device could pretty much compete with what was a decent desktop machine in the infancy of OSGi. Sensors have become affordable and omnipresent, like in mobile phones for example. The trend of interconnecting the different islands of smart devices and turning them into smart ambient environments has continued and resulted in an unprecedented popularity of the Internet of Things (IoT). Even after all of this, the key observations that initially motivated OSGi haven’t changed. On the plus side, IoT deployments are still dynamic, if not even more due to their inherently distributed character, and challenging to maintain. Unfortunately, OSGi framework implementations still perform slower than desired on embedded devices and are notoriously difficult to set up.

Eclipse Concierge

This has motivated the Eclipse Concierge team to pick up the old R3 implementation and turn it into a modern R5 implementation without compromising the design principles or losing the embedded devices focus. Now that we’ve reached the point where we are passing all relevant compliance tests, we are releasing the first major version of the new Concierge. This release is called 5.0.0 to align with the revision of the OSGi core specifications that it implements. The best news is that despite the significant growth of the standard in terms of functionality and APIs, the footprint of Eclipse Concierge R5 is still below 250 kiB for the JAR without debug symbols and 330 kiB with debug symbols. We have also measured a significant performance advantage over other frameworks, in terms of startup time and service registry performance. We will continue to optimize Concierge in the future.

Besides keeping the code fast and lean, we have also focused on making Concierge as easy to use as an OSGi framework can get. Creating a Concierge deployment mainly involves copying the framework JAR and the bundles to the device and then creating an init.xargs file, similar to what Knopflerfish uses. The content of the file can consist of Java property declarations or directives like install, start, or istart (install and start) of bundles. In order to make the creation of the startup file easier and less verbose, Concierge supports both variable substitution against declared system variables and wildcards in file names. You can see this in Listing 1 which uses substitution to factor out the common URL and a wildcard to make the deployment resilient against switching to different nightly builds of the shell bundle.

Listing 1

# xargs sample file to load some Felix bundles
# uncomment for clean starts
# -Dorg.osgi.framework.storage.clean=onFirstInit
# use a separate profile
-Dorg.eclipse.concierge.profile=felix

# repos to load bundles from
-Drepo=https://archive.apache.org/dist/felix/
# load bundles
-istart bundles/org.eclipse.concierge.shell-5.0.0.*.jar
-istart ${repo}/org.apache.felix.scr-1.8.0.jar
-istart ${repo}/org.apache.felix.eventadmin-1.4.2.jar
-install ${repo}/org.apache.felix.metatype-1.0.12.jar
-install ${repo}/org.apache.felix.configadmin-1.8.4.jar
-level 1 -start ${repo}/org.apache.felix.metatype-1.0.12.jar
-level 2 -start ${repo}/org.apache.felix.configadmin-1.8.4.jar

Besides command line and xargs files, Concierge can be embedded into Java applications and launched through the standardized launch API and by tools like bnd.

Despite the minimalistic design, there is one optional service that Concierge provides out of the box as part of the framework, which is the log service. The rationale behind this is that debugging and tracing on embedded systems is often particularly hard due to the headless nature of many devices. Providing the log service in the framework allows the Concierge framework implementation to use the same log as the applications. This makes it easier to correlate behavior observed in the application to OSGi-specific events like package resolution or service registry activity. In turn, also mitigating some of the commonly observed pain points when initially adopting OSGi in applications. On the contrary, when operating in closed environments where full OSGi specification compliance is not paramount, certain functionality like support for legacy bundles (bundle manifest version 1) can be statically disabled to further reduce the footprint. This is enabled by a micro-kernel design which aims at making such legacy support features optional. In fact, it is even possible to generate a Concierge framework that operates exclusively on the R5 generic requirement/capability model by removing even support for bundle manifest version 2. Finally, the parsing of xargs files is another micro-service that can be removed without any modifications to the code.

With the 5.0.0 release of the framework, we also ship a set of bundles that include, e.g., a minimal command line shell for interaction with the framework or implementations of package admin and startlevel services for legacy (R4) bundles which depend on them. We plan to soon add a RESTful interface to manage the framework and a small event admin implementation, as well as bundles, to interact with the hardware capabilities of select popular embedded devices.

The near future

Mastering the first release is a crucial first step for our project, but we will face another challenge soon, which is to catalyze the adoption among other Eclipse IoT projects and elsewhere. We have already successfully brought up Kura on Concierge to illustrate the advantages and applicability of our implementation, but much more is needed to achieve long-term success. The next step will be to grow our community and bring Concierge to the developers of applications for IoT and embedded systems. To achieve this we’re already working with students from UNICAMP in Brazil, as part of the Facebook Open Academy program, which aims at exposing CS students to the open source process. We are looking forward to some exciting new demos and features that the students will be working on until the end of the year.

More information

Visit the Concierge website and download the 5.0.0 release here.