Eclipse Paho and the Challenges of Project Maintenance
Eclipse Paho is quite different from other Eclipse Foundation projects in that it’s a collection of libraries and utilities designed specifically for use with the MQTT protocol, rather than a single deliverable entity. There are separate GitHub repositories for each sub-project, including MQTT client libraries in various programming languages: C, C++, Java, Python, Go, and Rust, among others. Each repository is typically the responsibility of a separate committer, so the opportunities for collaboration arise out of the project’s intent: to help foster a community around MQTT.
Before I get into the challenges we’re having with project maintenance and how you can help, here’s some additional background and insight into MQTT and Eclipse Paho.
Eclipse Paho Has Evolved in Alignment With MQTT
MQTT was created in the late 1990s to allow large networks of IoT devices to be built. Eclipse Paho was created in 2011 by IBM and started with Java and C client libraries. I’ve been involved from the beginning, contributing the C client, and then as project leader since 2014.
Since Eclipse Paho was created, MQTT has evolved from a specification published by IBM to two OASIS standards: MQTT Version 3.1.1, published in 2014, and MQTT Version 5.0, published in 2019. These advances have brought MQTT to new audiences and increased its popularity (Figure 1).
Figure 1: Google Trends Chart Showing Increase in MQTT Popularity
Apart from new libraries for additional programming languages, most of the enhancements to Eclipse Paho over the last decade have been implementations of standardized MQTT versions. The latest revision, MQTT Version 5.0, contained significant improvements, and the implementation effort associated with these improvements is significant. Eclipse Paho currently supports multiple languages and MQTT versions (Figure 2).
Figure 2: Eclipse Paho Support for Languages and MQTT Versions
MQTT Standardization Is Ongoing
MQTT-SN is a variant of MQTT that was designed for non-TCP networks, originally ZigBee and UDP. It can extend the reach of MQTT-style messaging to WANs and mesh networks, and is being standardized at OASIS. Eclipse Paho already has MQTT-SN implementations and I expect they will be enhanced to conform to the new standard.
Sparkplug is a specification for MQTT topic names and message contents to enable easier interoperation of industrial applications. Eclipse Tahu is the project for SparkPlug implementations.
The Challenges of Life as an Open Source Maintainer
Even without adding any major new functionality to Eclipse Paho, staying ahead of issues and pull requests (PRs) on a popular project is a constant workload. Happily, this is a side effect of being successful; if you have no issues, it’s likely you have no users. I do find that the faster and more comprehensively you respond, the more issues tend to arrive. These can be simple requests for help or requests to add new functions and fix bugs. Overall, my experience maintaining the C client is rewarding, but it’s also challenging, and sometimes exhausting.
Through issues that are created, hearsay, and my experience working at IBM, I know some of the Eclipse Paho client libraries are used by major companies, either in software development kits (SDKs), or in solutions. Every one of these projects, with very minor exceptions, seems to ignore the question of support for their essential open source components. I’m sure this aspect wouldn’t be ignored if the components were purchased. Is this due to their demonstrated reliability? Or is it because there is no apparent way of paying?
I feel there are many projects in danger of ending up in the position shown in this image from xkcd.com.
Most Eclipse Paho Team Members Volunteer Their Time
When Eclipse Paho was first created, the majority of committers came from IBM. As time went on, due to the project’s visibility, other libraries and utilities were contributed and their maintainers joined the project. Most of these maintainers were working in their free time. This is even more true today. The current roster of Eclipse Paho committers can be found here. You’ll see that most of the work is being done by individuals on their own time. I retired from IBM in 2019.
I recently held an Eclipse Paho team meeting at 19:00 on a Friday evening because I had to find a suitable time slot to accommodate team members from the UK, U.S., and New Zealand. But it was fun. It’s been a great pleasure to collaborate with, and learn from, Eclipse Paho contributors from all over the world. The enthusiasm for MQTT and its ability to enable creative solutions gives me a warm feeling. This collaboration is one of the main reasons I’ve continued to work on Eclipse Paho.
Eclipse Paho Needs More Help
Fortunately, I’ve been able to attract sponsors to the project since I left IBM. HiveMQ sponsors general project work, and KeySight sponsors specific improvements to the C client through the GitHub Sponsors program. These sponsorships don’t cover the full amount of time put into Eclipse Paho, and represent just a small fraction of the benefits consumers of the client libraries have accrued.
Over the last two years, I’ve had discussions with IBM, Amazon, and Microsoft, among others, about supporting Eclipse Paho as they use one or more components. Unfortunately, most of these discussions have not resulted in significant contributions.
I’ve also had discussions with a number of individuals who find the process of becoming a committer too onerous. Fortunately, most are willing to sign the Eclipse Contributor Agreement (ECA) so they can open a PR, but some are not. Some people would like to help by managing PRs and issues for a short time, say three to six months, then move on. The “overhead” of becoming a committer is too much for them.
Get Involved in Eclipse Paho Today
If anyone has any thoughts on how to encourage more people to become committers, or contribute in other ways to help the project continue, I would very much like to hear them.
Contributions can take several forms:
- Become a committer
- Create a PR
- Open an issue
- Provide financial support
We welcome everyone with an interest in getting involved in Eclipse Paho. You can find more details about the options listed above and provide comments here.