How to Eat the World with DSLs

Everybody repeats the popular quote by investor Marc Andreessen: “ Software is Eating the World”. This phrase describes how software is going everywhere these days and how it disrupts traditional businesses. Industries that seemed to be well-established and developed are taken over by startups with software in often just a couple of months. Think of how Netflix has disrupted the TV business or how Uber changed the world of taxis. Classic travel agencies are having a hard time competing with platforms like Airbnb. Even the automotive industry got disrupted by a startup you might have heard of called Tesla, mainly because of software. Speaking of the automotive world, did you know that an average modern car runs on around 100 million lines of code?

These are only some of the most popular cases. Software truly is everywhere and, in the future there will be only very few products in which software is not playing a big and important role.

Who Writes All That Code?

“Software everywhere” also means that software systems are getting bigger and more complex, simply because they do more. At the same time everything must be connected, which adds another dimension of complexity. Who writes, and more importantly, who maintains all that code?

The straight forward answer is, of course: software engineers. If you know how to program these days you will very likely find a good job because every company needs to write software. However, software is built for specific purposes. Programmers not only need to know how to write code, but also need to understand the domain for which the software is developed. On top of it all, the domain is not usually trivial. Ideally, you would have employees who are good at programming AND have a deep understanding of the business domain. Good luck finding such personnel!

In practice, we need to build teams composed of people with different strengths, but we need to be careful as the communication overhead increases with the number of individuals in a team. Actually, this number grows exponentially or, more accurately, as a combinatorial explosion. Always remember, "nine woman can’t make a baby in one month”.

In other words, we should be doing everything we can to minimize the number of people and maximize their productivity. Hiring only super motivated and talented domain experts that are also extremely good software engineers is definitely a good recipe. But, if you can't find enough of these talented individuals, you can do other things to improve the productivity of your team and minimize the communication overhead.

Tools To The Rescue

Professional people should use professional tools. For software engineering these tools are debuggers, compilers, code editors, profilers, and many more. Such tools are often combined in an Integrated Development Environment (IDE). These are generic tools for programmers with which they implement the software using a generic language such as Java.

That is all very nice for the coders, but how can we put the domain experts in the loop? They cannot write code. Do we really want them to write prose text about requirements and domain concepts and let the programmers translate that into code? Shouldn’t we try to allow them to participate more actively in the software development process? Maybe we can put the relevant code in a form they can at least read and understand, so they can reason about the actual software rather than about some outdated requirements document.

DSLs Can Bridge The Gap

A domain-specific language (DSL) is a programming language that is tailored for a particular problem domain and a particular group of people. DSLs are formal, so whatever you write using DSLs will have a specific meaning and can be understood by a machine. The notation of a DSL on the other hand is tailored towards the domain people, so it isn’t weighed down by all of the generic complexity of a programming language. Instead it offers powerful concepts to solve and describe domain problems.

Imagine a payroll software and all the laws and guidelines that require implementation. There is a ton of mathematical rules involved to compute a certain payroll. Think about how these rules differ between the various industries and how the rules change on an annual basis. Still the software needs to be able to recalculate a payroll from any point in the past, applying the rules valid at that point in time. I once did a workshop with a company that had such a product. In their case, the payroll experts wrote down all the needed formulas in Excel sheets and put text prose next to them to explain to the software engineers how to include them in the software system. The result was a sea of hundreds of fat Excel sheets full of bugs because the information was only semi-formal and not testable. The software developers would then take these Excel sheets and translate them to code (C# in that particular case). Bugs were only found and fixed in the resulting software and both the system and the Excel sheets were hardly maintainable anymore.

They consulted me because they had heard about DSLs and wanted to know whether they could help improve the communication in the team and make the ever changing rules of payrolls more maintainable. In a two-day workshop we designed a DSL that allowed the domain experts to write the Excel formulas (we reused the Excel syntax, as it was familiar) in a text file, tested them and even integrated them directly in the product. So they were actually writing code and they finally had a single-sourced solution for their highly evolving domain logic. Storing the formulas as text further helped with versioning and working closer with the software team. The DSL supported a handful of powerful concepts that were needed in that domain. For instance, we added the concept of declarative validity ranges. They allowed to annotate a certain formula with a start and an end date. In the past that information was littered across the code in the form of lengthy if-then-else cascades. These are very hard to grasp when you come back after some time and want to add another rule.

Besides the single sourcing which eliminated a lot of redundancy, they could finally look at the source of the system together and discuss and reason about it - no more misunderstandings. We used Eclipse Xtext to design the DSL, so they got a full-featured editor with content assist, error checking and so on.

With Xtext, such a DSL can be implemented in a very short amount of time. In addition, it is easy to enhance and maintain such a DSL over time. The framework supports the whole stack of what a DSL implementation needs and it even offers advanced editing support for various different platforms. You can ship your DSL as a trimmed down RCP app without all the complexity of a typical Eclipse IDE. Alternatively you can build an update site for developers to install and update the DSL editors. With Xtext’s new web editor integration you can even edit your DSL filesin any kind of web application. Imagine an admin tool for the payroll system where you can alter or add forms to the running system.

Summary

What should you take away from this article? If applied wisely, DSLs can dramatically improve the productivity of your team and the maintainability of software systems. The toughest part is definitely identifying the sweet spots for such powerful abstractions, such that the investment pays off. I like to think of DSLs as an extension of what you can do with general purpose programming: Framework developers usually create building blocks for application developers for the same reasons that a language engineer designs a DSL for non-coding but logically thinking team members.