What is a DSL?

With Kotlin the term ‘Kotlin DSL’ usually refers to DSLs built in Kotlin using specific Kotlin features (as discussed in ‘Kotlin DSLs’), this page, however, is about DSLs in general.

• Introduction:
  • DSL: The general definition
  • DSL vs ‘a DSL’
• The Types of DSL:
  1. External DSL
  2. Internal Detached DSL
  3. Internal Augmentation DSL
• Detached vs Internal DSL: A continuity?
• Language Augmentation in general Vs An Augmentation DSL
• Conclusion: DSL types can be quite different

Introduction:

DSL: The general definition

The acronym ‘DSL’ stands for ‘Domain Specific Language’. A ‘Domain’ being effectively a particular application or field of expertise. ‘Specific’ is self-explanatory, but what exactly is meant by ‘language’ does warrant further exploration later.

Contrasting with ‘general purpose languages’ which attempt to allow for solving any programming problem, a DSL can be purpose designed for a specific ‘domain’ or a specific type of problem.

The term DSL is a broad term, covering some different types of DSLs.  Sometimes people use the term DSL when they are referring to a specific type of DSL, resulting in the term appearing to mean different things in different contexts.

Martin Fowler (who has written books on DSLs that can be very worthwhile reading) described two different main types of DSL, External and Internal, which differ by how they are implemented.  Next, Martin Fowler explains that the second Implementation type, Internal, itself provides two types of DSL, the Internal Mini-language and the Internal Language Extension. This results in a total of three different types of DSL.

DSL vs a DSL

There is a sematic difference between ‘language’ and ‘a language’.  Consider the two phrases “he likes to use language which is considered antiquated’ and “he likes to use a language which is considered antiquated”.  The first suggests vocabulary within a language e.g. antiquated words within the English language, the second suggests use of a language such as ancient Greek or Latin.

Similarly. ‘domain specific language’ can be though of a terms within a language which are specific to a particular domain’ while ‘a domain specific language’ suggests an entirely new language developed for use in a specific domain.

The Types of DSL: External,  Detached & Augmentation

DSLs come in two main forms: external and internal. An external DSL is a language that is parsed independently of the host general purpose language: good examples include regular expressions and CSS: Martin fowler.

These are DSLs like SQL, or HTML. Languages only applicable within a specific domain (such a databases, or web pages) which are stand-alone languages, but with functionality focused on that specific field or domain, and too limited to be used as a general purpose language.  Implementing a DSL as an external DSLs enables the DSL to be unrelated to the programming language used to write the DSL.

Externals DSLs generally have the same goal as a Detached DSL, but built using a different implementation method.

The key advantage for external DSLs is that by being independent of any base language, they work unchanged with any general language.  So SQL is the same DSL when working with Java, Python, Kotlin or C#.

The first problem with independent DSLs is that the task written using the DSL often also need some general purpose language functionality. So the task will then be written in two languages.  A general purpose language for part of the solution, and a DSL for another part.  The project requires two different languages.

The second problem with independent DSLs is that the features of the general purpose language are not accessible from within the DSL. This means the DSL may need to duplicate features already available in any general purpose languages. Such duplicated features are generally inferior to those in general purpose languages.  E.g. numeric expressions in SQL are not as powerful as most general purpose languages, and there is often a syntax change from the general purpose language.

2. Internal Detached DSLs

When people talk about internal DSLs I see two styles: internal mini-languages and language enhancements.

An internal minilanguage is really using an internal DSL to do the same thing as you would with an external DSL.  Source: Martin Fowler.

Unlike an external DSL, you are limited by the syntax and programming model of your host language, but you do not need to bother with building a parser. You are also able to use the host language features in complicated cases should you need to.

Martin Fowler

Under Martin Fowlers definition, a detached DSL is the first of two types of Internal DSL.  These Internal Detached DSLs, like External DSLs,  are building their own ‘mini-language’ for a specific domain.  Detached DSLS are building ‘a domain specific language‘ as opposed to ‘domain specific language’ vocabulary for an existing language.  With a Detached DSLs, the new stand-alone language is created within an existing language. To achieve being a standalone language,  the DSLs needs to be separated or ‘detached’ from the host language.  Even if such a language is ‘fully-detached’ from the host language, it is will normally be the case that some host language syntax is available from within the DSL.  In all cases, the rules and syntax of the DSL will be shaped by what can be built within the framework of the host language.

This Detached DSL is the type of DSL usually referred to in the discussion of Kotlin DSLs, and of Gradle build files are an example of a Groovy Internal, Detached DSL.

As the goals are the same as External DSLs in creating what can be seen as a standalone language, these DSLs ideally require little understanding of the host language by those using the DSL.  So build.gradle files require, at least in theory, almost no understanding of the Groovy language, or perhaps more realistically, an understanding of only a tiny subset of the host language.  Kotlinx.html is a Kotlin example of this type of DSL built within Kotlin, and the actual Kolinx.html syntax can seem very different to regular Kotlin syntax, even though all code is actually Kotlin.

3: Internal Augmentation DSL.

The alternative way of using internal DSLs is quite different to anything you might do with an external DSL. This is where you are using DSL techniques to enhance the host language. A good example of this is many of the facilities of Ruby on Rails.  Martin Fowler.

Why build a complete language if you can just add features to an existing language?  This third type of DSL no longer has the goal of creating a standalone language. It is ‘domain specific language’ more as a parallel to a set of jargon words for a specific domain can be used in a conversation that is based in English.  The jargon provides new language, but the conversation overall is still in English.   To understand the conversation, you need to know English as well as the specific jargon.  Code using an augmentation DSL will still also make use of the host language.   The program is still seen as in the original language, but using some additional definitions specific to the augmentation DSL. The goal of the augmentation DLS is to add new vocabulary or capability to an existing language, and this makes Augmentation DSLs quite different to the previous DSL types. Instead of an entire stand alone new language, the result is an extension or augmentation to an existing ‘host’ language.  Effectively extending the power of the original host language to have new vocabulary and perhaps also new grammar. This enables the simple and concise expression of ideas and concepts from a specific domain while continuing to use the host language. The augmentation is to be used in combination with the power and flexibility of the host language, which allows for more general areas of a programming in combination with programming for the specialist domain.

Such augmentations still require users to know the host language, but  provide a more homogenous solution than the combination of a stand-alone language with a general purpose language.   For example, while a Python program can build SQL commands to send directly to an SQL database server, an augmentation to python such as SQLAlchemy allow the same power as the SQL language, all within the general syntax of Python.

Detached vs Augmentation DSLs: A continuity?

Both Detached DSLs and Augmentation DSLs are build inside an existing language, and the same set of language features can be used to build either type of DSL.   It is only the goal that is different.  Build a syntax that feels detached from the host language,  or build a syntax that integrates with the host language.

The reality is not every detached DSL is fully detached from the host language, and many do require knowing the host language.

There is a clear test for a fully Detached DSL:  If the DSL can be read, or written, by people with knowledge only of the DSL without needing knowledge of the host language, then it is a fully detached language. Gradle Build files are an example of a internal detached DSL that passes this test, as you can write build files without knowing  the host language (which can be either Groovy or Kotlin).

However,  just because the DSL syntax can be used fully detached from the host language, does not mean actual code in the DSL always will be fully detached from the host language.   For example, Gradle build files can make use of the host language syntax within the build file, and when that host syntax is used, the result is  a build file that does require a knowledge of the host language (which can actually be either Groovy or Kotlin). So for some code, even with a DSL capable of fully detached use,  working with that code will require knowledge of the host language.

Fully detached code can be designed to be  possible, but with the host language syntax available, it cannot be guaranteed all code will be fully detached.

Further, in practice many examples seek to be only partially detached from the host language.  In fact our own example all fit this pattern, as the semi-detached code actually exists interspersed with Kotlin code and there is no goal to enable code be read without knowing Kotlin.

Martin Fowler quotes the examples of the Rake DSL as being able to be categorised as either an independent language or an extension, which in my terminology would suggest it is more to the centre of the continuum.

When we use the term ‘Kotlin DSL’ or even ‘Python DSL’, we mean a DSL made by augmenting Kotlin or Python with both extra ‘vocabulary’ for domain specific features, are rarely.  The DSL is a set of new language constructs which extends an existing language.

Technically, this is always an extended language, but if the goal is to allow the use of these extensions by themselves you have independent language DSL, and if the goal is to allow programs in the host language access to new additional syntax, you have a Language Extensions DSL

An Augmentation DSL vs Language Augmentation

As discussed in languages, all but the simplest human communication makes use of language augmentation, and all but the simplest programs defines variables, functions and other elements that then become part of the language used elsewhere in the program.  An augmentation DSL is created when a specific block of language augmentation (definitions of variables, functions classes or even syntax) is separated from any specific application using that augmentation, and is provided for the use of any application which may require the same functionality.

Conclusion: DSL types can be quite different.

The Rake DSL(Detached/Augmentation hybrid DSL), or Gradle(Detached DSL) or HTML(External DSL):  these are all greatly different examples that all can be called DSL.

When the term DSL is used, it can refer to DSLs in general, but more often one of three entirely different types can be being discussed, and being discussed as if all DSLs are of that type, which can be confusing if you are often dealing with one of the other DSL types.  The term DSL is an extension of the language programming jargon, but perhaps it would be useful to have three additional terms, (making a four-word language extension) with an agreed adjective for each of the three types of DSL.