With Kotlin, the term ‘DSL’ has taken on a specific meaning, and that more specific meaning of Kotlin DSLs other pages on this site and elsewhere. 

In fact, Kotlin DSLs are not just Domain Specific Languages, but Domain Specific Extension Languages.

The principle of a DSL, is that languages benefit from The DSLE methodology is a key use of Kotlin DSLs but DSLE Methodology can be used in any language, and no specific language support is needed, although Kotlin DSLs or language features such as present in Python can assist with elegant applications of DSL methodology.

DSLE Methodology also is different from, but not an alternative to, design patterns such as:

MVC, MVP, MVVM, MVI etc

which will also be discussed elsewhere. In brief, these design patters breakdown an application into logical units, and each logical unit can be coded with or without the use of DSLE methodology.

This page concentrates on the concept of DSL Extension (DSLE) Methodology.  DSLE methodology is to consider software development as finding and/or creating the independently maintainable components that act as language extensions, and allow the expression of what the program does within a single concise function or module written in the extended language , in a manner where business logic not only executes correctly, but is also easy for a person to read, understand and when necessary, modify.

In simple terms, extend the language until the problem is simple.

• Introduction
  • Who uses DSLE methodology?
  • Introduction to DSLs
    • DSL: the general DSL definition
    • DSL: independent language DSLs compared to language extension DSLs (DSLEs)?
    • When does a program module create a DSLE?
    • What is different between a DSLE and a ‘library’ or framework?
  • Human Language and DSLs
    • Why consider human languages?
    • The Dictionary is an insufficient reference.
    • Jargon as a form of DSL
    • Situation Specific Language extensions
    • Specific Human language: Independent Language vs Extensions.
    • Conclusion on Human DSLs
• DSLE Methodology: The Basics
  • DSL vs DSLE
  • DSL Methodology: Building Blocks
  • More Layers
  • The Core concept
  • When to apply DSL Methodology
• Implementing DSLE Methodology
  • The Core concept
  • ‘Buy’ or Build?
  • software modules: group extensions together
  • Leveraging existing Language Extensions
• DSLE Project examples
• Conclusion

Introduction

Who Uses DSLE methodology?

DSLE methodology is a name for a methodology that is already in extensive use, and on major projects, but often in an unplanned manner. If anything other than the name is new, it is the idea planning DSLE methodology, rather than an DSLE solution that just happens that way. As you will see, many major projects have used DSLE technology, even though not all originally planned it that way.

Introductions to DSLs and DSLEs

DSL: The general definition

A DSL is an acronym for Domain Specific Language.  A ‘Domain’ being effectively a specific application. Contrasting with ‘General purpose languages’ which attempt to allow for solving any programming problem, a DSL can be purpose designed for a specific ‘domain’ or a specific type of problem.

The label DSLs can make it sound the goal is to use Kotlin (or Python) to build an entirely new language, but DSLEs are definitely simpler. All kotlin DSLs are extensions of Kotlin, and although sometimes use cases can focus on the extension and make little use of the underlying language, that underlying language is always still there.

Two Types of DSL: Independent DSLs and Language (extensions) DSLs

Independent DSLs

There are DSLs like SQL, which are independent stand-alone languages to tackle specific fields or domains.  In cases like SQL, the language of the DSL is indepenant from other languages. These, although DSLs, are quite different from DSLEs, and it will be shown that indepenant DSLs do not normally exist outside of programming.

The benefit of indepenant DSLs is that there is not underlying language to learn first, and everyone who knows the DSL can communicate, without any need to understand a common underlying language.

The drawback of independent DSLs, is that not it is not normal for all communication to be specific to a one domain. Yes, an independent domain specific language for plumbers the world over would give plumbers terms for all tools and concepts specific to their field, but it will not allow them to communicate about the weather or lunch unless it defines plumber DSL words for everything a plumber may need to talk about. The same problem occurs with SQL, where not only is there syntax for database queries, but for expression and calculations, and even flow.

In DSLE methodology, the recommendation is to avoid independent DSLs. Note, that Kotlin DSLs are all DSLEs rather than independent DSLs. Use Kotlinx.html as opposed to HTML where possible in Kotlin projects and use language wrappers for SQL in preference to directly using SQL.

Language (extension) DSLEs.

There is also a solution where a DSLE is an extension to a general purpose language. For example, in place of developing in SQL with Python, SQLAlchemy brings the power of the SQL language to Python.

When we use the term ‘Kotlin DSL’ or even ‘Python DSL’, we mean a DSL made by extending Kotlin or Python with extra ‘vocabulary’ for domain specific features.  The DSL is a set of new language elements which extends an existing language.  This type of DSL is seen as the preferred solution, as nothing new needs to learnt for implementing that which can be done outside the DSLE. In programming, with an extension, there is less syntax to learn and greater consistency. In place of a new independent language, there is only extra ‘vocabulary’ available within a language we already know to use.

When does a program create a DSLE?

Every program modules add new components such as functions and classes to a language. When does adding new features to an existing language, create a new DSLE? As with many things, the extremes are evident. The “hello world” program would be regarded to not create a DSLE as it only defines always defined ‘main’ function, and at the other extreme, some packages clearly state that they implement a DSLE. However, if you had half the features, would it still constitute a DSLE? At what point does adding additional ‘vocabulary’ reach the definition of being a DSLE?

The reality is that every program, beyond “hello world”, creates some new vocabulary somewhere, either a function or a variable or some definition. However, if the program is called “hello”, then the computer gains new syntax in ‘the shell’ such that typing ‘hello‘ now does something, and prints “Hello world”. The shell gains “hello” as one new word of vocabulary. While one word is not much of a language extension, the concept of extending a language is common throughout programming, so the principle of building a DSLE always applies at some level.

What is different between a DSLE and a ‘library’ or framework?

Many frameworks or libraries are standard parts of the underlying language, can be added . There also a huge collection of open source libraries and/or frameworks that are not inlcuded in the core lang Every framework or library does extend the underlying language, therefore can be considered a Language Extension. DSLE.

Human language and DSLs/DSLEs

Why consider human languages?

A significant part of the human brain has evolved specifically to process language.  Since the first move from machine code to assembler, the goal has been for computer programs also to be processed by humans.  Just how do our brains handle domain specific language?  Don’t we take years to learn one single language and find learning another quite difficult?

Jargon as a DSLE equivalent

One spoken language parallel to a computer extended DSL is ‘jargon’. Many ‘domains’ evolve their jargon or extensions to the language. Jargon more concisely and more specifically communicates the concepts needed in specific a domain than regular ‘non-jargon’ language does, but is generally used in combination with an underlying general-purpose language, such as English or French or Chinese. People who already speak a general purpose language can learn one or more jargon vocabularies in a much shorter time than learning the general-purpose language.

The dictionary is a delineator between ‘language’ and ‘extensions’.

If you are reading this document, it can be assumed you can read English, and the reference for English is the dictionary.  But there may be words even on this page, where the dictionary is actually not that helpful, because the dictionary does cover all words as used in a computing related document. Wikipedia can be a useful source of far more information behind the words, but there are also words that change with context (e.g Python).  All of this means the language becomes more like the syntax for expressing meaning, but it can take a lot of language to actually convey meaning.  Just as Python or Kotlin have a language syntax, but language extensions are built within that base syntax.  To understand the meaning of what is written, we may often encounter extension terms that until we learn them, may require use of the dictionary, the encyclopedia, other web searches, or for local context specific things like ‘the broom closet’, we may have to ask.  All of this is paralleled in code. We at first can need the definition of an object or function, but once we are familiar we should not need keep referring to that source.

Situation Specific Language extensions.

However humans also learn far more localised and situation specific language extensions.
Consider a random page from a novel. Most of the words can be found in the appropriate language dictionary, or an encyclopedia, because the are part of the general language (e.g. English). But there are words that are not sufficiently explained by either dictionary or encyclopedia, because they have  a specific meaning in the context of the novel, and the meanings are explained through the novel.  Names are one class of such words. Names can make reading a page at random a challenge. Read a random page from a novel and we skip the explanation of the specific meaning.  Just who is ‘Harry’ or ‘Sally’? Have they met?  What is their relation to the protagonist? Novels are designed to be read sequentially, so there is no index to easily find what has been defined, and usually no clear list of what is defined.  Depending on the novel, significant amounts can be specific to the novel.  Consider Lord of the Rings.  Not only are characters explored, but also types of creatures, new locations and imaginary world.  It can be described as a “Lord of the Rings Universe” being created.

So even to navigate an individual literary work, a new extended vocabulary can be required, varying from knowledge of just a few character names through to an entire altered universe.

Specific Human language: Independent Language vs Extension.

While jargon can seem impossible to understand for a ‘layman’ who only speaks regular ‘non-jargon’ language. The reality is, jargon normally does not create a replacement for regular language and communication in jargon alone is rarely sufficient. Even very domain specific communication requires a mixture of regular language and jargon together.

Imagine, for example, a French person and a Chinese person who are both from the same industry and use the same jargon, but other than that jargon are unable to communicate. Even adding words like ‘very’ to the jargon would be a problem. Like independent DSLs, jargon needs to coexist with a general purpose language.
Just like the French and Chinese colleagues, every independent DSL also needs some ‘normal words’ so they add their own limited set of ‘normal words’. This means Independent DSLs have to revisit many things already present in general programming languages, and the result is still restrictive.
The French and Chinese hypothetical colleagues would be far better placed if they both spoke a common regular language in addition to just the jargon.

Conclusion

Human communication in human language actually relies on language extension using a base set of rules.  This suggests that our thinking should be well adapted to the same approach within programs.

DSLE Methodology: The basics

Simplest Language extensions.

Language extension is at the core of programming, as programs are largely a series of declarations or definitions. Even a variable definition, defines new language element. For the relevant scope, this word now means this.

To bring language defined elsewhere into our program, in python we have ‘import’ and to give increased scope of what can be imported there is ‘pip install’.

DSLE methodology: building blocks

It is generally agreed that there is a maximum number of lines for a well defined function. Opinions on the actual limit vary,  but generally the recommendation range from that which can be seen on the screen at one time, through to as high as around 100 lines.

Now consider that, every program is described by a single function, usually called ‘main‘.  With a very simple program, all the code could be held in main, but as the code grows, that limit of  around 25 to 100 lines in one function will become a restraint.

How to describe the program in the main function, and keep main small enough to read and understand?

As the program grows, the developer can move some code to ‘other functions’ and in main simply call these functions containing the moved code.  This is one way the size of main can be controlled.

But simply moving ‘chunks’ of main into functions is not DSLE methodology. DSLE methodology is to create buildings block functions, that do not themselves contain business logic of the program, and allow writing the logic of main in a more concise way.  The logic of main, stays in main, it is only logic to turn steps into extensions of the language that is moved to the DSLE functions.

Main stays readable if the concepts of the functions are clear.  Usually any  functions with ‘moved code’ will need to be generalised to convert them into building blocks, and the individual application specific nature come from parameters specified when those blocks are called. Then understanding what the program does can still be clear just by considering that main function.  A new developer may not need to read beyond main to learn, to understand what the program does, and may limit going beyond main to an area of functionality of particular interest, and infer the meaning of other new ‘vocabulary’.

More Levels

The concept is that main describes the program at the top level, but main will need the use of either program specific building blocks or ‘extended language’ and/or  language building blocks, know as packages, which are common to several applications.  In the case of “hello world”, the only other function is the print function, and that function is considered part of the language.  If you know the python language, you know how the python print works.  However if the other functions  beyond main are the ‘moved code’ described above, then these other functions will most likely be unique.  They are an extension to the language for the use of this main function, and unique to this main function.   While the function names can convey what is done at a high level, to know exactly what these functions do, a person reading the program will need to go and then read these functions.  To read main, this is the equivalent to looking up a word in the dictionary.

As the solution grows in detail, in turn these functions become complex and will also require their own extensions to stay within size constraints. As the program system grows the number of levels of extension to the original language grows.

The Core concept.

The core concept, is that the end result is the main is written using an extended language, and those extensions are build on other extensions. Each level  should be readable without looking up what each component of the new language means in full detail.   Each level is written in terms of an underlying extended language, and the program should be broken up into components that define new language blocks or ‘jargons’, which are separate from layers build using those jargon language blocks.

The role of each level or block is to provide the language extensions to make the level above simple to understand. The lowest level of the program is the only level of the program written in python or kotlin or whatever itself, as all other levels are built on the extensions.

A web server will rarely be built in ‘raw’ python alone, but will normally be built on a software stack of template engines, routing engines, database engines etc.  The language of the project becomes not just python, but python plus all those extensions.  Then the project may add its own extensions.

But to work in the project, you have to learn the language of each of the extensions, or at least the language of the extensions being used in the area of the project you are working.

Every project of any scale is not just built on a language, but on the language plus the set of extensions to that language.

When to apply DSLE methodology?

It follows from the concept of ‘the main function describes the program’ that a very simple program such as hello world already achieves the goal of main being conveying what the program does. In fact any project simple enough to be contained in a single file, or  unlikely to require changes beyond a month from when the program is written, is too small scale or time frame to benefit significantly from DSL methodology,

The main relevance of DSL methodology is for long term projects with continued updates, developed by a team producing several releases over a time scale of more than one year.

It is with this type of software that reading what the code does can become a challenge even to the author of that block of code over time.

Implementing DSLE methodology

the goals

DSL methodology is simple units of code that stand alone and can be understood alone, and used as ‘black boxes’ by other units of code. The steps to implementing are all striving to achieve these goals.

The goals DSL methodology are:

• allow each part of the system to be expressed in the simplest language possible and with the smallest possible language extension
• keep system specific functionality (“business logic”) at the highest layer possible, and avoid buried functionality
• building blocks should be as generic as possible and able to be understood without considering the overall application

software modules:  group related extensions together, design as open source

Building blocks should where possible be grouped in to logical modules, that together provide a specific type of new functionality.  These modules should be considered to have functionality independent of the central application or “business logic”, and be able to have their own documentation, and perhaps own repository.

In fact, given that such extensions should not contain the logic of the main application, it may be possible to open source these extensions, even where the main application and “business logic” itself is proprietary and would not be open source.

Leveraging existing Language Extensions

Most often best described as packages, there are readymade language extensions which can give great capabilities.  Selection of these packages becomes very important as each selection becomes additional ‘extended language’ that the team must become familiar with.

The real world is, that beyond well established quite generic packages, there are many packages that takes steps towards keeping your code simple, but in the end in your usage still leave a program to big to maintain.  There choices then are:

• extend an existing package
• build a new package that uses the existing package internally but exposes only a new api
• use the existing package as inspiration and effectively fork
• build an alternative package

DSLE Project examples

What makes and example?

In a DSLE methodology project, the goal is to identify building blocks that enable the application itself to contain reduced code, but are written independently of the application, can be considered as a separate project. The team working on the ‘building blocks’ component(s) can write and test their code independent of the now simpler application level code that uses those building blocks. So when does a DSLE project become known? Answer: When the building blocks themselves becomes a popular library of DSL extensions.

Example: Django.

Django was created in the fall of 2003, when the web programmers at the Lawrence Journal-World newspaper, Adrian Holovaty and Simon Willison, began using Python to build applications.

Django Wikipedia page

Like many other ‘popular of the shelf’ DSLE packages, Django was initially developed for a a project that identified much of that project could be considered reusable code. Now there are even advertisements for ‘Django programmers’, as programming is Django is not just programming in Python, it is programming in an extended language.

Example: Ruby on Rails

David Heinemeier Hansson extracted Ruby on Rails from his work on the project management tool Basecamp at the web application company also called Basecamp (37Signals at the time)

Ruby On Rails Wikipedia page.

Once again, Ruby on rails initially as set of building blocks for a specific application, that were created in a manner that allow separating (and in this case reusing) those building blocks independently of the initial application.

Conclusion

The ‘building blocks’ sub-project of most projects will never be reused outside the original organisation, let alone become as popular as Ruby on Rails. But even Ruby on Rails started as a set of building blocks used by on single project. The original development was for use within a single project.

The goal of DSLE methodology is to identify sets of building blocks that create language features that make the actual application simpler, and develop those building blocks independently of the actual application code, free of the application specific code.

If well executed, the result is building blocks that can be understood, coded and tested independently of the application, and a far simpler application build using those building blocks.