Do not create your own PS (DSL) to extend the functionality of the application.

    When you want to give the user the opportunity to write plugins for your application, you get up before choosing how to provide an API. Under the cut, I'll show you why the worst solution for this would be to invent your own programming language and parse the sources, and where furniture is.


    PL is not the main function of the application.

    Imagine that we opened the production of modular furniture. There are some basic elements: tabletops, coasters, bedside tables, etc. There is a production line related to the processing of wood: machines, saws, varnishes, all with the latest technology. But all this must somehow be stapled. We know that there are 100,500 companies that specialize in the production of hardware and bolts, there are some standards for furniture fasteners that were invented by a community of professionals to make their customers easier. How far-sighted a solution would be to deploy an additional line for the production of our own bolts, nuts and corners?

    What can we win?

    • We can thus brand our product so that the hamster would feel its own "or it’s", and carry more money.
    • Maybe this will allow us not to pay copyright for one of the bolts, or to solve the problem of logistics.
    • We can enter a new market for bolts and nuts by setting our standard, which is faster, better and higher.

    But let's keep it clean.

    • "Ilitarity" is the work of salespeople. Salesman will do or will not make the brand Elite both with the new production line and without it.
    • Copyright, as a rule (not always, of course), is cheaper than developing from scratch. And solving the problem of delivering a single item by deploying a new production line, you are only aggravating it.
    • If we want to try for ourselves a new field of activity - no need to associate it with what we are already a pro. It may seem that it is easier to push the bolts with furniture, but if the bolts do not take off, they will carry the furniture with them. At least, that which already stands at clients.

    Returning to our sheep: if you make an ecosystem for extensions of your application, then you have an application . It does something good, something that you are good at.

    KSP - Kontakt Script Processor, or the bolt production line in the digital audio world


    I will tell one story about such a language:

    Kontakt is a ompomler from the Austrian company Native Instruments . Currently, it is very difficult to find a project using virtual tools, in which it is not used. Over the past 10 years, Kontakt has taken most of the sampled instrument market. The secret is simple: in due time, Kontakt proposed two innovations that turned the approach to the development of sampled virtual tools.

    The first innovation was directly related to its main function: it was very careful with the memory (and the samples in wav are the one that still eat, both HDD and RAM). NI made a lossless compression format with fast decoding and wrote a revolutionary audio buffering system for its time.

    The second innovation was KSP

    Before contact, there were two ways to functionally organize recorded samples into a MIDI-controlled instrument:

    • Write your own engine from scratch in C ++, or another language that can use the VST SDK from Steinberg (and there are other plug-in formats, let's say AAX).
    • Use a ready-made sampler made for musicians who are not familiar with programming, but have sounds that need to be organized into some kind of system. Let's say Giga Studio . But such romplers, as a rule, were either closed, or in order to finish to fit their needs, they required no less staff than bare development under the VST SDK.

    The contact pleased both those and those: for quick prototyping there is a convenient GUI, understandable to any musician who read the manual, and for further refinement there is a programming language with conditions, functions (from version 4) and the standard library representing the API to the most part of the functionality implemented through the GUI, as well as to the parameters for playing back samples directly. Among other things, since version 2, it has become possible to customize the interface with all sorts of whistles and perdeals, which allowed us to show our uniqueness on an almost unlimited scale. And the developers code is hidden from the eyes twice: obfuscation and protection against changing tools .

    Given the increasing popularity of the engine, as well as the impressive period of active development of the rompler, today Kontakt is something like a Kalashnikov in the Digital Audio world. It is easy to learn, reliable as a tank, has the ability to dove it in reasonable limits for a loved one, and holds a huge market of satisfied users.

    Not so rosy

    The inevitable happened: innovation in the form of KSP has become a scourge. Trying to make the syntax available for dummies, which are musicians, instead of solving the API implementation on human PL, Nativ wrote their own interpreter of their own language, whose architecture did not initially imply such a violent flight of imagination of tool developers that we are seeing now. Already by version 3, the Nativs had lost hope of keeping up with the appetites of users, and they simply began to rivet the new functions of the standard library, allowing users to deal with the code development environment on their own.

    Moreover, even then Nils Lieberg KScriptEditor appeared , forked with Scintilla , which for a long time served as the main IDE for KSP. It is ridiculous to say, but when the Nativs realized that the contact could not cope with the size of the source being fed, they introduced functions into the language without even bothering to pass arguments to them. And a month later in KScriptEditor appeared taskfunc, passing arguments to functions that do not accept arguments.

    After a while, Nils realized that he was attacking the Nativs rake: it makes no sense to develop his own IDE. He moved the compiler and the implemented IDE functionality to SublimeText2, and waved the handle. At the moment, the reins of the SublimeKSP are borne by the developer, it seems, from Fluffy Audio .

    For the third time on the same rake

    Well, you understand)

    And again, the code generator, which is a language with an import system, a parser, a compiler, a syntax different from KSP, but still supporting backward compatibility with it, for an unknown reason, turns out to be a terrible mountain of crutches that cannot be thrown away. due to the backward compatibility of projects of library developers who have been developing their KSP engines for years.

    Suppose the import system works globally with respect to the file from which the compilation starts, so in order to compile one module that is in a subfolder, you need to completely change its import paths according to its position in the project structure. And the guy who supports him would be happy to change it, but then he will break the projects of the same Spitfire Audio for a long time. And this fact alone complicates the modular (we will keep silent about the unit) testing to hell.

    It would seem that the solution to the problem is to use symlinks, but something somewhere there does not work as expected, and symlinks only work partially. This kind of problem is not one thing. Among other things, after Nils, the development was not carried out by modifying the compiler itself, which already received the parsing code. And, again, for reasons of backward compatibility, adding plug-in plug-ins of extended syntax, each of which receives the source code that was originally cut into strings, parses them independently, and performs modifications.

    Considering that most of the preprocessor logic rests on macros and inline functions that expand the code into a huge canvas, which holds 80% of always true or always false conditions (by substituting constants for the condition input), which are minimized back at the stage parsing AST, the compile time of the "correct" source is comparable to C projects, this is in an interpreted language for teapots.

    To say that for KSP developers has become a pain - to say nothing.

    Not a single contact.

    I cannot give examples from other areas, but from the DigitalAudio realm:

    • Lemur is an application for shovels with a desktop editor that allows you to quickly make beautiful interfaces for your shovel communication using the OSC protocol . It has its own programming language , which can be used in special script objects scattered throughout the project tree. There is no way to make a compiler for it like what is done for KSP.
    • Reaper - DAW with a developed ecosystem for developing extensions. As a result, wherever possible, I duplicated my JSFX (ReaScript) JP as an API for C ++, lua, and Python.
    • HISE is a young harvester for writing and assembling VST \ VSTi , which will sooner or later kill Kontakt from the Swedish developer Christoph Haart . Inside the editor itself, it allows you to write on modified JavaScript, which is parsed and compiled into a binary already by C ++ objects. The idea with its own parser for introducing additional entities (for example, register variables, if I translated correctly) worked until users transferred their code from HISE to their favorite IDE with syntax highlighting, static analysis and formatting tools by JsPrettier. Now Christophe has sketched a couple of header files for compiling static libraries in C ++, which can then be used as modules in the editor. In parallel, he continues to complement HISEScript (because it is no longer possible to call it JavaScript) with new functions, but we know that ...


    Write your own application, devoting yourself to its main functionality, do not waste time on the parser, semantics and syntax. This is interesting until you start, but with a high probability will lead to a dead end. A programming language cannot be a part of an application: it is a kind of a separate production line that requires a lot of time for servicing, modifying and supporting the community. In turn, if you hope that you lower the threshold of entry for dummies - drop this thing. This teapot, as a rule, is afraid of typing anything at all, and will not bother yourself with your simple syntax.

    While novice plugin developers for your program, you can simply make a small QuickStartGuide , introducing them to the basic concepts of the chosen YP for expanding functionality and slowly feeding it your API, which is part of the ecosystem of this language.

    PS No, writing your own parser for a ready PL is also a bad idea.

    I will be glad to any criticism of the article, the first pancake and all the cases.

    Also popular now: