Extension Language Evolution: The Lua Story

from per. The source material dates from 2001, so some points may seem funny. Also, all references to "today", "at the moment", etc. belong to that period.
The presentation is on behalf of the author, as in the original.
All links are added by me.

The presentation is organized in chronological order. We will start with our experiments, which formed the basis for the creation of Lua in 1993, and go through eight years of discussion, solutions, work and entertainment.

Introduction

There is an old joke: "a camel is a horse developed by a committee." Among developers of programming languages, this joke is almost as popular as the legend of committee-developed languages. The legend is supported by languages ​​such as Algol 68, PL / I and Ada, developed by committees and not fulfilling expectations.
However, besides committees, there is an alternative explanation for the partial failure of these languages: they were all born large. Each of them followed a downward design process, in which the language is fully described before the programmer can try it in practice, and even before the first compiler appears.

On the other hand, many successful languages ​​have proven themselves even before their full development. They followed the upstream design process, appearing as small languages ​​with modest goals. When people began to use such a poorly developed language, new opportunities were added to it (or, sometimes, removed), clarity was made in ambiguous places (or, on the contrary, everything was made even more confusing). Therefore, the development path is an important topic when learning a programming language. For example, SIGPLAN has already sponsored two conferences on the history of programming languages.

This document describes the history of the Lua programming language. Since its emergence as a language for a pair of its own specific “home” projects, Lua has gone beyond our most optimistic expectations. In our opinion, the main reasons for this success lie in our approach to language design: to keep the language simple and compact, and its implementation simple, compact, fast, portable and free.

Lua was developed (or rather put forward) by a committee; albeit small - with only three members - but a committee. In hindsight, we understand that the initial development of the language by a small committee had a positive effect on the language. We added a new opportunity only by unanimous consent, otherwise it was postponed for the future. It’s much easier to add new features later than to remove them. A similar development process kept the language simple; and the simplicity of the language is our most valuable contribution. The other most significant qualities of Lua - speed, compactness and portability, stemmed from its simplicity.

From the first versions, Lua had “real” users, not just us alone. They made an important contribution to the language by discussions, complaints, user reports and questions. On the other hand, our small committee played an important role: its structure gave us enough inertia to listen to users without following all their suggestions.

Start

Our first experience at TeCGraf ( from the translation: “Computer Graphics Technology Group of the Catholic University of Rio de Janeiro in Brazil”), associated with the development of its PL, occurred in the data entry application. Engineers PETROBRAS (Brazilian oil company) several times a day needed to prepare files with the source data for the simulators. It was a boring and error-prone process, since simulation programs required strictly formatted source files, usually consisting of simple columns of numbers without any indication of what number it means. Of course, each number had a certain meaning, understandable to engineers at first glance at the diagram of a specific simulation. PETROBRAS turned to TeCGraf to create a graphical frontend for similar source data. Numbers could be entered interactively after a simple click on the corresponding part of the diagram - and this was a much simpler and more visual task than editing numerical columns. Moreover,
To simplify the development of this application in TeCGraf, we decided to program everything in the same way, developing a simple declarative language for describing all tasks on the input data. Here is an example section of a typical program in a given language called DEL (data entry language):

:e      gasket            "gasket properties"
mat     s                  # material
m       f       0          # factor m
y       f       0          # settlement stress
t       i       1          # facing type
:p
gasket.m>30
gasket.m<3000
gasket.y>335.8
gasket.y<2576.8

Expression : e declares an entity (named “gasket” in the example) that has several fields with default values. The expression : p imposes some restrictions on the values ​​in gasket, thus implementing data validation. DEL also has expressions for describing input and output data.
An entity in DEL is, in general, a structure or entry in traditional PLs. The difference is that her name also appears in a graphical metafile containing a diagram with which engineers enter data as described previously.

This simple language has proven successful both in TeCGraf, simplifying development, and among users by the ease of adapting data entry applications. Soon, users needed more features from DEL, such as Boolean expressions to control whether the entity was ready to enter data, and DEL got harder. When users started asking about conditional blocks and loops, it became clear that we needed a complete programming language.

Around the same time, we started working on another project for PETROBRAS called PGM: a customizable report generator for lithological profiles. As the name implies, the reports created by this program had to be highly customizable: the user could create and position tracks, choose colors, fonts and texts; each track could have a grid with its own settings (logarithmic / linear, with vertical and horizontal cutoffs, etc.); each curve had its own automatic scale; and much more.

All these settings were specified by end users, usually geologists or engineers, and the program should work on small machines, such as PCs with MS-DOS. We thought that the best way to customize the application is through a specialized description language, which we called Sol: an acronym for Simple Object Language, and at the same time it was translated from Portuguese as “sun”.

Since the report generator used many different objects with many attributes for each, we decided not to introduce these objects and attributes into the language. Instead, the language allowed to describe types. The main task of the interpreter was to read the description, check the correctness of the description of objects and attributes, and transfer information to the main program. To implement the interaction between the main program and the interpreter, the latter was implemented as a C library, linked to the main program. Thus, the main program had full access to all configuration information through the library API. Moreover, the program could register a callback function for each described data type that was called by the interpreter when creating an object of this type.

Here is a typical Sol code example:

- объявляем тип 'track', с числовыми атрибутами 'x' и 'y',
- плюс бестиповый атрибут 'z'. для 'y' и 'z' заданы значения по умолчанию.
type @track { x:number,y:number= 23, z=0}
- объявляем тип 'line', с атрибутами 't' (трек)
- и 'z' (список чисел).
- у 't' есть значение по умолчанию - трек (тип 'track') с атрибутами x=8, y=23, и z=0.
type @line { t:@track=@track{x=8},z:number*}
- создаем объект 't1' типа 'track'
t1 = @track { y = 9, x = 10, z="hi!"}
- создаем линию (тип 'line') 'l' с t=@track{x=9, y=10}
- и z=[2,3,4] (список)
l = @line { t= @track{x=t1.y, y=t1.x}, z=[2,3,4] }

The syntax of Sol was strongly influenced by BiBTeX and UIL (User Interface Language) , a user interface description language in Motif.
In March 1993, we completed the first implementation of the Sol language, but never imagined it. In mid-1993, we realized that DEL and Sol could be combined into a single, more powerful language. The lithologic profile visualization program soon required procedural programming support to create more complex layers. On the other hand, data entry programs also required descriptive tools to program their user interface.

So, we decided that we needed a full-fledged programming language with assignments, control structures, procedures, and all that. Also, the language needed data description tools by analogy with Sol. Moreover, since many potential users of the language were not professional programmers, complex syntax (and semantics) should be avoided in the language. Finally, the implementation of the new language was supposed to be well portable.

The portability requirement turned out to be one of the main advantages: those two applications ( from the lane it seems to mean DEL and Sol) should be portable, and so should the language. PETROBRAS, as a state-owned company, could not choose specific hardware, since it was purchased with very strict restrictions on spending government money. Because of this, PETROBRAS had a very diverse collection of computers, each of which had to run the software created in TeCGraf for PETROBRAS: it included PC DOS, Windows (3.1 at that time), Macintosh and all possible Unix.

At this point, we could take an existing language, and not create another new one. The main candidates were Tcl and, with a big lag, Forth and Perl. Perl is not an extension language. Plus, in 1993, Tcl and Perl only worked on Unix platforms. All three languages ​​also have complex syntax. And none of them had good data description support. So we started work on a new language.

Soon, we realized that for our purposes in a language, it is not necessary to declare data types. Instead, we could use the language itself to write type-checking functions based on the basic language features of reflection (such as type information at runtime ( from the first speech about RTTI )). View Assignment

t1 = @track {y = 9, x = 10, z="hi!"}

valid in Sol is also valid in the new language, but with a different meaning: it creates an object (in this case, an associative table) with the specified fields, and then calls the track function to check the object (and, sometimes, to set the default values).
Since the language was a modified version of Sol (“sun”), a friend at TeCGraf suggested the name Lua (“moon” in Portuguese), and so the language Lua was born.

Lua inherited from Sol the syntax of records and the construction of lists, but combined their implementation using associative tables: records used strings (field names) as indices; the lists used integer indices. In addition to these data description capabilities, Lua had no new concepts, since we needed an easy general language. So, we started with a small set of control structures whose syntax was borrowed from Modula (while, if, repeat until). From the CLU we took multiple assignment and return of several values ​​as a result of calling the function (a much more understandable approach than in-out parameters or passing by reference). From C ++, we took the idea of ​​the local scope of variables at the place of their declaration.

One of the small (even small) innovations was the string concatenation syntax. Since the language allowed the implicit conversion of strings to numbers, the use of the + operator would be ambiguous, we added the syntax ".." (two points) for such an operation.

The controversy caused the use of ";" (semicolons). We believed that the requirement to use a semicolon would confuse engineers with a good knowledge of FORTRAN, while on the other hand, not using it would confuse those who knew C or Pascal. In the end, we came to the decision of the optional use of a semicolon (a typical committee decision).

Initially, Lua had seven data types: numbers (stored in floating point format), strings, (associative) tables, nil (a data type with a unique value also called nil), userdata (a simple C pointer to represent C data structures inside Lua), Lua functions and C functions. (After eight years of language evolution, the only change in this list was only the unification of Lua and C functions into a single type). To keep the language compact, we did not include a boolean data type. By analogy with Lisp, nil is cast to false, while all other values ​​are cast to true. This is one of the few savings that we now sometimes regret.

Lua also adopted from Sol an implementation approach as a library. The implementation followed the principle now supported by Extreme Programming: "the simplest implementation that can work." We used lex for the lexical scanner and yacc for the parser. The parser translated the program into bytecode, which was then executed by a simple stack interpreter. The language had a very small standard library and it was very easy to add new features to C.

Despite the simple implementation - or perhaps because of it - Lua exceeded our expectations. Both projects (PGM and ED) have successfully used Lua (and PGM is still in use). In the end, other projects at TeCGraf started using Lua.

The early years (1994–1996)

New users create new requests. Not surprisingly, one of the first queries was to increase Lua's performance. Using Lua to describe data posed an atypical problem for a regular scripting language.
Shortly after we started using Lua, we noticed its potential for use as a language for graphical metafiles. The capabilities of data description in Lua made it possible to use it as a graphic format. Compared to other programmable metafiles, using Lua provided all the benefits of a full-fledged procedural language. VRML format, for example, uses Javascript to model procedural objects, which leads to heterogeneity (and therefore ambiguity) of the code. With Lua, the union of procedural objects in the scene description is natural. Fragments of the procedural code can be combined with declarative expressions to model complex objects while maintaining general clarity.

The data entry program (ED) was the first to use Lua for its graphical metafiles. It was common to have diagrams with thousands of parts, described by thousands of elements in Lua in a file of hundreds of kilobytes. This meant that, in terms of programming languages, Lua handles huge programs and expressions. And the fact that Lua compiled such programs on the fly (such as the “just-in-time” compiler) meant the very high speed of the Lua compiler itself. The first victim of the chasing the producer was lex. Replacing the scanner generated by lex with self-written code almost doubled the speed of the Lua compiler.
We also created new opcodes for designers. The source code for the list designer looked like this:

@[30, 40, 50]

what turned into a similar bytecode:

CREATETABLE
PUSHNUMBER 1                 # индекс
PUSHNUMBER 30                # значение
SETTABLE
PUSHNUMBER 2                 # индекс
PUSHNUMBER 40                # значение
SETTABLE
PUSHNUMBER 3                 # индекс
PUSHNUMBER 50                # значение
SETTABLE

With the new scheme, the code began to look like this:

CREATETABLE
PUSHNUMBER 30                # значение
PUSHNUMBER 40                # значение
PUSHNUMBER 50                # значение
SETTABLE 1 3                 # задание элементов с индексами от 1 до 3

For long constructors, it was not possible to stack all their elements on the stack before saving. Because of this, the code generator from time to time issued the SETTABLE opcode to reset the stack.
(Since then, we have always tried to improve compilation time. Now Lua compiles the program with 30,000 assignments six times faster than Perl, and eight times faster than Python).

We released a new version of Lua with these optimizations in July 1994 under the name Lua 1.1. The version was available for download via ftp. The previous version of Lua 1.0 was never publicly available. After some time, we also released the first documentation describing Lua.

Lua 1.1 had a limited user license. The language could be freely used for academic purposes, but not for commercial use. (Despite the license, the language itself has always been open source). But such a license did not work. Most competitors, such as Perl and Tcl, were free. Moreover, restrictions on commercial use interfered even with academic use, as some academic projects planned to enter the market later. So the release of the next version of the language, Lua 2.1, was free.

Lua version 2

Lua 2.1 (released in 1995) has brought many important changes. One of them was not in the language itself, but in the development process: we believed that it is always worth trying to improve the language, even at the cost of a small backward incompatibility.
In version 2.1, we introduced many incompatibilities with version 1.1 (but provided tools to help with code porting). We threw the @ syntax out of the table constructors and unified the use of curly braces for both records and lists. Throwing out @ was a trivial change, but changed the perception of the language, not just its appearance.

More importantly, we have simplified the semantic of designers. In Lua 1.1, the construction

@track{x=1, y=10}

had a special meaning. In Lua 2.1, the design

track{x=1, y=10}

is syntactic sugar for

track({x=1, y=10})

, that is, it creates a new table and passes it with a single parameter to the track function.

From the very beginning, we developed Lua as an extension language, so C programs could register their own functions transparently called from Lua. With this approach, it was easy to extend Lua with domain-specific primitives, which allowed the end user to adapt the language to specific tasks.

In version 2.1, we introduced the concept of fallbacks: user-defined functions that are called by Lua in case of undefined situations. ( from lane. This approach is a kind of superset of operator overloading. The principle is similar, but there are more possibilities) Lua becomes a language that can be expanded in two ways: expanding the set of "primitive" functions and expanding their semantics through fallbacks. That's why now we call Lua an extensible extension language.

We declared fallbacks for arithmetic, operator comparison, string concatenation, table access, etc. ( from the lane in more detail here and somewhat ahead here ). After the user sets it, a similar function is called whenever the operands of this operation are not suitable for their types. For example, when adding two values, one of which is not a number, fallback is called, and the result of its call is used as the total amount.

The operation of accessing the table is of particular interest (and is the main reason for the appearance of fallbacks): if, when x = a [i] is executed, the value of a [i] is nil ( from the lane, ie, table a does not contain field i ) , then the fallback function is called (if given), whose result is used as the value for a [i]. This simple new functionality allowed programmers to implement various semantics of access to tables. In particular, you can implement some kind of inheritance through delegation:

function Index (a,i)
    if i == "parent" then       -- чтобы избежать цикла
        return nil
    end
    local p = a.parent
    if type(p) == "table" then
        return p[i]             -- возможнен повторный вызов функции Index
    else
        return nil
    end
end
setfallback("index", Index)

This code goes up the chain of “parents” until it finds the right field or reaches the end. With this code, the following example will output “red” even if b does not have a color box.

a = Window{x=100, y=200, color="red"}
b = Window{x=300, y=400, parent=a}
print(b.color)

There is no magic or hard-coded behavior in delegation through the “parent” field. This is the choice of the developer. It can use a different name for the field, or implement more complex multiple inheritance, allowing the “parent” field itself to be a table traversed sequentially, or somehow else.

Another fallback will be called for the expression a [i] if a is not a table at all. This fallback "gettable", triggered to get the value a [i] in a situation of the form x = a [i], and fallback "settable", triggered when written to a [i] in a situation of the form a [i] = x.

There are many uses for these table fallbacks. Interlanguage interaction is a very powerful opportunity: when astores a value of type userdata (a pointer to something in C code), fallback implements transparent access to values ​​inside the data structures of the main program.

Our decision not to sew up rigidly similar behaviors in the language implementation led to one of the main concepts of Lua: meta-mechanisms. Instead of littering the language with many possibilities, we have provided ways to realize these opportunities for those and only those who need it.

The meta-mechanism of fallbacks allows Lua to support OOP in the context of some implemented types of inheritance and operator overloading. We even added a little syntactic sugar to describe and use the “methods”: functions can be declared in the format a: f (x, y, z), in which case a hidden parameter self is added when calling, making a: f (10,20 , 30) equivalent to af (a, 10,20,30).

In May 1996, we released Lua 2.4. The main innovation of this new version was an external compiler called luac. This program compiled Lua code and saved the bytecode and row tables into a binary file. The format of such a file was chosen for easy loading and portability between different platforms. With luac, programs could avoid parsing and code generation at startup, which was expensive, especially for large static programs such as graphic metafiles.

Our first Lua publication already considered the possibility of an external compiler, but we only needed it after the widespread distribution of Lua in TeCGraf and the huge graphic metafiles with Lua code created by graphic editors.

In addition to speeding up loading, luac also allows you to perform syntax checks during compilation and protect the source code from changes by the user. However, precompilation does not speed up execution, since Lua always precompiles the source code before execution.

luac is implemented in "client mode", that is, it uses modules that implement Lua as a simple client, even though private header files are used to access internal data structures that need to be saved. One advantage of this policy is the separation of the Lua core implementation into clearly separated modules. Due to this, it is now easy to remove parsing modules (lexer, parser and code generator), which occupy 40% of the Lua 4.0 kernel code, leaving only a tiny module for loading pre-compiled code fragments. This can be useful for very small Lua implementations for embedding in small devices like mobile phones or robots (for example, Crazy Ivan, the robot that won RoboCup in 2000 and 2001 in Germany, had brains implemented on Lua).

Introducing the World (1996–2000)

В июне 1996 мы опубликовали академическую статью про Lua в Software: Practice & Experience (от пер. по ссылке скачивание за денежку. нашел в другом месте судя по всему тот же самый материал и конвертнул в pdf, если кому интересно). В декабре 1996 журнал Dr. Dobb's опубликовал статью про Lua. Данные публикации, нацеленные на разные сообщества, привели к международной известности Lua.

Вскоре после статьи в Dr. Dobb's мы получили множество писем про Lua. Одно из первых писем было следующим:

From: Bret Mogilefsky 
To: "'lua@icad.puc-rio.br'" 
Subject: LUA rocks!  Question, too.
Date: Thu, 9 Jan 1997 13:21:41 -0800
Привет...
Прочитав статью Dr. Dobbs про Lua, я с нетерпением проверил его,
and the result exceeded all my possible expectations!
Its elegance and simplicity amazed me. My congratulations for
developing such a well-designed language.
About me: I'm working on adventure at LucasArts Entertainment Co.
and I want to try replacing our old SCUMM scripting language with Lua.
[...]

It turned out that Bret Mogilefsky was the lead programmer of Grim Fandango , the main adventure game of LucasArts in 1997. In another letter, he writes to us that “The HUGE part of the game is written in Lua” (authorial emphasis). This first use of Lua in the game attracted the attention of many game developers around the world. After some time, Lua increasingly began to appear in game newsgroups like rec.games.programmer and comp.ai.games .
Due to its small size, good performance, portability and ease of integration, Lua has gained great popularity for expanding the functionality of games. Nowadays, some gaming companies use Lua (for example, LucasArts, BioWare, Slingshot Game Technology and Loewen Entertainment) and knowledge of Lua is a competitive advantage when looking for work in the gaming industry. According to our estimates, half of Lua users are involved in game development, but it’s hard to calculate more precisely because of the great secrecy in the gaming industry. For example, although Bret Mogilefsky adapted Lua for Grim Fandango, the details are not available to the public.

Embedding a scripting language in a game provides several advantages. The scripting language can be used to describe sprites and physics of objects, control AI and character, as well as to interact with input devices. For example, the engine may not know anything about such things as “damage”, “speed”, “weapons”, etc. Choosing a simple language allows game designers to use programmable tools. This is crucial for game development, as it allows designers to experiment with their creations. Scripting languages ​​also facilitate rapid prototyping and facilitate the implementation of debuggers.

Not so long ago, in 2000, LucasArts released another game using Lua, Escape from Monkey Island., which was the fourth in the adventure series Monkey Island. In this game, to pay tribute to Lua, the authors renamed the in-game bar from SCUMM (the programming language previously used by developers) to the Lua Bar.

In addition to its widespread use in computer games (for example, Grim Fandango, Baldur's Gate, MDK2, Escape from Monkey Island), Lua is used in many other areas around the world.

One of the first applications of Lua outside of PUC-Rio ( from the lane the same Catholic University of Rio de Janeiro, of which TeCGraf is a subdivision ) was the Smithsonian Astrophysical Observatory . They developed a generalized aperture programto simulate the effects of photon flux on physical obstacles. This program was part of efforts to promote the AXAF (Advanced X-ray Astrophysics Object) program , the third of four NASA's Great Space Observatories.

Performance Technologies used Lua to implement a command line interface for the CPC4400 - an hot-plug ethernet switch. Using Lua as the scripting language for the CPC4400, the user can associate events that occur (such as connection status, topology changes, RMON alarms) with scripts on Lua.

Tollgrade Communications used Lua in their next-generation product to test the DigiTest telephone network. Lua was used for the user interface, automated test scripts and analysis results.

Lua is also used at InCor Heart Institute (Instituto do Coração, São Paulo) in Brazil, at CEPEL (research center of the state electric power company ELETROBRAS) in Brazil, at the Weierstrass Institute in Berlin, at Berlin Technical University , and in many other places .

In 1998, Cameron Laird and Kathryn Soraiz in their column on scripting languagesSunWorld magazine estimated that "there are about a few hundred thousand Lua programmers in the world." In their opinion, this is a "small number of users", but for us it is a clear sign of the growing popularity of the language.

Lua version 3

Lua 3.0 (July 1997) replaced fallbacks with a more powerful tagging concept. Fallbacks were global in nature: user-defined functions were called whenever an event occurred, and only one function could be set for a particular event. This made it difficult to combine Lua modules, which, for example, used a different approach to implementing inheritance. Although it was possible to implement fallback function call chains, this approach was slow and error prone, and hardly anyone would use it.
From Lua 3.0, a programmer could create tags and associate tables and userdata with them. Tag methods are essentially all the same fallbacks selected according to the operator tag. With tags and tag methods, different tables (and userdata) could use different fallback functions for their operations.

The concept of tags is proposed by Lua for custom types. In other words, a tag is just a number representing a new type. When you associate a table with a specific tag, you are actually declaring a new type for that table: the type (or tag) indicates how it implements these statements.

At the initial introduction of fallbacks, most of them described the behavior of Lua in error situations, such as accessing an index from a non-table or trying to call a non-function. So we thought fallbacks were an exception handling mechanism. With the introduction of custom tags, fallbacks (now called tag methods) have become the main mechanism for describing the behavior of new types, even though we still use them to extend the behavior of basic types.

Despite the new state of affairs, for a long time we perceived tag methods as error handling mechanisms and could not associate tags with types. Only recently we understood the importance of custom tags and tag methods as a mechanism for creating custom types. Lua 4.1 will reinforce this understanding by allowing users to specify names for such new types (now only basic types have names).

Lua 3.0 also introduced support for conditional compilation in a C-like preprocessor format. Like any language feature, it was very easy to add this (although this complicated the lexer), and soon the programmers began to use it (programmers use any language features). With the advent of new functionality, the demand for its further development immediately grows. One of the most frequent requests was to add macros, but the lengthy discussion did not translate into a clear sentence either on the mailing list or between us. Each of the proposals required huge changes in the lexer and parser, without showing any obvious benefits. So the preprocessor remained static from Lua 3.0 to version 3.2 (for two years).

In the end, we decided that the preprocessor did more harm than good, making the code cumbersome and luring users into endless discussions, and deleted it in Lua 4.0. And without a preprocessor, Lua got cleaner. Over the years, we have sought to make Lua simpler and remove the dark corners of the language, which we once considered new features, but which were used only by rare programmers, and later became generally considered errors.

Lua version 4

Prior to version 3.2, only one “state” of the Lua machine could be active at one time. We had an API for state transition, but it was somewhat inconvenient to use. To simplify the development of the API, we did not include an explicit state parameter in the function - there was only one single global state. Now it’s clear that this was a mistake. By the time Lua 3.2 was released, it became clear that many applications would be easier if they could conveniently work with multiple Lua states. For example, we made a special version of Lua 3.2, included in CGILua - an extension of browsers for returning dynamic pages ( from the lane we still mean the server side ) and CGI programming in Lua. Previously, LucasArts did something similar for Lua 3.1.
When discussing our plans for Lua 3.3, the highest priority was given to the explicit state APIs. However, this raised the issue of backward compatibility. In the end, due to incompatibilities, we decided to rewrite the API for the next version, which became Lua 4.0. Now the API not only contains explicit states, but it has also become simpler and more efficient. We were afraid that the transition to the new API would not be the easiest, as this was the first case of major changes to the API since Lua 1.1. We received several complaints on the mailing list, but in general the change was not traumatic at all. Many Lua programmers did not interact directly with the API, many used it only through automated tools, and many believed that the benefits exceeded the transition price.

We released Lua 4.0 in November 2000. In addition to the new API, this version brought many other small improvements, such as a loop.

Anyone who has worked with programming languages ​​knows how easily the "holivars" are kindled on this issue. An interesting feature of these wars is that the more common the topic, the hotter the discussion. For example, people are very pleased to discuss the use of semicolons rather than higher-order functions. One of the reasons for this, of course, is that significantly more people have an idea on the first topic than on the second. But another more important reason is the strong dependence of the routine of the topic on the level of knowledge of the language. If the language is not used to create amazing and thoughtful tools, if it does not have a good grip, nobody will use it.

Since version 1.1, the for construct has been in the wishes of most Lua users. The main cause of dissatisfaction was the forgetfulness of people to increment at the end of the iteration of the while loop , which led to a loop.

We pretty soon agreed with that. But, although we all agreed on the need for a for loop, we could not agree on specific syntactic constructs for it. We considered a Pascal (or Modula) style design too restrictive, since it did not involve iterating over table elements or over file lines. Moreover, translating the identifier to into the list of reserved words would lead to unacceptable backward incompatibility. On the other hand, the C-style for loop did not fit Lua.

With the introduction of closures and anonymous functions in version 3.1 (July 1998), we decided to use higher-order functions for iterations. (In fact, the need for for was one of the main reasons for introducing anonymous functions in Lua). Lua 3.1 came out with two predefined functions for iteration:

foreach(table, f)
foreachi(table, f)

The foreach function applies f to all key-value pairs selected randomly from a given table. The foreachi function is similar, but treats the table as a list (or array): it iterates over elements with numeric keys in ascending order of key values. Although we provided only these two workarounds, it was easy to create new iterators.
But although it was easy to create new ones, but for more than two years, almost no one did them. The first reason was the discomfort for many programmers to use anonymous functions and higher-order functions in a procedural language. And the second and, in our opinion, much more important reason was the lack of need for most developers in other iterators. This means that for many years we tried to achieve what the real user does not need. With this in mind, we quickly developed two for loop formats: for numerical iteration and for table traversal.

The for statement was one of the most successful language changes since the first version. First, it covered the most commonly used loop applications, while while was available for the most generalized loops. Secondly, because of its strict format, it was easy to add special opcodes for implementation, which made numerical iteration with an empty loop body more than twice as fast as a similar loop through while.

Conclusion

Lua now has an established user base. An active mailing list for nearly 500 people from more than 30 different countries of the world is active. The site (www.lua.org) has approximately 500 visitors per day from 50 countries. The language is used in the spectrum from adventure to web servers to testing the telephone network for Ethernet switches.
Several ftp servers offer Lua source codes, and several other sites distribute platform-specific versions, such as DLLs for Windows, SIS for EPOC, RPM for Linux, binaries for RISC OS, etc. Moreover, some magazines distribute Lua on additional CDs (for example, Dr. Dobb's, Linux Magazine France and Japanese C Magazine).

The main contribution of Lua as a language is to provide meta mechanisms instead of functionality. The success of Lua as a product came from its simplicity, small size and portability of its implementation, which made it possible to use Lua on many different platforms, including small devices like palm, PDAs, specialized boards and robots. Cameron Laird and Kathryn Soraiz predicted in 1998 that "the inevitable explosion of the widespread use of embedded devices (computers in your car, bathroom, and kitchen appliances) can only benefit Lua." Then we did not pay much attention to it, but they were right.

Lua also contributed to academic activity through several abstracts and publications on both the language itself and its technological use.

Success comes at a price. In the process of language evolution, backward compatibility hindered innovation more and more. However, we did not allow compatibility to halt progress - this is just one of many ingredients (albeit strong) in the alchemy of language design.

Finally, holding on to a language is much more than just developing it. Full attention to detail is important in all aspects: language design, creating a community of users and listening to their words, while adhering to the original design decisions.

Acknowledgments

Lua would never have been without the help of many people. Everyone in TeCGraf took any part: using the language, discussing it, disseminating it outside of TeCGraf. Special thanks to Marcelo Gattass, the head of TeCGraf, who always supported us and gave us complete freedom regarding the language and its implementation. Lua was really the first TeCGraf product to be publicly available on the Internet even before the rise in popularity.
Without users, Lua would be just another language. Users and their goals - this is the maximum test for the language. From them we received reports on bugs, design flaws, new ways of perceiving reality. Special thanks to the members of the mailing list for their discussions, suggestions, and mainly for the patience with our authoritarian approach from time to time.

from per. If the topic is interesting, then I can collect information on newer versions of Lua, as well as about the implementation of luajit.
Thank you for reading :)