Music, Mathematica and the Computing Universe: Automatically creating music based on cellular automata

Stephen Wolfram’s translation of the post “ Music, Mathematica, and the Computational Universe ” about the wonderful WolframTones resource , which was recently resumed on the new Wolfram Cloud site (the site created in 2005 was unavailable for a couple of years since it used solutions not supported by modern browsers).
I express my deep gratitude to Kirill Guzenko for help with the translation.

---

How difficult is it to create human music? One to pass the musical analogue of the Turing test ?

Although music usually has a certain formal structure , which the Pythagoreans noted 2500 years ago, in essence it is very humane : a reflection of pure creativity, which is the essence of the defining characteristic of human abilities.

But what is creativity? Is this what was needed throughout biological and cultural evolution? And can it also exist in systems that have nothing to do with people?

In his work on the book New Kind of Science ( A the New Kind of Science) I researched the computing universe of possible programs and found that even very simple programs can show a strikingly rich and complex nature, along with, for example, what can be found in nature. And, based on the developed principle of computational equivalence , I came to the conclusion that there can be nothing that fundamentally distinguishes our human abilities from any processes that occur in nature, or even in very simple programs.

But what about music? Some people, opposing the principle of computational equivalence, used their belief that “ there cannot be simple programs that can produce serious music ” as an argument .

And I was curious:Is music really something special and exclusively human? Or all the same, it can be perfectly created automatically using calculations?

In 2003, after ten years of my reclusive work on A New Kind of Science , worldly problems gradually ceased to be alien to me, and one of them was that the ringtone on my cell phone was like everyone else's. So I thought: if some kind of original individualized music can actually be created automatically, then you could just change everyone's melodies on your mobile, and everyone would have their own .

After some time, we decided to experiment with the possibilities of creating music using programs.

It turned out a long history of trying to create music by the rules. Most of the received impression was too " robotic " or random. But what I came to in A New Kind of Science seemed to give us new opportunities, because it was convincingly shown there that even with the rules of simple programs you can create such complex and vibrant things that we observe in nature and which we admire .

We started with the simplest experiment: we took cellular automata so familiar to us ( Wikipedia article) and used fragments of the templates they produce to form the score. I had no idea what we would get as a result. And, of course, some cellular automata with simple patterns of behavior produced absolutely boring music. However, to some of my surprise, in the computing universe behind the cellular automaton, which generates very rich and pleasant fragments of music, I did not have to go far.

The fact is that there is always some kind of simple program behind the logic in music. But the key point of A New Kind of Science is that the main program may be simple, but produce a rich and complex picture .

But will there be aesthetics in it? As for the visual part, it has long been known to me that cellular automata can produce something very entertaining and interesting. And taking into account what we knew about cellular automata, this was not particularly surprising. Because I knew that they can grasp the essence of many processes in nature. And since we find nature aesthetic, the same should be true for cellular automata.

But while nature uses only a few specific kinds of rules, the universe of cellular automata is infinite. In a sense, the computational universe generalizes our real Universe. It contains the basic mechanisms, but allows an infinite number of variations, each with an aesthetic that summarizes the aesthetics of our world.

In the early 90s, Mathematicagot support for sound generation . And, with all the power of a symbolic language, Mathematica is the perfect platform to implement our algorithms and start creating music. The results far exceeded even our wildest expectations. We used ideas from the theory of music to dress up the still “bare” cellular automata, and very soon we were already creating orchestral music passages, and they sounded surprisingly good.

People passing by our office, hearing the music being played, often stopped and asked: “ What kind of song are you listening to? ” We made music that was good enough for people to consider it made by people; we can say that we have successfully passed the analogue of the Turing test for music.

Well, we recently finished making the Wolfram Tones website . And many began to use it. I must say that I thought it was just an interesting experiment, and maybe a good way to create simple melodies, that is, nothing serious was supposed from the point of view of music.

But I was wrong. Pretty quickly, a wide variety of composers began using the site. They tell us that they have found it useful as a source of ideas and inspiration for their compositions. It was a little strange. We started by challenging the impossibility of computers to achieve something close to human creativity. Nevertheless, people experienced in their field turned to our automatic system to search for what one might think was exceptional for a person: creative inspiration .

I saw this as a good test of the principle of computational equivalence. As one researcher put it: " It is worth hearing the music produced by simple programs, as they begin to look much more like us ."

In the computing world, each program actually defines its own artificial world - whose sounds and logic we perceive in the music it reproduces. Some of these worlds are boring, barren lands that generate dull and monotonous music. Others abound in randomness and noise. However, one of the hundreds or thousands of programs carries something beautiful: rich, textured, sometimes familiar, and sometimes exotic musical forms.


( Listen to the audio corresponding to this cellular automaton)

At Wolfram Tonesyou can press all kinds of buttons, looking at random for music that fits into the concept of the various musical genres we define. The site also gives the opportunity to gradually change the rules for musical fragments, making it possible to direct the music by artificial selection in the right direction. When you use, Wolfram Tones feels as if you are taking pictures of nature. We are exploring the computing universe in order to find the very places, the very programs that carry the meaning and aesthetics that we need.

Wolfram Tones was launched on September 16, 2005. And so, since then it is on the Internet and with the help of Mathematica creates music. I must admit that for a while I did not look at his logs. However, having looked there now, I found that it was used tens of millions of times to create tens of millions of musical compositions.


( Listen to the audio corresponding to this cellular automaton)

By the standards, for example, using Wolfram | Alpha is nothing. But for musical compositions, this is a huge number. Itunes currently contains about 14 million songs and represents the majority of what has ever been released. But in just a few years, Wolfram Tonescreated more songs. Using only calculations, in a sense he surpassed our view in creating musical production, alone creating more original music than all the time before it.

In order to produce results instantly, the site encodes music in MIDI (something that Mathematica now supports directly in symbolic form ). Many Wolfram Tones songs have been created in MP3 format. A complete redistribution of roles can be called the case when I went to a concert several years ago , where people played a piece on violins that was completely created using Wolfram Tones .

Can simple programs completely create an entire symphony? Songs in WolframTones tell us some stories from the computing world that have a duration of perhaps no more than a minute. Judging from my experience, in order to create a larger fragment - one that tells us some longer story - we will need higher-level structures. However, we do not have any obstacles in creating a simple program that would provide a similar structure. The dominant story can be absolutely complete, like many other stories that sound in our world with the music of the laws of nature.

However, how much can be gained from so little? What does the shortest program look like creating some interesting piece of music?

It is very easy to start creating programs in Mathematica :

Sound[
 	Map[
  		Function[
   			SoundNote[DeleteCases[3 * Range[21] * Reverse[#], 0] - 24,
    				0.1
    			]
   		],
  		Transpose @ CellularAutomaton[90, {{1}, 0}, 20]
  	]
 ]

( Listen to the audio corresponding to this cellular machine)

Sound[
 	Map[Function[SoundNote[#, 1 / 6, "Warm"]],
  		Map[Pick[{0, 5, 9, 12, 16, 21}, #, 1] &,
   			CellularAutomaton[30,
    				{{1, 0, 0, 0, 0, 0}, 0},
    				13,
    				{13, 5}
    			]
   		]
  	]
 ]

( Listen to the audio corresponding to this cellular automaton)

And we plan to organize a contest to see how good results can be achieved, especially using various modern algorithmic tools like image processing , which are available in Mathematica.  But in the end, we will not be able to rely on human creativity alone. After all, we want, in fact, to automate creativity, to go beyond what people could imagine, and not just explore the computing universe in search of interesting programs and ideas.

When creating music, we can work at the level of notes, combinations of notes or even sound signals, generalizing the waysconstruction of signals pleasant for a person , which traditionally used physical musical devices (or their digital analogues).

Of course, creativity in the computing universe is not limited to music. There was, for example, quite a lot of research regarding fine art, architecture. Can we create a building using just one simple rule? If so, then the very logic of the building structure will tell us a lot about ergonomics.

Can automatically created things, such as music or whatever, be truly valuable to us? Or do we always need a story that connects what we see with the whole body of human culture? Returning to the foregoing, our understanding of nature makes it clear that there is no need for human history. Instead, it seems necessary to link with a certain comprehensive logic, which, in essence, is precisely what the concept of the computing universe provides.

When I look at Wolfram | Alpha, I am captivated by the joy of realizing how much human knowledge can be captured and made computable. A new frontier is to capture not only knowledge, but also creativity. To be able, for example, to achieve a goal, to understand the creative path to achieving it. Music is the essence of pure creativity, and what we have learned, as the principle of computational equivalence suggests, is that even in this area, things like Wolfram Tones are surprisingly good at getting a creatively rich result.

We want to realize a higher level of automation, thereby dramatically expanding our creative capabilities, which, without a doubt, will add many exciting new features.