Rhythm synchronization in music games

I recently started work at Unity on the beatbox music game Boots-Cut . In the process of prototyping the basic mechanics of the game, I found that it is quite difficult to correctly synchronize notes with music. There are quite a few articles on the Internet on this topic. Therefore, in my article I will try to give the most important tips on developing a music game (especially in Unity).

It turned out that the following three aspects are most important:

• Use AudioSettings.dspTimeinstead Time.timeSinceLevelLoadto track the position in a song.
• You must always use the position in the song to update the movements.
• Do not update notes in each frame according to the time difference, interpolate them.

We will take this into account and get to work!

Main class

You need to create a class SongManagerto track the position in the song, create notes and other song management features.

Position tracking

In all musical games, you need to track the position in the song in order to know which note should be created. Below are the fields required to track the position in a song:

//текущая позиция в песне (в секундах)
float songPosition;
//текущая позиция в песне (в ударах)
float songPosInBeats;
//длительность удара
float secPerBeat;
//сколько времени (в секундах) прошло после начала песни
float dsptimesong;

We initialize these fields in the function Start():

void Start()
{
    //вычисление количества секунд в одном ударе
    //объявление bpm выполняется ниже
    secPerBeat = 60f / bpm;
    //запись времени начала песни
    dsptimesong = (float) AudioSettings.dspTime;
    //начало песни
    GetComponent().Play();
}

For convenience, we will convert bpmto secPerBeat. Later it secPerBeatwill be used to calculate the position in the song in beats, which is very important for creating notes.

In addition, we record the song's start time at dsptimesong. We use AudioSettings.dspTimeinstead Time.timeSinceLevelLoad, because it is Time.timeSinceLevelLoadupdated only in each frame, and is AudioSettings.dspTimeupdated more often, since this is an audio timer. To maintain the tempo of a song, you need to use an audio timer. In this way, we can avoid the delay caused by the time difference between frame updates and audio updates.

The function Update()calculates the position in the song using AudioSettings.dspTime:

void Update()
{
    //вычисление позиции в секундах
    songPosition = (float) (AudioSettings.dspTime - dsptimesong);
    //вычисление позиции в ударах
    songPosInBeats = songPosition / secPerBeat;
}

We calculate the position in seconds by subtracting the current AudioSettings.dspTimestart time of the song ( dsptimesong) from the current time . We got a position in seconds, but in the world of music, notes are recorded in beats. Therefore, it is better to convert the position in seconds to the position in beats. Dividing songPositionby secPerBeat(second / (second / hit)), we get the position in beats.

Look at the picture: The



position of the notes in the beats: 1, 2, 2.5, 3, 3.5, 4.5, and the duration of the beat is 0.5 s. Therefore, if 1.75 s ( songPosition == 1.75) elapsed after the start of the song , then we know that we are in position 1.75  ( songPosition) / 0.5  ( secPerBeat) = 3.5 beats, and it is necessary to create a note for beat 3.5.

Song info

Let's move on to the fields in which we recorded information about the song:

//количество ударов в минуту
float bpm;
//сохранение всех позиций нот в ударах
float[] notes;
//индекс ноты, которую нужно создать следующей
int nextIndex = 0;

For simplicity, I demonstrate a song with only one track of notes ( three tracks were made in Guitar Hero Mobile , and only one in Taikono Tatsujin ).

bpmIs the number of beats per minute. As we saw, for convenience they are converted to secPerBeat.

notes- This is an array in which all notes positions in beats are stored. For example, for the notes shown in the figure, the array noteswill contain {1f, 2f, 2.5f, 3f, 3.5f, 4.5f}:



And, finally, nextIndexthis is the integer needed to traverse the array. It is initialized to 0 because the next note to be created will be the first note in the song. When creating a note, the counter is nextIndexincremented by one.

Creating notes

We determine whether a note should be created in a function Update(). However, you must first determine how many strokes will be shown in advance.

For example, for the following track: the



current position in beats is 1, but beat 3 has already been created. This means that 3 hits are shown in advance.

Add below the songPosInBeats = songPosition / secPerBeat;following lines:

if (nextIndex < notes.Length && notes[nextIndex] < songPosInBeats + beatsShownInAdvance)
{
    Instantiate( /* префаб ноты */ );
    //инициализация полей ноты
    nextIndex++;
}

First you need to check if there are any notes left in the song ( nextIndex < notes.Length). If there are still notes, then we check to see if the song hit the beat where the next note ( notes[nextIndex] < songPosInBeats + beatsShownInAdvance) should be created . If reached, create a note and increase nextIndexto track the next note to be created.

Movement of notes

Finally, let's talk about how to move the created notes according to the tempo of the song. This is quite simple, if you recall the item “Do not update notes in each frame according to the time difference, interpolate them.”

Always update movement by position in a song because:

1. The audio timer has a time difference with the frame timer
2. Beats can be exactly in the middle of two frames (which leads to a time difference)

So how do you move the notes? By interpolation!

To simplify, I cut out all the code in the class MusicNoteand leave only the function Update()in which we move each note:

//функция обновления нот
void Update()
{
    transform.position = Vector2.Lerp(
        SpawnPos,
        RemovePos,
        (BeatsShownInAdvance - (beatOfThisNote - songPosInBeats)) / BeatsShownInAdvance
    );    
}

In the diagram below, this is clearly visible:

Conclusion

I talked about the basics of programming a music game. Following these principles, you can create games with synchronization. In games with several tracks, you can create nested arrays notes, deleting notes is performed by checking the position relative to the deletion line, notes of long duration are implemented by tracking the initial and final beat, etc.

Thank you for reading the article, I hope it will be useful. My own music game Boots-Cuts will be ready next year, stay tuned.