Introduction

So, you want or tried to create a rhythm game, but game elements and music quickly out of sync, and now you do not know what to do. This article will help you with this. I played rhythm games from high school and often hung out on DDR in the local arcade hall. Today I am always looking for new games of this genre, and projects such as Crypt of the Necrodancer or Bit.Trip.Runner show that much more can be done in this genre. I worked a little on prototypes of rhythm games in Unity, and as a result I spent a month creating a short rhythm game / puzzle Atomic Beats . In this article, I will talk about the most useful code-building techniques that I learned in creating these games. I could not find information about them anywhere else, or it was presented in less detail.

First, I must express my deep gratitude to Yu Chao for the post of Music Syncing in Rhythm Games [ translation into Habré ]. Yu reviewed the basics of synchronizing audio timings with the game engine in Unity and uploaded the source code for his Boots-Cut game, which helped me a lot in creating my project. You can study his post if you want to learn a brief introduction to Unity's music synchronization, but I will cover this topic in more detail and more extensively. My code actively uses information from the article and Boots-Cut code.

At the heart of any rhythm game are timings. People are extremely sensitive to any distortion in the rhythm timings, so it is very important that all actions, movements and input in the rhythm game are directly synchronized with the music. Unfortunately, traditional Unity time tracking methods like Time.timeSinceLevelLoad and Time.time quickly lose synchronization with the sound being played. Therefore, we will gain direct access to the audio system using AudioSettings.dspTime, which uses the true number of audio samples processed by the audio system. Thanks to this, it always maintains synchronization with the music being played back (perhaps this is not the case with very long audio files, when sampling effects come into play, but in the case of songs of a normal length, the system should work perfectly). This function will be the core of our composition time tracking, and based on it we will create the main class.

Class conductor

The Conductor class is the main composition management class on the basis of which the rest of the rhythm game will be built. With it, we will track the position of the composition and manage all other synchronized actions. To track the composition we need a few variables

//Song beats per minute
//This is determined by the song you're trying to sync up to
public float songBpm;
//The number of seconds for each song beat
public float secPerBeat;
//Current song position, in seconds
public float songPosition;
//Current song position, in beats
public float songPositionInBeats;
//How many seconds have passed since the song started
public float dspSongTime;
//an AudioSource attached to this GameObject that will play the music.
public AudioSource musicSource;

When starting the scene, we need to perform calculations to determine the variables, and also record for reference the start time of the composition.

void Start()
{
    //Load the AudioSource attached to the Conductor GameObject
    musicSource = GetComponent();
    //Calculate the number of seconds in each beat
    secPerBeat = 60f / songBpm;
    //Record the time when the music starts
    dspSongTime = (float)AudioSettings.dspTime;
    //Start the music
    musicSource.Play();
}

If you create an empty GameObject with such a script attached to it, and then add the Audio Source with the composition and run the program, you will see that the script will record the start time of the composition, but nothing else will happen. We will also need to manually enter the BPM of the music that we add to the Audio Source.


Thanks to all these values, we can track the position in the composition in real time when updating the game. We will determine the timing of the composition, first in seconds, then in fractions. Fractions are a much more convenient way to track a composition, because they allow us to add actions and timings in time in parallel with the composition, for example, in fractions 1, 3 and 5.5, without the need to calculate seconds between fractions. Add the following calculations to the Update () function of the Conductor class:

void Update()
{
    //determine how many seconds since the song started
    songPosition = (float)(AudioSettings.dspTime - dspSongTime);
    //determine how many beats since the song started
    songPositionInBeats = songPosition / secPerBeat;
}

So we get the difference between the current time according to the audio system and the start time of the composition, which gives the total number of seconds that the composition is played. We will save it in the songPosition variable.


Note that the score in music usually starts with a unit with fractions 1-2-3-4 and so on, and songPositionInBeats starts at 0 and increases from this value, so the third part of the composition will correspond to songPositionInBeats, which is 2.0, not 3.0.

At this point, if you want to create a traditional Dance Dance Revolution-style game, then you need to create notes according to the fraction that you need to press them in, interpolate their position relative to the click line, and then record songPositionInBeats when the key is pressed, and Compare the value with the desired proportion of notes. Yu Chao discusses an example of such a scheme in his article . In order not to repeat myself, I will consider other potentially useful techniques that can be built on top of the Conductor class. I used them when creatingAtomic Beats .

We adapt to the initial share

If you create your own music for a rhythm game, it is easy to make the first beat exactly match the beginning of the music, which, if correctly specified, will reliably bind the Conductor class songPositionInBeats to the composition.


However, if you use ready-made music, then there is a high probability that there is a slight pause before the beginning of the composition. If this is not taken into account, then the Conductor class songPositionInBeats will think that the first beat started when the song started playing, and not the beat now. Everything that will be further tied to the values ​​of the shares is not synchronized with the music.


To fix this, you can add a variable that takes this offset into account. Add the following to the Conductor class:

//The offset to the first beat of the song in seconds
public float firstBeatOffset;

In Update (), the songPosition variable:

songPosition = (float)(AudioSettings.dspTime - dspSongTime);

replaced by:

songPosition = (float)(AudioSettings.dspTime - dspSongTime - firstBeatOffset);

Now songPosition will correctly calculate the position in the song, taking into account the true first beat. However, you will have to manually enter the offset to the first beat, so for each file it will be unique. In addition, during this shift there will be a short window in which songPosition will turn out to be negative. This may not affect the game, but some code, depending on the values ​​of songPosition or songPositionInBeats, may not be able to process negative numbers at this time.

Repetitions

If you work with a composition that plays from beginning to end, then the Conductor class shown above will be enough to track the position. But if you have a short track that is looped and you want to work with this loop, you need to build in Repeater support in Conductor.

If you have a perfectly looped fragment (for example, if the song tempo is 120bpm, and the looped fragment has a length of 4 beats, then it should be exactly 8.0 seconds at 2.0 seconds per share) loaded into the Audio Source class of the Conductor class, then check the loop box. Conductor will work the same as before and pass the total time after the first to songPositionlaunch the clip. To determine the position of the loop, we need to somehow tell Conductor how many shares are in one loop and how many loops have already been played. Add the following variables to the Conductor class:

//the number of beats in each loop
public float beatsPerLoop;
//the total number of loops completed since the looping clip first started
public int completedLoops = 0;
//The current position of the song within the loop in beats.
public float loopPositionInBeats;

Now with each update to SongPositionInBeats, we can also update the Update () position of the loop.

//calculate the loop position
    if (songPositionInBeats >= (completedLoops + 1) * beatsPerLoop)
        completedLoops++;
    loopPositionInBeats = songPositionInBeats - completedLoops * beatsPerLoop;

This gives us a marker that tells loopPositionInBeats how many shares we went through the loop, which is useful for many other synchronized items. Remember to enter the number of shares of the loop in GameObject Conductor.

We should also carefully consider the calculation of shares. Music always starts at 1, so the 4-part measurement takes the form 1-2-3-4-, and in our class loopPositionInBeats starts at 0.0 and loops over 4.0. Therefore, the exact middle of the loop, which when calculating the musical proportions will be 3, in loopPositionInBeats will have a value of 2.0. You can modify loopPositionInBeats to take this into account, but this will affect all other calculations, so be careful when inserting notes.

Also for the remaining tools it will be useful to add two more aspects to the Conductor class. Firstly, an analog version of LoopPositionInBeats called LoopPositionInAnalog, which measures the position in the loop in the range from 0 to 1.0. The second is an instance of the Conductor class for convenient calling from other classes. Add the following variables to the Conductor class:

//The current relative position of the song within the loop measured between 0 and 1.
public float loopPositionInAnalog;
//Conductor instance
public static Conductor instance;

In the Awake () function, add:

void Awake()
{
    instance = this;
}

and add to the Update () function:

loopPositionInAnalog = loopPositionInBeats / beatsPerLoop;

Turn Sync

It would be very useful to synchronize movement or rotation with the lobes so that the elements are in the right places. In my Atomic Beats game, I used this to dynamically rotate notes around a central axis. Initially, they were placed around the circumference in accordance with their share inside the loop, and then the entire playing area was rotated so that the notes were matched with the line of depression in their share.

To achieve this, create a new script called SyncedRotation, and attach it to the GameObject that you want to rotate. Add to the Update () function of the SyncedRotation script:

void Update()
{
    this.gameObject.transform.rotation = Quaternion.Euler(0, 0, Mathf.Lerp(0, 360, Conductor.instance.loopPositionInAnalog));
}

This code will interpolate the rotation of the GameObject that this game is tied to in the interval from 0 to 360 degrees, turning it so that it completes one full revolution at the end of each loop. This is useful as an example, but for looping or frame-by-frame animation, it would be more useful to sync loop animations so that they fit perfectly with the tempo.

Animation Sync

Unity Animator is extremely powerful, but not always accurate. For reliable alignment of animations and music, I had to compete with the Animator class and its tendency to gradually dissynchronize with the pace. In addition, it was difficult to adjust the same animations to different tempos, so that when switching between compositions, you did not have to redefine the key frames of the animation to the current tempo. Instead, we can go directly to the animation loop, and set the position in this loop according to where we are in the loop of the Conductor class.

First, create a new class called SyncedAnimation, and add the following variables to it:

//The animator controller attached to this GameObject
public Animator animator;
//Records the animation state or animation that the Animator is currently in
public AnimatorStateInfo animatorStateInfo;
//Used to address the current state within the Animator using the Play() function
public int currentState;

Attach it to a new or existing GameObject that you want to animate. In this example, we will simply move the object back and forth across the screen, but the same principle can be applied to any animation, be it before setting the property, or frame-by-frame animation. Add an Animator element to GameObject and create a new Animator Controller called SyncedAnimController, as well as an Animation Clip called BackAndForth. We load the controller into the Animator class attached to the GameObject, and add Animation to the animation tree as the default animation.


For example, I set up the animation so that it first moves the object to the right by 6 units, then to the left by -6, and then back to 0.


Now, to synchronize the animation, add the following code to the Start () function of the SyncedAnimation class, which initializes information about the Animator:

void Start()
{
    //Load the animator attached to this object
    animator = GetComponent();
    //Get the info about the current animator state
    animatorStateInfo = animator.GetCurrentAnimatorStateInfo(0);
    //Convert the current state name to an integer hash for identification
    currentState = animatorStateInfo.fullPathHash;
}

Then add the following code to Update () to set the animation:

void Update()
{
    //Start playing the current animation from wherever the current conductor loop is
    animator.Play(currentState, -1, (Conductor.instance.loopPositionInAnalog));
    //Set the speed to 0 so it will only change frames when you next update it
    animator.speed = 0;
}

So we position the animation in the exact frame of the animation relative to one full loop. For example, if you use the animation above, when you are in the middle of the loop, the GameObject position will just cross 0. This can be applied to any animation you create that you want to synchronize with the Conductor tempo.

It is also worth noting that to create a seamless loop of animations, you need to configure the tangents of individual keyframes of the animation on the animation curve. The Linear setting will create a straight line going from one key frame to the next, and Constant will keep the animation in one value until the next key frame, which will give a jerky and sharp movement.


Although this method is useful, it affects all transitions of the animation, because it causes animationState to remain in the state in which it was when the script was initially run. This method is useful for objects that need only endlessly use one synchronized animation, but to create more complex objects with different synchronized animations, you need to add code that processes these transitions and sets the currentState variable in accordance with the desired animation state.

Conclusion

These are just some of the aspects that have been helpful to me in creating Atomic Beats. Some of them were collected from other sources or created out of necessity, but most of them I could not find in the finished form, so I hope this comes in handy! Perhaps part of my system will no longer be useful in large projects due to CPU or audio system limitations, but it will be a good foundation for playing a game jam or a hobby project.


Creating a rhythm game, or game elements synchronized with music, can be difficult. To keep everything at a consistent pace, you may need a tricky code; a result that allows you to play at a constant pace can be very attractive to the player. Much more can be done in this genre than games in the traditional Dance Dance Revolution style, and I hope this article helps you realize such projects. I also recommend, if possible, evaluate my Atomic Beats game . I made it in one month in the spring of this year, it has 8 short tracks and it is free!