Creating Audio Plugins, Part 7

All posts in the series:
Part 1. Introduction and setup
Part 2. Learning code
Part 3. VST and AU
Part 4. Digital distortion
Part 5. Presets and GUI
Part 6. Signal synthesis
Part 7. Receiving MIDI messages
Part 8. Virtual keyboard
Part 9 Envelopes
Part 10. Refinement of the GUI
Part 11. Filter
Part 12. Low-frequency oscillator
Part 13. Redesign
Part 14. Polyphony 1
Part 15. Polyphony 2
Part 16. Antialiasing

---

So far, we have only generated a constant sound wave that just sounded at a given frequency. Let's see how you can respond to MIDI messages, turn on and off the generation of waves at the desired frequency, depending on the received note.

Receive MIDI Messages

MIDI Processing Basics

When the plugin is loaded into the host, it receives all MIDI messages from the track to which it is linked. When a note starts and ends, a function is called in the plugin ProcessMidiMsg. In addition to notes, MIDI messages can transmit portamento information (Pitch Bend) and control commands ( Control Changes , abbreviated CC), which can be used to automate plug-in parameters. A function ProcessMidiMsgis transmitted to the function IMidiMsgthat describes the MIDI event in its form independent of the format. In this description, there are parameters NoteNumber and Velocity, which contain information about the height of the sound volume of our oscillator.

Each time a MIDI message arrives, the system already plays the audio buffer that was previously filled. There is no way to cram new audio exactly when a MIDI message is received. These events need to remember before the next function call ProcessDoubleReplacing. It is also necessary to remember the time of receipt of the message, so we will leave this information intact for the next filling of the buffer.

The tool to perform these tasks will be IMidiQueue .

MIDI Recipient

We will use our Synthesis project . If you use a version control system, it's time to commit the project. Create a new class MIDIReceiver and check that the .cpp compiled for each Target. In MIDIReceiver.h, insert the interface between #defineand #endif:

#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wextra-tokens"
#include "IPlug_include_in_plug_hdr.h"
#pragma clang diagnostic pop
#include "IMidiQueue.h"
class MIDIReceiver {
private:
    IMidiQueue mMidiQueue;
    static const int keyCount = 128;
    int mNumKeys; // how many keys are being played at the moment (via midi)
    bool mKeyStatus[keyCount]; // array of on/off for each key (index is note number)
    int mLastNoteNumber;
    double mLastFrequency;
    int mLastVelocity;
    int mOffset;
    inline double noteNumberToFrequency(int noteNumber) { return 440.0 * pow(2.0, (noteNumber - 69.0) / 12.0); }
public:
    MIDIReceiver() :
    mNumKeys(0),
    mLastNoteNumber(-1),
    mLastFrequency(-1.0),
    mLastVelocity(0),
    mOffset(0) {
        for (int i = 0; i < keyCount; i++) {
            mKeyStatus[i] = false;
        }
    };
    // Returns true if the key with a given index is currently pressed
    inline bool getKeyStatus(int keyIndex) const { return mKeyStatus[keyIndex]; }
    // Returns the number of keys currently pressed
    inline int getNumKeys() const { return mNumKeys; }
    // Returns the last pressed note number
    inline int getLastNoteNumber() const { return mLastNoteNumber; }
    inline double getLastFrequency() const { return mLastFrequency; }
    inline int getLastVelocity() const { return mLastVelocity; }
    void advance();
    void onMessageReceived(IMidiMsg* midiMessage);
    inline void Flush(int nFrames) { mMidiQueue.Flush(nFrames); mOffset = 0; }
    inline void Resize(int blockSize) { mMidiQueue.Resize(blockSize); }
};

Here we need to enable IPlug_include_in_plug_hdr.h , because otherwise IMidiQueue.h will create errors.
As you can see, we have the privatefacility IMidiQueueto store a queue of MIDI messages. We also store information about which notes are currently played and how many of them are all played. Three parameters mLast...are needed, because our plugin will be monophonic: each next note will drown out the previous ones (the so-called priority of the last note ). The function noteNumberToFrequencyconverts the number of MIDI notes to a frequency in hertz. We use it because the class Oscillatorworks with frequency, not with note number.
The section publiccontains a number of built-in ( inline) getters and passes Flushand Resizeto the queuemMidiQueue.
In the body Flushwe set mOffsetequal to zero. A call mMidiQueue.Flush(nFrames)means that we remove part of the size from the beginning of the queue nFrames, since we already processed the events of this part in the previous function call advance. Zeroing mOffsetensures that the next time in the process of implementation advancewe will also handle the queue. The words constfollowing the brackets mean that the function will not change the immutable members of its class .

Let's add an implementation onMessageReceivedto MIDIReceiver.cpp :

void MIDIReceiver::onMessageReceived(IMidiMsg* midiMessage) {
    IMidiMsg::EStatusMsg status = midiMessage->StatusMsg();
    // We're only interested in Note On/Off messages (not CC, pitch, etc.)
    if(status == IMidiMsg::kNoteOn || status == IMidiMsg::kNoteOff) {
        mMidiQueue.Add(midiMessage);
    }
}

This function is called every time a MIDI message is received. We are currently only interested in note on and note off messages (start / stop playing a note), and we add them to the queue mMidiQueue.
The next interesting feature is advance:

void MIDIReceiver::advance() {
    while (!mMidiQueue.Empty()) {
        IMidiMsg* midiMessage = mMidiQueue.Peek();
        if (midiMessage->mOffset > mOffset) break;
        IMidiMsg::EStatusMsg status = midiMessage->StatusMsg();
        int noteNumber = midiMessage->NoteNumber();
        int velocity = midiMessage->Velocity();
        // There are only note on/off messages in the queue, see ::OnMessageReceived
        if (status == IMidiMsg::kNoteOn && velocity) {
            if(mKeyStatus[noteNumber] == false) {
                mKeyStatus[noteNumber] = true;
                mNumKeys += 1;
            }
            // A key pressed later overrides any previously pressed key:
            if (noteNumber != mLastNoteNumber) {
                mLastNoteNumber = noteNumber;
                mLastFrequency = noteNumberToFrequency(mLastNoteNumber);
                mLastVelocity = velocity;
            }
        } else {
            if(mKeyStatus[noteNumber] == true) {
                mKeyStatus[noteNumber] = false;
                mNumKeys -= 1;
            }
            // If the last note was released, nothing should play:
            if (noteNumber == mLastNoteNumber) {
                mLastNoteNumber = -1;
                mLastFrequency = -1;
                mLastVelocity = 0;
            }
        }
        mMidiQueue.Remove();
    }
    mOffset++;
}

This function is called every sample while the audio buffer is full. As long as there are messages in the queue, we process them and remove them from the beginning (using Peekand Remove). But we do this only for those MIDI messages whose offset ( mOffset) is not greater than the buffer offset. This means that we process each message in the corresponding sample, leaving the relative time shifts intact.
After reading the values noteNumber, the Velocityconditional statement ifseparates the messages note on and note off (the absence of a velocity value is interpreted as note off ). In both cases, we track which notes are played and how many of them are currently playing. Values ​​are also updated.mLast...to prioritize the last note. Further, it is logical that it is here that the frequency of sound should be updated, which we are doing. At the very end, it is mOffsetupdated so that the recipient knows how far in the buffer this message is at the moment. You can also tell the recipient this in another way - by passing the offset as an argument.
So, we have a class that receives all incoming MIDI note on / off messages. It keeps track of which notes are being played, what is the last note and what is its frequency. Let's use it.

Using a MIDI Recipient

To get started, carefully make these changes to resource.h :

// #define PLUG_CHANNEL_IO "1-1 2-2"
#if (defined(AAX_API) || defined(RTAS_API)) 
#define PLUG_CHANNEL_IO "1-1 2-2"
#else
// no audio input. mono or stereo output
#define PLUG_CHANNEL_IO "0-1 0-2"
#endif
// ...
#define PLUG_IS_INST 1
// ...
#define EFFECT_TYPE_VST3 "Instrument|Synth"
// ...
#define PLUG_DOES_MIDI 1

These lines tell the host that our plugin is "capable of midi." 0-1and 0-2indicate that the plugin does not have an audio input and there is one output, i.e. mono ( 0-1), or it does not have audio inputs and has stereo output ( 0-2).
Now add #include "MIDIReceiver.h"after Oscillator.hto Synthesis.h . In the same place, in the section public, add the declaration of the member function:

// to receive MIDI messages:
void ProcessMidiMsg(IMidiMsg* pMsg);

Add an object MIDIReceiverin the section private:

private:
    // ...
    MIDIReceiver mMIDIReceiver;

In Synthesis.cpp, write this simple function:

void Synthesis::ProcessMidiMsg(IMidiMsg* pMsg) {
    mMIDIReceiver.onMessageReceived(pMsg);
}

It is called every time a MIDI message is received, and we send the message to our recipient.
Let's clean things up a bit now. Edit both enumsat the top:

enum EParams
{
    kNumParams
};
enum ELayout
{
    kWidth = GUI_WIDTH,
    kHeight = GUI_HEIGHT
};

And create only one default preset:

void Synthesis::CreatePresets() {
    MakeDefaultPreset((char *) "-", kNumPrograms);
}

When changing the parameters of the plugin, nothing needs to be done:

void Synthesis::OnParamChange(int paramIdx)
{
    IMutexLock lock(this);
}

The handle in the interface will no longer be useful to us. Let's reduce the constructor to the minimum necessary:

Synthesis::Synthesis(IPlugInstanceInfo instanceInfo)
    :   IPLUG_CTOR(kNumParams, kNumPrograms, instanceInfo) {
    TRACE;
    IGraphics* pGraphics = MakeGraphics(this, kWidth, kHeight);
    pGraphics->AttachPanelBackground(&COLOR_RED);
    AttachGraphics(pGraphics);
    CreatePresets();
}

When the sound settings in the system change, we need to tell the oscillator a new sampling frequency:

void Synthesis::Reset()
{
    TRACE;
    IMutexLock lock(this);
    mOscillator.setSampleRate(GetSampleRate());
}

We still have a function ProcessDoubleReplacing. Think about it: mMIDIReceiver.advance()you need to perform each sample. After that, we find out the frequency and volume using getLastVelocityand getLastFrequencyfrom the MIDI receiver. Then call mOscillator.setFrequency()and mOscillator.generate()to fill the audio buffer the sound of the desired frequency.
The function generatewas created to process the whole buffer; the MIDI receiver works at the level of a separate sample: messages can have any offset within the buffer, which means it mLastFrequencycan change on any sample. We'll have to refine the class Oscillatorso that it also works at the sample level.

First we take it twoPIout generateand move it to the Oscillator.hprivate section . While we're here, let's add the linden variable right awaybool to indicate whether the oscillator is muted (i.e. no note is played):

const double twoPI;
bool isMuted;

We initialize them by adding a constructor to the initialization list. This is what it looks like:

Oscillator() :
    mOscillatorMode(OSCILLATOR_MODE_SINE),
    mPI(2*acos(0.0)),
    twoPI(2 * mPI), // This line is new
    isMuted(true),  // And this line
    mFrequency(440.0),
    mPhase(0.0),
    mSampleRate(44100.0) { updateIncrement(); };

Add the inline setter to the public section:

inline void setMuted(bool muted) { isMuted = muted; }

And immediately below this, insert the following line:

double nextSample();

We will call this function every sample and receive audio data from the oscillator.
Add the following code to Oscilator.cpp :

double Oscillator::nextSample() {
    double value = 0.0;
    if(isMuted) return value;
    switch (mOscillatorMode) {
        case OSCILLATOR_MODE_SINE:
            value = sin(mPhase);
            break;
        case OSCILLATOR_MODE_SAW:
            value = 1.0 - (2.0 * mPhase / twoPI);
            break;
        case OSCILLATOR_MODE_SQUARE:
            if (mPhase <= mPI) {
                value = 1.0;
            } else {
                value = -1.0;
            }
            break;
        case OSCILLATOR_MODE_TRIANGLE:
            value = -1.0 + (2.0 * mPhase / twoPI);
            value = 2.0 * (fabs(value) - 0.5);
            break;
    }
    mPhase += mPhaseIncrement;
    while (mPhase >= twoPI) {
        mPhase -= twoPI;
    }
    return value;
}

As you can see, it is used here twoPI. It would be redundant to calculate this value every sample, so we added two pi as a constant in the class.
When the oscillator does not generate anything, we return zero. The construction switchis already familiar to you, although here we do not use a cycle for. Here we simply generate a single value for the buffer, instead of filling it in whole. Also, a similar structure allows us to endure the phase increment, avoiding repetition.
This was a good example of refactoring, caused by insufficient code flexibility. Of course, we could think for an hour or two before starting to write a functiongeneratewith a “buffer” approach. But this implementation took us less than an hour entirely. In simple applications (such as this), it is sometimes more efficient to implement an approach and see how the code handles the task in practice. Most often, as we just saw, it turns out that the whole idea was correct (the principle of computing different sound waves), but some aspect of the problem was missed. On the other hand, if you are developing a public API, then changing something later, to put it mildly, is inconvenient, so you need to think it over in advance. In general, it depends on the situation.

The function setFrequencywill also be called every sample. So, updateIncrementit will also be called very often. But it is not yet optimized:

void Oscillator::updateIncrement() {
    mPhaseIncrement = mFrequency * 2 * mPI / mSampleRate;
}

2 * mPI * mSampleRatechanges only when the sampling rate changes. So the result of this calculation is better to remember and recount it only inside Oscillator::setSampleRate. It is worth remembering that transcendent optimization can make the code unreadable and even ugly. In the specific case, we will not have performance problems, because we are writing an elementary monophonic syntax. When we get to polyphony, it will be another matter, and then we will certainly optimize.
Now we can rewrite ProcessDoubleReplacingin Synthesis.cpp :

void Synthesis::ProcessDoubleReplacing(
    double** inputs,
    double** outputs,
    int nFrames)
{
    // Mutex is already locked for us.
    double *leftOutput = outputs[0];
    double *rightOutput = outputs[1];
    for (int i = 0; i < nFrames; ++i) {
        mMIDIReceiver.advance();
        int velocity = mMIDIReceiver.getLastVelocity();
        if (velocity > 0) {
            mOscillator.setFrequency(mMIDIReceiver.getLastFrequency());
            mOscillator.setMuted(false);
        } else {
            mOscillator.setMuted(true);
        }
        leftOutput[i] = rightOutput[i] = mOscillator.nextSample() * velocity / 127.0;
    }
    mMIDIReceiver.Flush(nFrames);
}

In a for loop, the MIDI receiver first updates the values ​​(called advance). If a note ( velocity > 0) sounds , we update the oscillator frequency and let it sound. Otherwise, we drown it (then it nextSamplewill return zeros).
Further, it all comes down to simply calling nextSampleto get the value, change the volume ( velocitythis is an integer between 0and 127) and write the result to the output buffers. At the end it is called Flushto remove the beginning of the queue.

Test

Run VST or AU. If the AU does not appear on the host, then you may have to change it PLUG_UNIQUE_IDin resource.h . If two plugins have the same ID, the host will ignore everything except one.
The plugin needs to send some MIDI data to the input. The easiest way is to use the REAPER's virtual keyboard ( View → Virtual MIDI Keyboard menu ). On the track with the plugin on the left there is a round red button. Go to the MIDI configuration by right-clicking on it and choose to receive messages from the virtual keyboard:



In the same menu, turn on Monitor Input . Now that the focus is on the virtual keyboard window, you can play the synthesizer with a regular keyboard. Enter your username or password from the password manager and listen to how it sounds.
If you have a MIDI keyboard, then by connecting it, you can test a stand-alone application. The main thing is to choose the correct MIDI input. If no sound is heard, try deleting ~ / Library / Application Support / Synthesis / settings.ini .

The entire project at this stage can be downloaded from here .

Next time we will add a nice keyboard to the interface :)

Original article:
martin-finke.de/blog/articles/audio-plugins-009-receiving-midi