
Creating Audio Plugins, Part 7
- Tutorial
All posts in the series:
Part 1. Introduction and setup
Part 2. Learning code
Part 3. VST and AU
Part 4. Digital distortion
Part 5. Presets and GUI
Part 6. Signal synthesis
Part 7. Receiving MIDI messages
Part 8. Virtual keyboard
Part 9 Envelopes
Part 10. Refinement of the GUI
Part 11. Filter
Part 12. Low-frequency oscillator
Part 13. Redesign
Part 14. Polyphony 1
Part 15. Polyphony 2
Part 16. Antialiasing
So far, we have only generated a constant sound wave that just sounded at a given frequency. Let's see how you can respond to MIDI messages, turn on and off the generation of waves at the desired frequency, depending on the received note.
When the plugin is loaded into the host, it receives all MIDI messages from the track to which it is linked. When a note starts and ends, a function is called in the plugin
Each time a MIDI message arrives, the system already plays the audio buffer that was previously filled. There is no way to cram new audio exactly when a MIDI message is received. These events need to remember before the next function call
The tool to perform these tasks will be IMidiQueue .
We will use our Synthesis project . If you use a version control system, it's time to commit the project. Create a new class MIDIReceiver and check that the .cpp compiled for each Target. In MIDIReceiver.h, insert the interface between
Here we need to enable IPlug_include_in_plug_hdr.h , because otherwise IMidiQueue.h will create errors.
As you can see, we have the
The section
In the body
Let's add an implementation
This function is called every time a MIDI message is received. We are currently only interested in note on and note off messages (start / stop playing a note), and we add them to the queue
The next interesting feature is
This function is called every sample while the audio buffer is full. As long as there are messages in the queue, we process them and remove them from the beginning (using
After reading the values
So, we have a class that receives all incoming MIDI note on / off messages. It keeps track of which notes are being played, what is the last note and what is its frequency. Let's use it.
To get started, carefully make these changes to resource.h :
These lines tell the host that our plugin is "capable of midi."
Now add
Add an object
In Synthesis.cpp, write this simple function:
It is called every time a MIDI message is received, and we send the message to our recipient.
Let's clean things up a bit now. Edit both
And create only one default preset:
When changing the parameters of the plugin, nothing needs to be done:
The handle in the interface will no longer be useful to us. Let's reduce the constructor to the minimum necessary:
When the sound settings in the system change, we need to tell the oscillator a new sampling frequency:
We still have a function
The function
First we take it
We initialize them by adding a constructor to the initialization list. This is what it looks like:
Add the inline setter to the public section:
And immediately below this, insert the following line:
We will call this function every sample and receive audio data from the oscillator.
Add the following code to Oscilator.cpp :
As you can see, it is used here
When the oscillator does not generate anything, we return zero. The construction
This was a good example of refactoring, caused by insufficient code flexibility. Of course, we could think for an hour or two before starting to write a function
The function
Now we can rewrite
In a for loop, the MIDI receiver first updates the values (called
Further, it all comes down to simply calling
Run VST or AU. If the AU does not appear on the host, then you may have to change it
The plugin needs to send some MIDI data to the input. The easiest way is to use the REAPER's virtual keyboard ( View → Virtual MIDI Keyboard menu ). On the track with the plugin on the left there is a round red button. Go to the MIDI configuration by right-clicking on it and choose to receive messages from the virtual keyboard:
In the same menu, turn on Monitor Input . Now that the focus is on the virtual keyboard window, you can play the synthesizer with a regular keyboard. Enter your username or password from the password manager and listen to how it sounds.
If you have a MIDI keyboard, then by connecting it, you can test a stand-alone application. The main thing is to choose the correct MIDI input. If no sound is heard, try deleting ~ / Library / Application Support / Synthesis / settings.ini .
The entire project at this stage can be downloaded from here .
Next time we will add a nice keyboard to the interface :)
Original article:
martin-finke.de/blog/articles/audio-plugins-009-receiving-midi
Part 1. Introduction and setup
Part 2. Learning code
Part 3. VST and AU
Part 4. Digital distortion
Part 5. Presets and GUI
Part 6. Signal synthesis
Part 7. Receiving MIDI messages
Part 8. Virtual keyboard
Part 9 Envelopes
Part 10. Refinement of the GUI
Part 11. Filter
Part 12. Low-frequency oscillator
Part 13. Redesign
Part 14. Polyphony 1
Part 15. Polyphony 2
Part 16. Antialiasing
So far, we have only generated a constant sound wave that just sounded at a given frequency. Let's see how you can respond to MIDI messages, turn on and off the generation of waves at the desired frequency, depending on the received note.
Receive MIDI Messages
MIDI Processing Basics
When the plugin is loaded into the host, it receives all MIDI messages from the track to which it is linked. When a note starts and ends, a function is called in the plugin
ProcessMidiMsg
. In addition to notes, MIDI messages can transmit portamento information (Pitch Bend) and control commands ( Control Changes , abbreviated CC), which can be used to automate plug-in parameters. A function ProcessMidiMsg
is transmitted to the function IMidiMsg
that describes the MIDI event in its form independent of the format. In this description, there are parameters NoteNumber
and Velocity
, which contain information about the height of the sound volume of our oscillator.Each time a MIDI message arrives, the system already plays the audio buffer that was previously filled. There is no way to cram new audio exactly when a MIDI message is received. These events need to remember before the next function call
ProcessDoubleReplacing
. It is also necessary to remember the time of receipt of the message, so we will leave this information intact for the next filling of the buffer. The tool to perform these tasks will be IMidiQueue .
MIDI Recipient
We will use our Synthesis project . If you use a version control system, it's time to commit the project. Create a new class MIDIReceiver and check that the .cpp compiled for each Target. In MIDIReceiver.h, insert the interface between
#define
and #endif
:#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wextra-tokens"
#include "IPlug_include_in_plug_hdr.h"
#pragma clang diagnostic pop
#include "IMidiQueue.h"
class MIDIReceiver {
private:
IMidiQueue mMidiQueue;
static const int keyCount = 128;
int mNumKeys; // how many keys are being played at the moment (via midi)
bool mKeyStatus[keyCount]; // array of on/off for each key (index is note number)
int mLastNoteNumber;
double mLastFrequency;
int mLastVelocity;
int mOffset;
inline double noteNumberToFrequency(int noteNumber) { return 440.0 * pow(2.0, (noteNumber - 69.0) / 12.0); }
public:
MIDIReceiver() :
mNumKeys(0),
mLastNoteNumber(-1),
mLastFrequency(-1.0),
mLastVelocity(0),
mOffset(0) {
for (int i = 0; i < keyCount; i++) {
mKeyStatus[i] = false;
}
};
// Returns true if the key with a given index is currently pressed
inline bool getKeyStatus(int keyIndex) const { return mKeyStatus[keyIndex]; }
// Returns the number of keys currently pressed
inline int getNumKeys() const { return mNumKeys; }
// Returns the last pressed note number
inline int getLastNoteNumber() const { return mLastNoteNumber; }
inline double getLastFrequency() const { return mLastFrequency; }
inline int getLastVelocity() const { return mLastVelocity; }
void advance();
void onMessageReceived(IMidiMsg* midiMessage);
inline void Flush(int nFrames) { mMidiQueue.Flush(nFrames); mOffset = 0; }
inline void Resize(int blockSize) { mMidiQueue.Resize(blockSize); }
};
Here we need to enable IPlug_include_in_plug_hdr.h , because otherwise IMidiQueue.h will create errors.
As you can see, we have the
private
facility IMidiQueue
to store a queue of MIDI messages. We also store information about which notes are currently played and how many of them are all played. Three parameters mLast...
are needed, because our plugin will be monophonic: each next note will drown out the previous ones (the so-called priority of the last note ). The function noteNumberToFrequency
converts the number of MIDI notes to a frequency in hertz. We use it because the class Oscillator
works with frequency, not with note number. The section
public
contains a number of built-in ( inline
) getters and passes Flush
and Resize
to the queuemMidiQueue
. In the body
Flush
we set mOffset
equal to zero. A call mMidiQueue.Flush(nFrames)
means that we remove part of the size from the beginning of the queue nFrames
, since we already processed the events of this part in the previous function call advance
. Zeroing mOffset
ensures that the next time in the process of implementation advance
we will also handle the queue. The words const
following the brackets mean that the function will not change the immutable members of its class . Let's add an implementation
onMessageReceived
to MIDIReceiver.cpp :void MIDIReceiver::onMessageReceived(IMidiMsg* midiMessage) {
IMidiMsg::EStatusMsg status = midiMessage->StatusMsg();
// We're only interested in Note On/Off messages (not CC, pitch, etc.)
if(status == IMidiMsg::kNoteOn || status == IMidiMsg::kNoteOff) {
mMidiQueue.Add(midiMessage);
}
}
This function is called every time a MIDI message is received. We are currently only interested in note on and note off messages (start / stop playing a note), and we add them to the queue
mMidiQueue
. The next interesting feature is
advance
:void MIDIReceiver::advance() {
while (!mMidiQueue.Empty()) {
IMidiMsg* midiMessage = mMidiQueue.Peek();
if (midiMessage->mOffset > mOffset) break;
IMidiMsg::EStatusMsg status = midiMessage->StatusMsg();
int noteNumber = midiMessage->NoteNumber();
int velocity = midiMessage->Velocity();
// There are only note on/off messages in the queue, see ::OnMessageReceived
if (status == IMidiMsg::kNoteOn && velocity) {
if(mKeyStatus[noteNumber] == false) {
mKeyStatus[noteNumber] = true;
mNumKeys += 1;
}
// A key pressed later overrides any previously pressed key:
if (noteNumber != mLastNoteNumber) {
mLastNoteNumber = noteNumber;
mLastFrequency = noteNumberToFrequency(mLastNoteNumber);
mLastVelocity = velocity;
}
} else {
if(mKeyStatus[noteNumber] == true) {
mKeyStatus[noteNumber] = false;
mNumKeys -= 1;
}
// If the last note was released, nothing should play:
if (noteNumber == mLastNoteNumber) {
mLastNoteNumber = -1;
mLastFrequency = -1;
mLastVelocity = 0;
}
}
mMidiQueue.Remove();
}
mOffset++;
}
This function is called every sample while the audio buffer is full. As long as there are messages in the queue, we process them and remove them from the beginning (using
Peek
and Remove
). But we do this only for those MIDI messages whose offset ( mOffset
) is not greater than the buffer offset. This means that we process each message in the corresponding sample, leaving the relative time shifts intact. After reading the values
noteNumber
, the Velocity
conditional statement if
separates the messages note on and note off (the absence of a velocity value is interpreted as note off ). In both cases, we track which notes are played and how many of them are currently playing. Values are also updated.mLast...
to prioritize the last note. Further, it is logical that it is here that the frequency of sound should be updated, which we are doing. At the very end, it is mOffset
updated so that the recipient knows how far in the buffer this message is at the moment. You can also tell the recipient this in another way - by passing the offset as an argument. So, we have a class that receives all incoming MIDI note on / off messages. It keeps track of which notes are being played, what is the last note and what is its frequency. Let's use it.
Using a MIDI Recipient
To get started, carefully make these changes to resource.h :
// #define PLUG_CHANNEL_IO "1-1 2-2"
#if (defined(AAX_API) || defined(RTAS_API))
#define PLUG_CHANNEL_IO "1-1 2-2"
#else
// no audio input. mono or stereo output
#define PLUG_CHANNEL_IO "0-1 0-2"
#endif
// ...
#define PLUG_IS_INST 1
// ...
#define EFFECT_TYPE_VST3 "Instrument|Synth"
// ...
#define PLUG_DOES_MIDI 1
These lines tell the host that our plugin is "capable of midi."
0-1
and 0-2
indicate that the plugin does not have an audio input and there is one output, i.e. mono ( 0-1
), or it does not have audio inputs and has stereo output ( 0-2
). Now add
#include "MIDIReceiver.h"
after Oscillator.h
to Synthesis.h . In the same place, in the section public
, add the declaration of the member function:// to receive MIDI messages:
void ProcessMidiMsg(IMidiMsg* pMsg);
Add an object
MIDIReceiver
in the section private
:private:
// ...
MIDIReceiver mMIDIReceiver;
In Synthesis.cpp, write this simple function:
void Synthesis::ProcessMidiMsg(IMidiMsg* pMsg) {
mMIDIReceiver.onMessageReceived(pMsg);
}
It is called every time a MIDI message is received, and we send the message to our recipient.
Let's clean things up a bit now. Edit both
enums
at the top:enum EParams
{
kNumParams
};
enum ELayout
{
kWidth = GUI_WIDTH,
kHeight = GUI_HEIGHT
};
And create only one default preset:
void Synthesis::CreatePresets() {
MakeDefaultPreset((char *) "-", kNumPrograms);
}
When changing the parameters of the plugin, nothing needs to be done:
void Synthesis::OnParamChange(int paramIdx)
{
IMutexLock lock(this);
}
The handle in the interface will no longer be useful to us. Let's reduce the constructor to the minimum necessary:
Synthesis::Synthesis(IPlugInstanceInfo instanceInfo)
: IPLUG_CTOR(kNumParams, kNumPrograms, instanceInfo) {
TRACE;
IGraphics* pGraphics = MakeGraphics(this, kWidth, kHeight);
pGraphics->AttachPanelBackground(&COLOR_RED);
AttachGraphics(pGraphics);
CreatePresets();
}
When the sound settings in the system change, we need to tell the oscillator a new sampling frequency:
void Synthesis::Reset()
{
TRACE;
IMutexLock lock(this);
mOscillator.setSampleRate(GetSampleRate());
}
We still have a function
ProcessDoubleReplacing
. Think about it: mMIDIReceiver.advance()
you need to perform each sample. After that, we find out the frequency and volume using getLastVelocity
and getLastFrequency
from the MIDI receiver. Then call mOscillator.setFrequency()
and mOscillator.generate()
to fill the audio buffer the sound of the desired frequency. The function
generate
was created to process the whole buffer; the MIDI receiver works at the level of a separate sample: messages can have any offset within the buffer, which means it mLastFrequency
can change on any sample. We'll have to refine the class Oscillator
so that it also works at the sample level. First we take it
twoPI
out generate
and move it to the Oscillator.hprivate
section . While we're here, let's add the linden variable right awaybool
to indicate whether the oscillator is muted (i.e. no note is played):const double twoPI;
bool isMuted;
We initialize them by adding a constructor to the initialization list. This is what it looks like:
Oscillator() :
mOscillatorMode(OSCILLATOR_MODE_SINE),
mPI(2*acos(0.0)),
twoPI(2 * mPI), // This line is new
isMuted(true), // And this line
mFrequency(440.0),
mPhase(0.0),
mSampleRate(44100.0) { updateIncrement(); };
Add the inline setter to the public section:
inline void setMuted(bool muted) { isMuted = muted; }
And immediately below this, insert the following line:
double nextSample();
We will call this function every sample and receive audio data from the oscillator.
Add the following code to Oscilator.cpp :
double Oscillator::nextSample() {
double value = 0.0;
if(isMuted) return value;
switch (mOscillatorMode) {
case OSCILLATOR_MODE_SINE:
value = sin(mPhase);
break;
case OSCILLATOR_MODE_SAW:
value = 1.0 - (2.0 * mPhase / twoPI);
break;
case OSCILLATOR_MODE_SQUARE:
if (mPhase <= mPI) {
value = 1.0;
} else {
value = -1.0;
}
break;
case OSCILLATOR_MODE_TRIANGLE:
value = -1.0 + (2.0 * mPhase / twoPI);
value = 2.0 * (fabs(value) - 0.5);
break;
}
mPhase += mPhaseIncrement;
while (mPhase >= twoPI) {
mPhase -= twoPI;
}
return value;
}
As you can see, it is used here
twoPI
. It would be redundant to calculate this value every sample, so we added two pi as a constant in the class. When the oscillator does not generate anything, we return zero. The construction
switch
is already familiar to you, although here we do not use a cycle for
. Here we simply generate a single value for the buffer, instead of filling it in whole. Also, a similar structure allows us to endure the phase increment, avoiding repetition. This was a good example of refactoring, caused by insufficient code flexibility. Of course, we could think for an hour or two before starting to write a function
generate
with a “buffer” approach. But this implementation took us less than an hour entirely. In simple applications (such as this), it is sometimes more efficient to implement an approach and see how the code handles the task in practice. Most often, as we just saw, it turns out that the whole idea was correct (the principle of computing different sound waves), but some aspect of the problem was missed. On the other hand, if you are developing a public API, then changing something later, to put it mildly, is inconvenient, so you need to think it over in advance. In general, it depends on the situation. The function
setFrequency
will also be called every sample. So, updateIncrement
it will also be called very often. But it is not yet optimized:void Oscillator::updateIncrement() {
mPhaseIncrement = mFrequency * 2 * mPI / mSampleRate;
}
2 * mPI * mSampleRate
changes only when the sampling rate changes. So the result of this calculation is better to remember and recount it only inside Oscillator::setSampleRate
. It is worth remembering that transcendent optimization can make the code unreadable and even ugly. In the specific case, we will not have performance problems, because we are writing an elementary monophonic syntax. When we get to polyphony, it will be another matter, and then we will certainly optimize. Now we can rewrite
ProcessDoubleReplacing
in Synthesis.cpp :void Synthesis::ProcessDoubleReplacing(
double** inputs,
double** outputs,
int nFrames)
{
// Mutex is already locked for us.
double *leftOutput = outputs[0];
double *rightOutput = outputs[1];
for (int i = 0; i < nFrames; ++i) {
mMIDIReceiver.advance();
int velocity = mMIDIReceiver.getLastVelocity();
if (velocity > 0) {
mOscillator.setFrequency(mMIDIReceiver.getLastFrequency());
mOscillator.setMuted(false);
} else {
mOscillator.setMuted(true);
}
leftOutput[i] = rightOutput[i] = mOscillator.nextSample() * velocity / 127.0;
}
mMIDIReceiver.Flush(nFrames);
}
In a for loop, the MIDI receiver first updates the values (called
advance
). If a note ( velocity > 0
) sounds , we update the oscillator frequency and let it sound. Otherwise, we drown it (then it nextSample
will return zeros). Further, it all comes down to simply calling
nextSample
to get the value, change the volume ( velocity
this is an integer between 0
and 127
) and write the result to the output buffers. At the end it is called Flush
to remove the beginning of the queue.Test
Run VST or AU. If the AU does not appear on the host, then you may have to change it
PLUG_UNIQUE_ID
in resource.h . If two plugins have the same ID, the host will ignore everything except one. The plugin needs to send some MIDI data to the input. The easiest way is to use the REAPER's virtual keyboard ( View → Virtual MIDI Keyboard menu ). On the track with the plugin on the left there is a round red button. Go to the MIDI configuration by right-clicking on it and choose to receive messages from the virtual keyboard:
In the same menu, turn on Monitor Input . Now that the focus is on the virtual keyboard window, you can play the synthesizer with a regular keyboard. Enter your username or password from the password manager and listen to how it sounds.
If you have a MIDI keyboard, then by connecting it, you can test a stand-alone application. The main thing is to choose the correct MIDI input. If no sound is heard, try deleting ~ / Library / Application Support / Synthesis / settings.ini .
The entire project at this stage can be downloaded from here .
Next time we will add a nice keyboard to the interface :)
Original article:
martin-finke.de/blog/articles/audio-plugins-009-receiving-midi