September 8, 2013 at 15:06Simple but Efficient Voice Activity Detection Real-Time Algorithm
The following is a translation of
A SIMPLE BUT EFFICIENT REAL-TIME VOICE ACTIVITY DETECTION ALGORITHM
M.H. Moattar and MM Homayonpour
Laboratory for Intelligent Sound and Speech Processing (LISSP), Computer Engineering and Information Technology Dept., Amirkabir University of Technology, Tehran, Iran.
Original link
Voice Activity Detection (VAD) is a very important method in speech and audio processing applications. The effectiveness of most, if not all speech / audio processing methods is highly dependent on the efficiency of the VAD algorithm used. An ideal detector of voice activity should be independent of the scope of the application, the noise level and be the least dependent on the maximum parameters of the application in which it is used. This article proposes a near-perfect VAD algorithm that is both easy to implement and noise resistant. The proposed method uses such short-term characteristics as Spectral Flatness (SF) (spectral flatness, evenness) and Short-term Energy, which makes the method suitable for real-time use. This method has been tested on several recordings with different noise levels and compared with recently proposed methods. The experiments showed satisfactory results at different noise levels.
Voice Activity Detection (VAD), that is, detecting silence in a speech or audio signal, is a very important task for many applications that work with audio or speech, including encoding, recognition, improving speech intelligibility, and indexing audio. For example, the GSM 729 standard [1] uses two VAD modules for encoding with a different number of bits in a sample. Noise immunity VAD is also very important for speech recognition (Automatic Speech Recognition ASR). A good detector will improve the accuracy and speed of any ASR in noisy environments.
According to [2], the necessary characteristics for an ideal voice activity detector are: reliability, stability, accuracy, adaptability, simplicity, the possibility of real-time use, without information about the noise present. Achieving noise immunity is the hardest. In conditions of high SNR (Signal-to-noise ratio), the simplest VAD algorithms work satisfactorily, but under conditions of low SNR all VAD algorithms degrade to a certain extent. At the same time, the VAD algorithm should remain simple in order to meet the real-time applicability requirement. Therefore, simplicity and noise resistance are two essential characteristics of a practical speech activity detector.
Many VAD algorithms have been proposed, the main difference of which is in the characteristics used. Among all the characteristics, Short-term Energy and zero-crossing rate were used more often because of their simplicity. However, they are greatly degraded in the presence of noise. In order to correct this drawback, various stable acoustic characteristics were proposed based on the autocorrelation function [3, 4], spectrum based spectrum [5], power in the narrow-band segment (power in the band-limited region) [1, 6 , 7], MFCC (Mel-frequency Cepstral Coefficients [4] - Cepstral coefficients of the tonal frequency. You can read in the book spbu), delta spectral frequencies (delta line spectral frequencies) [6] and higher-order statistics [8]. Experiments have shown that using these characteristics leads to an increase in VAD noise immunity.
On the other hand, some methods use noise models [11], or use the improved speech spectrum obtained after statistical noise filtering by the Wiener filter [7, 12]. Most of the characteristics assume stationary noise for a certain period, so they are sensitive to changes in the SNR of the processed signal. Some papers suggest noise computation and adaptation to improve VAD stability [13], but these methods have great computational complexity.
Also, there are VADs standards that are used to create new detection methods. Among them are GSM 729 [1], ETSI AMR [14] and AFE [15]. For example, the GSM 729 standard uses the linear spectrum of a frequency pair, full-band energy and low-band energy, zero-crossing rate and applies a classifier using fixed boundaries in a limited space [1].
In this work, the VAD algorithm is proposed, which is both easy to implement and can be used for real-time speech / audio processing, and also provides satisfactory noise immunity. Section 3 details the algorithm of the proposed VAD.
In the proposed method, we use three different characteristics for each frame. The first characteristic is short-term energy (E). Energy is the most commonly used characteristic in the definition of speech / silence. However, it becomes ineffective in noise conditions, especially at low SNRs. Therefore, we use two more characteristics that are calculated from frequencies.
The second characteristic is a measure of spectral flatness (SFM - Spectral Flatness Measure). The measure of spectrum noise is well shown in Voice / Non-Voice Detection and silence detection.
It is considered SFM according to the following formula:
SMF db = 10log 10 (G m / A m )
Where A m and G mthis is respectively the arithmetic mean and geometric mean of the speech spectrum.
In addition to these two characteristics, it was found that the component of the speech frame with the most dominant frequency component can be very useful for distinguishing frames with speech and silence. In this work, this characteristic is denoted by F. It is easily calculated by finding a frequency that corresponds to the maximum value of the spectrum | S (k) |.
In the proposed method for detecting voice activity, all three characteristics are calculated simultaneously for each frame.
Figure 1. Characteristic values on a clear speech signal
Figure 2. Characteristic value on a signal damaged by white noise
Figure 3. Characteristic value on a signal damaged by babble noise
Images 1-3 represent the effectiveness of these three characteristics on a clean and noise-damaged signal.
The proposed algorithm begins by splitting the audio signal into frames. In our implementation, the window function is not used. The first N frames are used to initialize the threshold value. For each incoming frame, three characteristics are calculated. An audio frame is considered speech if the value of more than one characteristic exceeds a threshold value. The full procedure of the proposed method is presented below:
The algorithm has three parameters that must be set first. These parameters were found automatically on a finite set of pure speech signals. Below are the optimal values obtained as a result of experiments.
Energy_PrimThresh = 40
F_PrimThresh (Hz) = 185
SF_PrimThresh = 5
The original article provides the results of experiments conducted under conditions of different noisiness. I would like to know the opinion of the community about this topic and about the translation itself. Does it make sense to continue to lay out translations on this topic? About all translation inaccuracies, errors, write in private messages.
I did not find how to indicate that this is a translation, except to assign a post to the Translations hub.
A SIMPLE BUT EFFICIENT REAL-TIME VOICE ACTIVITY DETECTION ALGORITHM
M.H. Moattar and MM Homayonpour
Laboratory for Intelligent Sound and Speech Processing (LISSP), Computer Engineering and Information Technology Dept., Amirkabir University of Technology, Tehran, Iran.
Original link
SUMMARY
Voice Activity Detection (VAD) is a very important method in speech and audio processing applications. The effectiveness of most, if not all speech / audio processing methods is highly dependent on the efficiency of the VAD algorithm used. An ideal detector of voice activity should be independent of the scope of the application, the noise level and be the least dependent on the maximum parameters of the application in which it is used. This article proposes a near-perfect VAD algorithm that is both easy to implement and noise resistant. The proposed method uses such short-term characteristics as Spectral Flatness (SF) (spectral flatness, evenness) and Short-term Energy, which makes the method suitable for real-time use. This method has been tested on several recordings with different noise levels and compared with recently proposed methods. The experiments showed satisfactory results at different noise levels.
1. INTRODUCTION
Voice Activity Detection (VAD), that is, detecting silence in a speech or audio signal, is a very important task for many applications that work with audio or speech, including encoding, recognition, improving speech intelligibility, and indexing audio. For example, the GSM 729 standard [1] uses two VAD modules for encoding with a different number of bits in a sample. Noise immunity VAD is also very important for speech recognition (Automatic Speech Recognition ASR). A good detector will improve the accuracy and speed of any ASR in noisy environments.
According to [2], the necessary characteristics for an ideal voice activity detector are: reliability, stability, accuracy, adaptability, simplicity, the possibility of real-time use, without information about the noise present. Achieving noise immunity is the hardest. In conditions of high SNR (Signal-to-noise ratio), the simplest VAD algorithms work satisfactorily, but under conditions of low SNR all VAD algorithms degrade to a certain extent. At the same time, the VAD algorithm should remain simple in order to meet the real-time applicability requirement. Therefore, simplicity and noise resistance are two essential characteristics of a practical speech activity detector.
Many VAD algorithms have been proposed, the main difference of which is in the characteristics used. Among all the characteristics, Short-term Energy and zero-crossing rate were used more often because of their simplicity. However, they are greatly degraded in the presence of noise. In order to correct this drawback, various stable acoustic characteristics were proposed based on the autocorrelation function [3, 4], spectrum based spectrum [5], power in the narrow-band segment (power in the band-limited region) [1, 6 , 7], MFCC (Mel-frequency Cepstral Coefficients [4] - Cepstral coefficients of the tonal frequency. You can read in the book spbu), delta spectral frequencies (delta line spectral frequencies) [6] and higher-order statistics [8]. Experiments have shown that using these characteristics leads to an increase in VAD noise immunity.
On the other hand, some methods use noise models [11], or use the improved speech spectrum obtained after statistical noise filtering by the Wiener filter [7, 12]. Most of the characteristics assume stationary noise for a certain period, so they are sensitive to changes in the SNR of the processed signal. Some papers suggest noise computation and adaptation to improve VAD stability [13], but these methods have great computational complexity.
Also, there are VADs standards that are used to create new detection methods. Among them are GSM 729 [1], ETSI AMR [14] and AFE [15]. For example, the GSM 729 standard uses the linear spectrum of a frequency pair, full-band energy and low-band energy, zero-crossing rate and applies a classifier using fixed boundaries in a limited space [1].
In this work, the VAD algorithm is proposed, which is both easy to implement and can be used for real-time speech / audio processing, and also provides satisfactory noise immunity. Section 3 details the algorithm of the proposed VAD.
2. SHORT-TERM FEATURE (short-term characteristics)
In the proposed method, we use three different characteristics for each frame. The first characteristic is short-term energy (E). Energy is the most commonly used characteristic in the definition of speech / silence. However, it becomes ineffective in noise conditions, especially at low SNRs. Therefore, we use two more characteristics that are calculated from frequencies.
The second characteristic is a measure of spectral flatness (SFM - Spectral Flatness Measure). The measure of spectrum noise is well shown in Voice / Non-Voice Detection and silence detection.
It is considered SFM according to the following formula:
SMF db = 10log 10 (G m / A m )
Where A m and G mthis is respectively the arithmetic mean and geometric mean of the speech spectrum.
In addition to these two characteristics, it was found that the component of the speech frame with the most dominant frequency component can be very useful for distinguishing frames with speech and silence. In this work, this characteristic is denoted by F. It is easily calculated by finding a frequency that corresponds to the maximum value of the spectrum | S (k) |.
In the proposed method for detecting voice activity, all three characteristics are calculated simultaneously for each frame.
Figure 1. Characteristic values on a clear speech signal
Figure 2. Characteristic value on a signal damaged by white noise
Figure 3. Characteristic value on a signal damaged by babble noise
Images 1-3 represent the effectiveness of these three characteristics on a clean and noise-damaged signal.
3. SUGGESTED VAD ALGORITHM
The proposed algorithm begins by splitting the audio signal into frames. In our implementation, the window function is not used. The first N frames are used to initialize the threshold value. For each incoming frame, three characteristics are calculated. An audio frame is considered speech if the value of more than one characteristic exceeds a threshold value. The full procedure of the proposed method is presented below:
1 - Устанавливаем Frame_Size = 10ms и вычисляется кол-во фреймов( Num_of_frames ) //перекрытие фреймов не требуется
2 - Устанавливаем по одному пороговому значению для каждой характеристики. // единственное что устанавливается извне
* Порог для Энергии (Energy_PrimTreshhold)
* Порог для F (F_PrimTreshhold)
* Порог для SFM (SF_PrimTreshhold)
3 - For i от 1 до Num_of_frames
3.1 - Вычисляем энергию фрейма (E(i))
3.2 - Применяем FFT для каждого фрейма
3.2.1 - Находим F(i) = arg max (S(k)) - составляющую с преобладающей частотой
3.2.2 - Вычисляем значение спектральной плоскостности как Measure(SFM(i))
3.3 - Предполагаем что некоторые из 30 первых фреймов - это тишина, находим
минимальное значение для Е (Min_E), F (Min_F), SMF (Min_SF)
3.4 - Устанавливаем порог принятия решения для E, F, SFM
* Tresh_E = Energy_PrimTresh * log(Min_E)
* Tresh_F = F_PrimTresh
* Tresh_SF = SF_PrimTresh
3.5 - Устанавливаем Counter = 0
* Если ((E(i) - Min_E) >= Tresh_E) тогда Counter++
* Если ((F(i) - Min_F) >= Tresh_F) тогда Counter++
* Если ((SFM(i) - Min_SF) >= Tresh_SF) тогда Counter++
3.6 - Если Counter > 1 то отмечаем текущий фрейм как речевой, иначе как тишину
3.7 - Если текущий фрейм отмечен как тишина, обновляем значения минимума tythubb
Min_E = ((Silence_Count * Min_E) + E(i)) / (Silence_Count + 1)
3.8 Tresh_E = Energy_PrimTresh * log(Min_E)
4 - Игнорировать тишину идущую менее чем 10 фреймов
5 - Игнорировать речь идущую менее чем 5 фреймов.
The algorithm has three parameters that must be set first. These parameters were found automatically on a finite set of pure speech signals. Below are the optimal values obtained as a result of experiments.
Energy_PrimThresh = 40
F_PrimThresh (Hz) = 185
SF_PrimThresh = 5
The original article provides the results of experiments conducted under conditions of different noisiness. I would like to know the opinion of the community about this topic and about the translation itself. Does it make sense to continue to lay out translations on this topic? About all translation inaccuracies, errors, write in private messages.
I did not find how to indicate that this is a translation, except to assign a post to the Translations hub.
References
[1] A. Benyassine, E. Shlomot, HY Su, D. Massaloux, C. Lamblin and JP Petit, “ITU-T Recommendation G.729 Annex B: a silence compression scheme for use with G.729 optimized for V. 70 digital simultaneous voice and data applications, ”IEEE Communications Magazine 35, pp. 64-73, 1997.
[2] MH Savoji, “A robust algorithm for accurate end pointing of speech,” Speech Communication, pp. 45-60, 1989.
[3] B. Kingsbury, G. Saon, L. Mangu, M. Padmanabhan and R. Sarikaya, “Robust speech recognition in noisy environments: The 2001 IBM SPINE evaluation system,” Proc. ICASSP, 1, pp. 53-56, 2002.
[4] T. Kristjansson, S. Deligne and P. Olsen, “Voicing features for robust speech detection,” Proc. Interspeech, pp. 369-372, 2005.
[5] RE Yantorno, KL Krishnamachari and JM Lovekin, “The spectral autocorrelation peak valley ratio (SAPVR) - A usable speech measure employed as a co-channel detection system,” Proc. IEEE Int. Workshop Intell. Signal Process. 2001.
[6] M. Marzinzik and B. Kollmeier, “Speech pause detection for noise spectrum estimation by tracking power envelope dynamics,” IEEE Trans. Speech Audio Process, 10, pp. 109-118, 2002.
[7] ETSI standard document, ETSI ES 202 050 V 1.1.3., 2003.
[8] K. Li, NS Swamy and MO Ahmad, “An improved voice activity detection using higher order statistics,” IEEE Trans. Speech Audio Process., 13, pp. 965-974, 2005.
[9] WH Shin, “Speech / non-speech classification using multiple features for robust endpoint detection,” ICASSP, 2000.
[10] GD Wuand and CT Lin, “Word boundary detection with mel scale frequency bank in noisy environment,” IEEE Trans. Speechand Audio Processing, 2000.
[11] A. Lee, K. Nakamura, R. Nisimura, H. Saruwatari and K. Shikano, “Noise robust real world spoken dialogue system using GMM based rejection of unintended inputs,” Interspeech, pp. 173-176, 2004.
[12] J. Sohn, NS Kim and W. Sung, “A statistical modelbased voice activity detection,” IEEE Signal Process. Lett., Pp. 1-3, 1999.
[13] B. Lee and M. Hasegawa-Johnson, “Minimum Mean Squared Error A Posteriori Estimation of High Variance Vehicular Noise,” in Proc. Biennial on DSP for In-Vehicle and Mobile Systems, Istanbul, Turkey, June 2007.
[14] ETSI EN 301 708 recommendations, “Voice activity detector for adaptive multi-rate (AMR) speech traffic channels,” 1999.
LINKS
[1] A. Benyassine, E. Shlomot, HY Su, D. Massaloux, C. Lamblin and JP Petit, “ITU-T Recommendation G.729 Annex B: a silence compression scheme for use with G.729 optimized for V. 70 digital simultaneous voice and data applications, ”IEEE Communications Magazine 35, pp. 64-73, 1997.
[2] MH Savoji, “A robust algorithm for accurate end pointing of speech,” Speech Communication, pp. 45-60, 1989.
[3] B. Kingsbury, G. Saon, L. Mangu, M. Padmanabhan and R. Sarikaya, “Robust speech recognition in noisy environments: The 2001 IBM SPINE evaluation system,” Proc. ICASSP, 1, pp. 53-56, 2002.
[4] T. Kristjansson, S. Deligne and P. Olsen, “Voicing features for robust speech detection,” Proc. Interspeech, pp. 369-372, 2005.
[5] RE Yantorno, KL Krishnamachari and JM Lovekin, “The spectral autocorrelation peak valley ratio (SAPVR) - A usable speech measure employed as a co-channel detection system,” Proc. IEEE Int. Workshop Intell. Signal Process. 2001.
[6] M. Marzinzik and B. Kollmeier, “Speech pause detection for noise spectrum estimation by tracking power envelope dynamics,” IEEE Trans. Speech Audio Process, 10, pp. 109-118, 2002.
[7] ETSI standard document, ETSI ES 202 050 V 1.1.3., 2003.
[8] K. Li, NS Swamy and MO Ahmad, “An improved voice activity detection using higher order statistics,” IEEE Trans. Speech Audio Process., 13, pp. 965-974, 2005.
[9] WH Shin, “Speech / non-speech classification using multiple features for robust endpoint detection,” ICASSP, 2000.
[10] GD Wuand and CT Lin, “Word boundary detection with mel scale frequency bank in noisy environment,” IEEE Trans. Speechand Audio Processing, 2000.
[11] A. Lee, K. Nakamura, R. Nisimura, H. Saruwatari and K. Shikano, “Noise robust real world spoken dialogue system using GMM based rejection of unintended inputs,” Interspeech, pp. 173-176, 2004.
[12] J. Sohn, NS Kim and W. Sung, “A statistical modelbased voice activity detection,” IEEE Signal Process. Lett., Pp. 1-3, 1999.
[13] B. Lee and M. Hasegawa-Johnson, “Minimum Mean Squared Error A Posteriori Estimation of High Variance Vehicular Noise,” in Proc. Biennial on DSP for In-Vehicle and Mobile Systems, Istanbul, Turkey, June 2007.
[14] ETSI EN 301 708 recommendations, “Voice activity detector for adaptive multi-rate (AMR) speech traffic channels,” 1999.
