Binaural Localization

[Alethia Tiell]

The site is secure.
The ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

To a first-order approximation, binaural localization cues are ambiguous: many source locations give rise to nearly the same interaural differences. For sources more than a meter away, binaural localization cues are approximately equal for any source on a cone centered on the interaural axis (i.e., the well-known "cone of confusion"). The current paper analyzes simple geometric approximations of a head to gain insight into localization performance for nearby sources. If the head is treated as a rigid, perfect sphere, interaural intensity differences (IIDs) can be broken down into two main components. One component depends on the head shadow and is constant along the cone of confusion (and covaries with the interaural time difference, or ITD). The other component depends only on the relative path lengths from the source to the two ears and is roughly constant for a sphere centered on the interaural axis. This second factor is large enough to be perceptible only when sources are within one or two meters of the listener. Results are not dramatically different if one assumes that the ears are separated by 160 deg along the surface of the sphere (rather than diametrically opposite one another). Thus for nearby sources, binaural information should allow listeners to locate sources within a volume around a circle centered on the interaural axis on a "torus of confusion." The volume of the torus of confusion increases as the source approaches the median plane, degenerating to a volume around the median plane in the limit.

Binaural sound source localization is an important and widely used perceptually based method and it has been applied to machine learning studies by many researchers based on head-related transfer function (HRTF). Because the HRTF is closely related to human physiological structure, the HRTFs vary between individuals. Related machine learning studies to date tend to focus on binaural localization in reverberant or noisy environments, or in conditions with multiple simultaneously active sound sources. In contrast, mismatched HRTF condition, in which the HRTFs used to generate the training and test sets are different, is rarely studied. This mismatch leads to a degradation of localization performance. A basic solution to this problem is to introduce more data to improve generalization performance, which requires a lot. However, simply increasing the data volume will result in data-inefficiency. In this paper, we propose a data-efficient method based on deep neural network (DNN) and clustering to improve binaural localization performance in the mismatched HRTF condition. Firstly, we analyze the relationship between binaural cues and the sound source localization with a classification DNN. Different HRTFs are used to generate training and test sets, respectively. On this basis, we study the localization performance of DNN model trained by each training set on different test sets. The result shows that the localization performance of the same model on different test sets is different, while the localization performance of different models on the same test set may be similar. The result also shows a clustering trend. Secondly, different HRTFs are divided into several clusters. Finally, the corresponding HRTFs of each cluster center are selected to generate a new training set and to train a more generalized DNN model. The experimental results show that the proposed method achieves better generalization performance than the baseline methods in the mismatched HRTF condition and has almost equal performance to the DNN trained with a large number of HRTFs, which means the proposed method is data-efficient.

Sound source localization is to estimate the direction of the sound source and is an important and widely used technique in many fields such as speech enhancement, video conferencing, and human-robot interaction [1]. Sound source localization algorithms have been widely researched so far, and they can be categorized into two classes. The first one is based on microphone array signal processing, which contains three kinds of algorithms: the algorithms based on the time difference of arrival (TDOA)[2], the algorithms based on beamforming [3], and the algorithms based on high-resolution spectral method [4, 5]. The second one is the binaural localization algorithms based on head-related transfer function (HRTF). Each algorithm has its own advantages and disadvantages.

In [9], Raspaud et al. introduce an individual parametric model for each HRTF based on the simple geometric consideration. The ITD and ILD are modeled as the product of a function of frequency and a function of azimuth and then are jointly estimated and compared with templates for localization. Besides, the individual parametric models of each HRTF are averaged, which may improve the generality in the mismatched HRTF condition, but the simple average parametric model may not accurately learn the complex relationship between the binaural cues and the sound locations. Based on [9], Parisi et al. [10] propose cepstrum prefiltering for robustness in the reverberant environment. Pang et al. [1] put forward with reverberation weighting and a more generalized parametric model to further improve the localization performance in the reverberant and noisy environments. A full-sphere binaural localization method is proposed in [11], which applies the Interaural Phase Difference (IPD) for lateral localization and spectral cues for polar angle localization. Although the HRTFs for the training and test sets are captured in different rooms, the models of the dummy head are the same.

Besides the methods based on ITD and ILD templates, some other ones are based on HRTF templates. The key idea of those methods is to identify the HRTF pair corresponding to a certain sound direction, to operate with the left and right channels of the binaural sound respectively, and to achieve the maximal correlation between the results of the left and right channels. A matched filtering approach is proposed in [12]. For a certain sound direction, it exchanges the left channel and right channel of the corresponding HRTF pair, and then respectively filters the left and right channel of the binaural sound.The correlation between the result of the left and right channels shows that the HRTF pair with the maximal correlation corresponds to the direction of the sound. However, the inversion of the HRTF may be unstable. In [13], the source cancelation algorithm is proposed to be an extension of the matched filtering approach without inversion. It divides the left channel and right channel of the HRTF pairs to obtain the templates, and then the division result of the left channel and right channel of the sound in the frequency domain is calculated and matched with those templates. In [14, 15], a cross channel method is proposed, it convolutes the binaural sounds with the HRIR pair corresponding to a certain direction crosswise. Specifically, it convolutes the right channel of sound with the left channel of the HRIR pair and the left channel of sound with the right channel of the HRIR pair. Then the calculated correlation between the result of the two channels indicates the HRIR pair with the maximal correlation corresponds to the sound direction. In [16], a two-step method is proposed to estimate a coarse direction by ITD and the final result by the cross channel method. It improves the accuracy in a noisy environment and decreases the complexity. These HRTF templates based methods also perform good but only work in the matched HRTF condition.

Besides the template-matching-based methods, some other methods are based on statistical models. The key idea of those methods is mapping the binaural features to the posterior probability of the sound source in each direction by the statistical models. In [17], Gaussian Mixture Model (GMM) is used to estimate the multiple source localization in the reverberant and noisy environments by ITD and ILD cues. The HRTF is assumed to be known, which means it works in the matched HRTF condition. By combining DNN and head movements, a multiple source localization method robust against the noisy and reverberant environment is proposed in [18], which is proved to generalize well on the test set generated by another HRTF. However, this method does not consider the performance in the mismatched HRTF condition. In [19], a convolutional neural network (CNN) with multitask learning-based method is proposed to localize the azimuth and elevation simultaneously. It achieves better performance than the method in [18]. While it works in the matched HRTF condition. In [20], a CNN-based sound localization method is proposed and proved to be robust to inter-subject and measurement variability, but this study only focuses on elevation localization. In [21], an end-to-end binaural sound localization approach is proposed, which estimates the azimuth directly from the waveform by CNN. This approach is robust to the reverberate condition; however, the performance in the HRTF-mismatched condition is not studied.

In this paper, we focus on the binaural localization in the mismatched HRTF condition rather than the reverberant and noisy conditions. Although in [9] and [1] a parametric model is proposed and the parameters of different HRTFs are considered to improve the generalization performance, the model may be relatively simple and may not be able to accurately analyze the localization mechanism. Due to the powerful modeling capability, DNN is effective in many areas. In [18], the DNN is introduced and shows significant performance. However, this work only focuses on the localization performance in noisy and reverberate environments rather than mismatched HRTF condition, which shows that the DNN trained by one HRTF generalize well on the test set generated by another HRTF. While we think this result may require further study, here, we consider the binaural localization problem as a classification problem, and we use DNN to map the binaural cues to the sound localization. Firstly, we use DNN to learn the relationship between binaural cues and the localization of the sound source and then compare the localization performance in the matched and mismatched HRTF conditions. The result shows that the localization performance in the matched HRTF condition is good, but the performance varies with the HRTF in the mismatched HRTF condition and the result shows a clustering trend. To improve the generalization performance, a basic idea is to introduce more HRTFs in training sets; however, this may result in data-inefficiency. Secondly, on this basis, clustering analysis is applied to the localization similarity between each HRTF. Different HRTFs are divided into several clusters. The result shows that the HRTF corresponding to each cluster center is a reasonable approximation of other HRTFs in the same cluster. Finally, the HRTFs corresponding to each cluster center are selected to generate a new training set and to train a more generalized DNN model. Compared with the baseline methods in [1, 2, 9, 18] in the mismatched HRTF condition, our method achieves better performance. Compared with the DNN trained by all HRTFs, our method achieves similar performance with low data computation, which means it is data-efficient.

c80f0f1006