Any special process in Praat but not in Parselmouth for Formants extraction?

[Elsie Fan]

Hi!

I am trying to extract formants from audio files in Python and find Parselmouth very useful. I found slightly different result between Praat and Parselmouth, especially during the silent time (see attached). Is there any special process done to prevent formant extraction during the silent period in Praat?

Thanks!

Elsie

[yannick...@gmail.com]

Hi Elsie

Well spotted! Thanks for letting me know!

I had a look in the code, and the only difference is that Praat, when drawing the formants, does not draw frames with an "intensity" lower than a certain threshold. In the "View & Edit" window, this threshold is set by the "Dynamic range (dB)" setting. Every Praat formant frame has an intensity calculated (check with "Inspect" on a formant object in the main "Object window"), and when drawing, Praat gets the maximum intensity over the time window that's being drawn, converts the dynamic range decibel value into a factor, and takes this as a cutoff for now drawing the formants. I would presume this is done to not draw formants during silence.

Crucially, I do not think this should affect in any way the formant values calculated. It should just affect the frames that get plotted. (If you find out this is not the case, and the values differ, do let me know! This would be a bug and important to investigate and fix in Parselmouth!)

Annoyingly, this "intensity" of a Formant object's frame is not too easily accessed, as Praat doesn't have any buttons in the GUI to do so. I can see two ways to go about this, if you want to do the same as Praat in Python:

1) Take a detour and go through a "Table". At a quick glance this seems to make sense to extract the F1 values for a given THRESHOLD:

```
formant_table = parselmouth.praat.call(formant, "Down to Table", True, True, 6, True, 15, True, 3, True)

frame_intensities = np.array([float(parselmouth.praat.call(formant_table, "Get value", i + 1, "intensity")) for i in range(formant.nx)])
frame_threshold = np.max(frame_intensities) / 10**(DYNAMIC_RANGE / 10)
selected_frames = frame_intensities >= frame_threshold

formant_values = np.array([formant.get_value_at_time(1, t) for t in formant.ts()])
formant_values[~selected_frames] = np.nan

plt.figure()
plt.plot(formant.xs(), formant_values, '.')
plt.ylim(0, 5500)
plt.show()
```

2) Calculate your own Intensity (or even use "To TextGrid (silences)", https://www.fon.hum.uva.nl/praat/manual/Sound__To_TextGrid__silences____.html), and decide based on this. It won't exactly match Praat unless you manage to perfectly match the window length and step size of the intensity and formant analysis, but it might be easier with Parselmouth (not going through this table thing), you have more control, and it's the same in spirit as Praat (i.e., not using the formant values if an analysis window is too silent).

I completely agree it's annoying you need to have these workarounds, but in my/Parselmouth's defense: Praat only does this when plotting (i.e., you can still query the formant at these points) and I don't see an easier way of getting to these intensity values manually withing Praat GUI/scripting either, if you want to extract them and plot externally (so at least Parselmouth is not making it harder).

If anything's unclear, or you have follow-up questions or issues, do not hesitate to let me know!

Kind regards

Yannick

[Elsie Fan]

Hi Yannick,

Thanks so much for looking into this and sharing the code! It is very helpful. The reason why I asked is that I want to use this intensity as a guidance to not include the formant values during silence in my model, as they are not real "formants". 

I compared the F1 from Praat and Parselmouth - see attached. The Praat F1 formant value seems to lie between Parselmouth's F1 and F2. I guess when the vowel is very clearly pronounced, the values are the same. Also in the plot, where the dots formed a line are the silence, and I noticed the different line values for Parselmouth F1 and F2. Does it mean that Parselmouth sets different frequency range for formants? I also attached the audio used in the plot in case you want to check it out.

Thanks!

Elsie

[yannick...@gmail.com]

Hi Elsie

Hmmm, that's not so good, indeed.

Parselmouth should be using the exact same underlying code of Praat (being one of the main reasons I wanted to create it). So I'm rather eager to figure out what's going wrong!

Could you tell me how you generated the output.png plot you attached? How did you get the Praat values into the plot?

Next, what version of Praat and Parselmouth did you use?
And probably most importantly: what parameter settings did you use in both cases?

I'll have a look myself, and try some things out as well. Thanks for the audio!

Kind regards

Yannick

[yannick...@gmail.com]

Hi Elsie

I played around a bit myself, and made some plots. In Praat, I used "Analyse Spectrum > To Formant (burg)" with Praat's parameters, then "Tabulate > Down to Table...", and then saved that to CSV.

Next, I plotted both these values for F1 and F2, and well as Parselmouth's analysis:

import numpy as np
import pandas as pd
import parselmouth

import matplotlib.pyplot as plt

sound = parselmouth.Sound("N_WLW-AM_2023-02-01-04-44-14.mp3")
formant = sound.to_formant_burg()

praat_df = pd.read_csv("N_WLW-AM_2023-02-01-04-44-14.csv")


plt.figure()
plt.plot(formant.xs(), [formant.get_value_at_time(1, x) for x in formant.xs()], '.')
plt.plot(formant.xs(), [formant.get_value_at_time(2, x) for x in formant.xs()], '.')
plt.xlim(0, sound.duration)
plt.ylim(0, 5500)
plt.title("Parselmouth")

plt.figure()
plt.plot(praat_df['time(s)'], praat_df['F1(Hz)'], '.')
plt.plot(praat_df['time(s)'], praat_df['F2(Hz)'], '.')
plt.xlim(0, sound.duration)
plt.ylim(0, 5500)
plt.title("Praat")

plt.figure()
plt.plot(formant.xs(), np.abs(praat_df['F1(Hz)'] - [formant.get_value_at_time(1, x) for x in formant.xs()]), '.')
plt.plot(formant.xs(), np.abs(praat_df['F2(Hz)'] - [formant.get_value_at_time(2, x) for x in formant.xs()]), '.')
plt.xlim(0, sound.duration)
plt.title("Difference")

plt.show()

And I can't really see any difference (apart from the rounding errors, cause "Down to Table" rounds to a specific number of decimals; 3 decimals, in this case, matching with a maximum absolute error of 5e-4).

I'm using a rather recent version of Praat (6.2.23) on Linux, and Parselmouth 0.4.1.

Am I missing something? Or could you perhaps try the same and see if we're still getting different results.
Do still let me know what you did to produce the graph above, and I'll also investigate that!

Kind regards

Yannick

[Elsie Fan]

Hi Yannick,

Thanks for looking into this! I agree that what you got from Praat and Parselmouth are not substantially different. I think the difference comes from how I queried formants in Praat... Instead of doing what you did, I was using object>View&Edit, then select entire waveform, select Formants>Formant listing. The formants that I got this way was different from what I got in Parselmouth, but when I'm using your way, the formants are the same. The Praat that I use is version 6.3.09 for Mac, Parselmouth 0.4.3.

Thanks again for your reply! I learn more about Praat and Parselmouth :)

Best,

Elsie

[yannick...@gmail.com]

Hi Elsie

Strange that you get such big differences, still. Underlying, Praat should use the same code in the "View & Edit" window as in the "To Formant (burg)" action in the main window.

I would expect that somewhere in these two ways in Praat, you're using different parameters values in both cases, but I can't be sure what exactly. At any rate, it must be something like that, if Praat itself is giving two different formant tracks, within the same version. As long as "To Formant (burg)" gives the same result as Parselmouth, I'd say there's no bug in Parselmouth though, luckily :-)

If you want, I'm happy to help further to look at where the differences lie, but in that case you'd need to check the parameters of the formant tracking in the View & Edit window and report those or take a screenshot.

Kind regards

Yannick