Adding Values to Klatt Grids using Parselmouth

[William Hobbs]

Hello everyone! Is it possible to add values to an existing Klatt Grid through Parselmouth? I'm trying to use the call function of Parselmouth in order to add points to a Pitch Tier and 2 formant tiers to a KlattGrid that I already wrote. I'm trying to adapt the code I have seen from the Parselmouth documentation and other websites I've found, but so far I've been unsuccessful. Any pointers would be beyond gratefully appreciated! William

[yannick...@gmail.com]

Hello William!

There is currently no Python API for KlattGrid functionality, but through use of `parselmouth.praat.call`, you should be able to do the exact things you can do in Praat with Parselmouth in Python.

Have a look at this example in the documentation: it's dealing with different functionality, but the use of "commands" is the same: https://parselmouth.readthedocs.io/en/stable/examples/pitch_manipulation.html

If after reading this, you get stuck or you don't know where to start, please do follow up and let me know!

Kind regards

Yannick Jadoul

[William Hobbs]

Dear Yannick, thank you so much for your response-- much appreciated! To tell you the truth, I am a little stuck. Here's what I am trying to do:

I am working on developing a synthetic voice that can learn to sing like an opera singer. In some ways, this is way easier than developing a Text-to-Speech model: I don't have to worry about pitch contour (already prescribed by the composer) or relative durations (also already predetermined). However, I do want to interface the synthesized voice with a Reinforcement Learning Model that will try out different combinations of F1, F2, BW1 and BW2 to find the combination that best approximates the ground truth human recording for each individual vowel. What I would need to do is have Parselmouth modify the four values above for each vowel in my synthetic audio sample in order to fine the best approximation. I am not able to find a way to use praat.call function to modify the formant values. I did use the code from the example from your documentation which does work very well, but that's not exactly what I'm trying to do. Let me know what other suggestions you have! Thank you SO MUCH for your helpful response!!! All the best William

[yannick...@gmail.com]

Dear William

I have never really worked with Praat's KlattGrid functionality. I'm looking at Praat's documentation and it seems fairly complex, but manageable: https://www.fon.hum.uva.nl/praat/manual/KlattGrid.html

The things you're interested in changing seem to be in https://www.fon.hum.uva.nl/praat/manual/FormantGrid.html? E.g., clicking "Add formant point..." in the GUI, https://www.fon.hum.uva.nl/praat/manual/FormantGrid__Add_formant_point___.html, could be translated in Parselmouth as: `parselmouth.praat.call(formant_grid_obj, "Add formant point", 2, 0.3, 2200)`.

But of course, this is only a very minor step in the whole process. I would suggest you read a bit (maybe there's some tutorial?) on how KlattGrid actually works in Praat, and then come back with a question that more precisely spells out where in the process of going from Praat to Python you get stuck. Could you please also show at that point some relevant steps of what you've already done?

I'm afraid I do not have enough time to figure out how Praat deals with your situation, but once you have a reasonably clear procedure in mind of which steps from Praat's GUI you want to automate, I can help again.

Kind regards

Yannick

[William Hobbs]

Hi Yannick! Thank you for responding so quickly and so helpfully. I've made some progress, and hopefully I can explain what I'm working on just a bit more clearly so that you might be able to help me get over the final hurdle here! 

I found Peter Zukerman's page on Adjusting Pitch Contour with Parselmouth-Praat. His methodology for adjusting pitch is to a) open a Manipulation Object, which would be the original synthesized recording; b) Create a Pitch Tier with the values you want to modify for whatever duration; c) synthesize the two using "Replace Pitch Tier" and "Get Resynthesis (Overlap-Add)". All of these functions are callable through Parselmouth. I was able to modify this procedure to obtain a kind of vibrato in the Sound version of the KlattGrid I have already written. Thank you Peter Zukerman!!!!

Now, what I'd like to do is develop a similar methodology that would allow me to choose formant and bandwidth values for sections of my pre-existing KlattGrid, synthesize those formant and bandwidth values into my Manipulation object, and play back the result. I'd use this as the basis for a Reinforcement Learning model that would measure error and adjust the formant and bandwidth values to get closer to a ground-truth recording made by a human.

The code you suggested works perfectly. I can create a Formant Grid and add Formant and Bandwidth values to my heart's content. Here's what it looks like:

manipulation_rig = call(rig_snd, "To Manipulation", 0.0113, 20, 750) 

NOTE: #"rig_snd' is the Parselmouth object I created from my original KlattGrid, excerpted from Verdi's Rigoletto.

formant_tier_rig_a = call("Create FormantGrid", "name", 0, 0.731, 2, 350, 300, 60, 50) 

NOTE: #create a pitchTier between 0 and 0.8 seconds to focus on the first vowel of the sung excerpt (0.02-0.731 in the KlattGrid-- the other values are just dummy values to fill in the boxes in the Praat menu).

call(formant_tier_rig_a, "Add formant point", 1, 0.02, 300) #creating an artificial first formant value 

call(formant_tier_rig_a, "Add formant point", 2, 0.02, 600) #creating an artificial second formant value

All of the above worked thanks to your very helpful suggestions. 

HOWEVER I would now want to synthesize these new formant values with the Parselmouth object which I've called "rig_snd" above. Praat does not seem to offer a way to interface a Formant Grid with a Manipulation Object the way it can synthesize a Pitch Tier with a Manipulation Object. Do you know how I might accomplish this? 

OH AND BY THE WAY: speaking of how I might measure error, I notice that calling "snd.values" in Parselmouth gives two arrays. Can you tell me what the values in these arrays represents? I'm thinking to use one of them as an error metric but I don't really understand what they are... Let me know!

Thanks again for your time and extremely helpful suggestions! I know you're busy and deeply appreciate any advice you can offer! All the best William

[William Hobbs]

QUICK UPDATE! I have looked at this more carefully within Praat, and it seems that one can only combine a Formant Grid with a KlattGrid or with a Sound Object. There does not seem to be a way to change formant values within Manipulation. When combining Formant Grids with Sound Objects, the only options are Filter (result is very quiet) or Filer (no scale) (result sounds dreadful). Combining Formant Grids with KlattGrids in Praat is more useful, so I tried doing this combination with Parselmouth as follows:

klatt = "Rigoletto.KlattGrid"      #It's 25.579 seconds long and has 7 formants. The new formant tier has to be the same length and same number of formants

formant_tier_rig= call("Create FormantGrid", "name", 0, 25.579, 7, 350, 300, 60, 50) 

call(formant_tier_rig, "Add formant point", 1, 0.02, 300)

call(formant_tier_rig, "Add formant point", 2, 0.02, 600)

rig_carcrash = call([formant_tier_rig, klatt], "Replace oral formant grid")

The error message I am getting is: "Command "Replace oral formant grid" not available for given objects."

Is this because Parselmouth does not support this kind of operation, or am I misunderstanding the code you gave me previously? The procedure does work in Praat-- combine the formant tier with the KlattGrid, then replace the oral formant grid.

Again, I know you are busy and I'm sorry to be annoying. But any help will be incredibly appreciated!!!! Thanks William

[yannick...@gmail.com]

Hi William

No problem, happy to help when I find the time! Sorry it took me a while, this time.

The second message and what you write there seems really important, indeed. You need to have a workflow that works in Praat itself, and matches the possibilities there. As you found, Manipulation doesn't seem to be the right way to go, if it doesn't allow you to change the formants.

The error you seem to be getting might be related to "Rigoletto.KlattGrid" being just a string, no? You do need to `parselmouth.read(...)` the saved Praat file in order to read get an actual object. However, I'm not sure whether that's the full issue, because I would expect a different error. Can you print `klatt.class_name` to see what the actual Praat object type is?

I agree that normally your line `call([formant_tier_rig, klatt], "Replace oral formant grid")` should work, if all objects are as expected.

I also just saw that KlattGrid itself has a bunch of modification actions, like "Add oral formant frequency point...". I don't know if that is helpful, but these commands might allow you to change the formants rather than replacing the whole FormantGrid. However, again, your approach should not give an error.

If none of these suggestions work, is there a chance to send me a private email, with e.g. the KlattGrid you have and with a minimal example of the script you're trying to run, which is giving you this error?

Kind regards

Yannick