Impersonation Voice Morphing in AudioKit

[E. M. Fisher]

Hello, 

I am currently working on a voice changing app that will be unique to other voice changers thanks to the help of AudioKit. One thing I have became super fascinated with recently that I am not sure is possible in AudioKit is voice morphing that allows you to impersonate other people. 

Here is a cool example of a voice morphing  program that can take a guys voice and turn it into the voice of a woman including having the inflections and pitch. 

https://www.youtube.com/watch?v=N3pUxkM3x5I - example at 1:20 

Is there anyway to easily morph a natural voice to become the intermediary to the spectrums of the natural voice and the spectrums of another voice in Audiokit?

There is clearly much more I would have to do such as voice training, but this would be a great first step to try to replicate MorphVox. 

Here is academic paper on the subject.

http://www.ijsps.com/uploadfile/2014/1210/20141210044237752.pdf

Thank you,

Eric

[Aurelius Prochazka Ph.D.]

I have seen a few really good pitch shifters that do gender reassignment by basically giving a lilt to the shift at low volumes. You can try to do something like that with AudioKit.  Also, that dude is very good at changing his voice while processing it so that the effect becomes even more convincing.  He reveals his technique in the update video linked to from this one.

Aure

[Paul Batchelor]

You could have some limited success using the pitch shifters in AK. 

Pitch shifters that are optimized for the voice often have some sort of formant frequency preservation to avoid chipmunkification, which AK does not have (yet...)

Cool looking paper. Thanks for the share!

-P

On Sunday, January 15, 2017 at 7:56:49 PM UTC-8, E. M. Fisher wrote:

[E. M. Fisher]

Thank you,
I will try that!

Sincerely,
Eric

[E. M. Fisher]

How hard would it be to add formant preservation to the code of the AK Pitch Shifter? I want to give it a shot.

[Paul Batchelor]

Pretty hard for many reasons. I would probably write a completely new pitch shifter instead of adding to the current ones to be honest. That would be easier. 

[E. M. Fisher]

I figured when I was looking deeply into the audiokit code. At what level do you think it would be best to write the code and by that I mean at the top in swift or lower in sporth? I will give the code to AudioKit if it is any good.

[Paul Batchelor]

Sporth isn't low-level enough for this kind of task. You'll actually need to write a Soundpipe module (this is C code). You *will* need to have a prior background in audio DSP to do this, in addition to programming in C.

If you still feel up to the task, make a github issue on the Soundpipe github repo: https://github.com/paulbatchelor/soundpipe. We can continue the discussion from there.

From there, once you've made a prototype, you can make a Soundpipe PR and I can review the plugin. After it has been implemented, it can then be imported into AudioKit as an AudioKit plugin.

[E. M. Fisher]

At the moment that is a bit too advanced for me. I am taking an audio processing class online which will teach me how to accomplish this including formant preservation. In the future I can probably help with adding that into AudioKit. I found I can accomplish formant preservation in C sound though fairly easily because they have it built in to some of the pitch codes.

Cheers,

Eric

[E. M. Fisher]

Just an update, I tried your idea Aurelius and I got my voice to sound like Isaac Hayes. 

So thank you for your help. :)