Need help on GOP

[Amol Bole]

Dear All,

We are working on GOP and we got the following output.

35363738 [ 1 -0.04500699 ] [ 4 -0.736316 ] [ 22 -0.6936717 ] [ 33 -0.5055847 ] [ 14 -0.5047328 ] [ 1 -4.575368 ] [ 5 -0.0970664 ] [ 11 -0.8301041 ] [ 19 -1.232405 ] [ 32 -2.149931 ] [ 5 0 ] [ 25 -0.8108375 ] [ 1 -1.426894 ] [ 4 -0.9305243 ] [ 16 -1.102926 ] [ 33 -0.6984282 ] [ 14 -0.7835178 ] [ 1 0 ]

In the output  [ 4 -0.736316 ] pure-phone is 4 and it's GOP -0.736316. Here, how we can understand correct phone or wrong phone? Also how we can convert this GOP value to accuracy? For reference I am attaching here GOP script. Correct me if I am wrong.

[Aman Deep]

How did you compute the .txt file?? i.e gop.txt, i did not got the .txt file after i ran run.sh file??

[李俊廷（ヤンヤン）]

Hi, Amol Bole

For your first question: how do we understand the correct phone or the wrong phone?

Observing the transcribe you provided, you remove the stage of compile-train-graphs-without-lexicon, instead, you use align.sh here so the compile-train-graphs will be used. It will use the lexicon to recognize the best path of phoneme sequence. However, the speechocean762 corpus provides a text-phone regulation mapping for each word in each utterance in order to align with the phoneme level score targets.

To my best knowledge, we can not know if the pronunciation of a specific phone is correct or wrong via the GOP score. Instead, it is a problem discussed in mispronunciation diagnosis and detection. To name a few: https://ieeexplore.ieee.org/abstract/document/10097226/

Amol Bole 在 2022年6月6日 星期一晚上7:37:01 [UTC+8] 的信中寫道：

[Karel Veselý]

Hi,
it seems that the format is [ <phone_index> <log_posterior> ].
If the scores were well calibrated, the default threshold p=0.5 corresponds to log-value -0.693 (higher is correct, lower is incorrect).

In practice, you may need to further calibrate these scores by passing through a logistic regression trained on labelled in-domain data.
Or, at least you can plot histogram of scores of correct/incorrect phones and set the threshold according to it.

Best,
Karel

Dne sobota 11. května 2024 v 4:01:25 UTC+2 uživatel 李俊廷（ヤンヤン） napsal:

[Anantha Krishnan]

ark,t: can be used to get the result in text

This code segment does the job.

# Get ids for pure phones instead of canonical ones

local/remove_phone_markers.pl $lang_dir/phones.txt \
data/phones-pure.txt data/phone-to-pure-phone.int

# Get alignment for frame - transition-ids mapping

steps/nnet2/align.sh --nj 1 ./data $lang_dir $mdl_path exp/nnet_ali

# Get frame - canonical_phone mapping

$cmd JOB=1:$nj exp/ali_test/log/ali_to_phones.JOB.log \
      ali-to-phones --per-frame=true $mdl_path/final.mdl \
        "ark,t:gunzip -c exp/nnet_ali/ali.JOB.gz|" \
        "ark,t:|gzip -c >exp/nnet_ali/ali-phone.JOB.gz"

# Get the posteriors (a matrix - no of rows is frames, and column values are senone emission probabilities) 

nnet-am-compute $mdl_path/final.mdl "ark,s,cs:copy-feats scp:data/feats.scp ark:- |" ark:data/posteriors.ark

# GoP result of compute-gop in gop.1.txt 

$cmd JOB=1:$nj exp/gop_test/log/compute_gop.JOB.log \
  compute-gop --phone-map=data/phone-to-pure-phone.int $mdl_path/final.mdl \
      "ark,t:gunzip -c exp/nnet_ali/ali-phone.JOB.gz|" \
      "ark:data/posteriors.ark" \
      "ark,t:exp/gop/gop.JOB.txt" "ark,t:exp/gop/phonefeat.JOB.txt"

Effectively, compute-gop computes the difference between the forced alignment phase and free recognition phase (you see 'max' in the denominator of gop, and in free recognition phase model would anyways choose the max or best possible state sequence that it believes). So GoP value closer to zero, better is the pronunciation.