Problems - Getting started

[Anna Fuchs]

Hello,
thank you very much for the nice tool.

I downloaded the jar file and the models (Trained Models (version 1.3)) from the webpage as well as the most current version through github.
Then I tried to test the jar file using an example from the github directory. (./AuToBI-master/test_data/bdc-test.wav and the corresponding .TextGrid file which contains the words)
The jar file runs fine without errors but as a result I only obtain L*+!H for the pitch accents.
Unfortunately I am not familiar with java programming and know I do not know if the results are correct. I am not sure if it is possible to run the tool without any knowledge of java.

I run the program with the following command.
java -jar AuToBI.jar \
-input_file=bdc-test.TextGrid \
-wav_file=bdc-test.wav \
-pitch_accent_detector=./Models/burnc_bdc_games.AuToBI_1.3.10022012/burnc_bdc_games.pitch_accent_detector.model \
-pitch_accent_classifier=./Models/burnc_bdc_games.AuToBI_1.3.10022012/burnc_bdc_games.pitch_accent_classifier.model \
-intonational_phrase_boundary_detector=./Models/burnc_bdc_games.AuToBI_1.3.10022012/burnc_bdc_games.intonational_phrase_detector.model \
-intermediate_phrase_boundary_detector=./Models/burnc_bdc_games.AuToBI_1.3.10022012/burnc_bdc_games.intermediate_phrase_detector.model \
-phrase_accent_classifier = ./Models/burnc_bdc_games.AuToBI_1.3.10022012/burnc_bdc_games.phrase_accent_classifier.model \
-boundary_tone_classifier = ./Models/burnc_bdc_games.AuToBI_1.3.10022012/burnc_bdc_games.phrase_accent_boundary_tone_classifier.model \
-out_file=./OUTPUT_1.1

Attaching files is not possible so here are a few lines from the OUTPUT_1.1:

name = "pitch_accent_hypothesis"
xmin = 0
xmax = 6.74775
intervals: size = 10
intervals [1]:
xmin = 4.36744
xmax = 4.51463
text = "L*+!H"
intervals [2]:
xmin = 4.51463
xmax = 4.71869
text = "DEACCENTED"
intervals [3]:
xmin = 4.71869
xmax = 4.90456
text = "DEACCENTED"
intervals [4]:
xmin = 4.90456
xmax = 5.14825
text = "L*+!H"
intervals [5]:
xmin = 5.14825
xmax = 5.60306
text = "L*+!H"
intervals [6]:
xmin = 5.60306
xmax = 5.67506
text = "DEACCENTED"
intervals [7]:
xmin = 5.67506
xmax = 5.77944
text = "DEACCENTED"
intervals [8]:
xmin = 5.77944
xmax = 5.94325

I am looking forward to you response.
Anna

[Andrew Rosenberg]

Hi Anna,

Thanks for getting in touch.

This is an issue with a change that was made in version 1.3.  The pitch accent classification component was changed to optimize Average Recall rather than accuracy.  Since the pitch accent distribution in american English is very skewed, optimizing accuracy leads to over prediction of H*.  The change now over predicts L*+!H. The least frequent type.  While in within and cross corpus evaluations this makes the average recall numbers look better, in practice, it leads to files that look similar to what you describe below.

The performance of pitch accent classification is still quite low, while I believe autobi represents the state of the art on this task, this example demonstrates how much further we need to go.

The pitch accent classifier is a component that will definitely be updated in an upcoming version.

Again, thanks for your interest and feedback
Andrew

sent from a mobile device.