Segmentation fault (core dumped) on multilevel forced alignment
349 views
Skip to first unread message
QA Collective
Feb 14, 2017, 8:45:40 PM2/14/17
to aeneas-forced-alignment
Hello Alberto & All,
First of all, congratulations on a really great piece of software. Even though I haven't quite got it working yet ... I can already see that it is a very professionally run project with attention to detail and excellent documentation and good old fashioned quality design! I'm anticipating that aeneas may be able to do a lot of useful work in my pursuit of training DNNs for speech recognition. Unfortunately, I've been getting a seg fault while running Aeneas on multilevel mode. I'll try to be detailed without going too far...
What I'm doing: trying to achieve word level alignment from a TV talk show which provides transcripts which are well written but do not include noises and misspeak from the speakers. I'm providing input transcript in MPLAIN format, requesting CSV or SMIL output and providing the '--presets-word' option.
What happens: Aeneas appears to work as expected (from what I can see from the logs displayed with -v) ... it extracts parts of the .WAV and performs an alignment and continues doing this perhaps 30 times before I receive a 'Segmentation fault (core dumped)' error and am returned to the linux prompt. See the attached text file for a sample of the log leading up to and including the seg fault.
I note that there are zeros in some key parts of the log that seem to equate to a zero length audio segment:
[DEBU] Synthesizer: Selecting TTS engine... done
[DEBU] ExecuteTask: Setting synthesizer... done
[DEBU] ExecuteTask: Text level 3, fragment 0
[DEBU] ExecuteTask: Len: 2
[DEBU] ExecuteTask: Level == 1 or more than one text fragment => compute tree
[DEBU] ExecuteTask: Setting begin: 0.000
[DEBU] ExecuteTask: Setting end: 0.000
[DEBU] AudioFileMFCC: Setting head middle tail...
[DEBU] AudioFileMFCC: Before: 0 0 721564 721564
[DEBU] AudioFileMFCC: After: 0 0 0 721564
[DEBU] AudioFileMFCC: Setting head middle tail... done
...
[DEBU] DTWStripe: n m delta: 0 133 4000
[DEBU] DTWStripe: Limiting delta to m
Segmentation fault (core dumped)
The command I'm running is:
python -m aeneas.tools.execute_task qanda_2012_ep99_climate.wav qanda_2012_ep99_climate.mp4.txt "task_language=eng|is_text_type=mplain|os_task_file_format=srt" qanda_2012_ep99_climate.srt --presets-word -v
I have ZIPped up the WAV file and transcript I'm using along with two log files from Aeneas - one with and one without the -r="c_extensions=False" parameter. If this helps diagnose my issue, you can download these files from the link here:
https://drive.google.com/open?id=0B0hkqO7j3QfbVG1jbXhBSkVHXzA
What I have tried: The one thing I have tried that works is running Aeneas on PLAIN formatted word level input (one word per line) and not passing the '--presets-word' parameter. Unfortunately the results returned by this have words being badly out of sync after about 60 seconds. This is a complete list of the things I've tried to get multilevel alignment working before writing this email ... with no success thus far.
First of all, congratulations on a really great piece of software. Even though I haven't quite got it working yet ... I can already see that it is a very professionally run project with attention to detail and excellent documentation and good old fashioned quality design! I'm anticipating that aeneas may be able to do a lot of useful work in my pursuit of training DNNs for speech recognition. Unfortunately, I've been getting a seg fault while running Aeneas on multilevel mode. I'll try to be detailed without going too far...
What I'm doing: trying to achieve word level alignment from a TV talk show which provides transcripts which are well written but do not include noises and misspeak from the speakers. I'm providing input transcript in MPLAIN format, requesting CSV or SMIL output and providing the '--presets-word' option.
What happens: Aeneas appears to work as expected (from what I can see from the logs displayed with -v) ... it extracts parts of the .WAV and performs an alignment and continues doing this perhaps 30 times before I receive a 'Segmentation fault (core dumped)' error and am returned to the linux prompt. See the attached text file for a sample of the log leading up to and including the seg fault.
I note that there are zeros in some key parts of the log that seem to equate to a zero length audio segment:
[DEBU] Synthesizer: Selecting TTS engine... done
[DEBU] ExecuteTask: Setting synthesizer... done
[DEBU] ExecuteTask: Text level 3, fragment 0
[DEBU] ExecuteTask: Len: 2
[DEBU] ExecuteTask: Level == 1 or more than one text fragment => compute tree
[DEBU] ExecuteTask: Setting begin: 0.000
[DEBU] ExecuteTask: Setting end: 0.000
[DEBU] AudioFileMFCC: Setting head middle tail...
[DEBU] AudioFileMFCC: Before: 0 0 721564 721564
[DEBU] AudioFileMFCC: After: 0 0 0 721564
[DEBU] AudioFileMFCC: Setting head middle tail... done
...
[DEBU] DTWStripe: n m delta: 0 133 4000
[DEBU] DTWStripe: Limiting delta to m
Segmentation fault (core dumped)
The command I'm running is:
python -m aeneas.tools.execute_task qanda_2012_ep99_climate.wav qanda_2012_ep99_climate.mp4.txt "task_language=eng|is_text_type=mplain|os_task_file_format=srt" qanda_2012_ep99_climate.srt --presets-word -v
I have ZIPped up the WAV file and transcript I'm using along with two log files from Aeneas - one with and one without the -r="c_extensions=False" parameter. If this helps diagnose my issue, you can download these files from the link here:
https://drive.google.com/open?id=0B0hkqO7j3QfbVG1jbXhBSkVHXzA
What I have tried: The one thing I have tried that works is running Aeneas on PLAIN formatted word level input (one word per line) and not passing the '--presets-word' parameter. Unfortunately the results returned by this have words being badly out of sync after about 60 seconds. This is a complete list of the things I've tried to get multilevel alignment working before writing this email ... with no success thus far.
- Ensure that all Aeneas diagnostics pass
- Ensure the input file is clean - removed punctuation and extra \n so that there is only 1 x \n between sentences and \n\n between paragraphs
- Install and run Aeneas on both Python 3.5 and 2.7
- Pass the -r="c_extensions=False" parameter
- Change requested output formats
- Provide a WAV file (PCM 16khz 1 chan little endian) in stead of the MP4 video file
The more options I try, whenever I see the log above I now wonder if the problem is:
- A bug in the C extensions
- Bad wav file input
- Possibly a combination of both the above?
- I need to compile the C extensions manually (but I thought this was done inside the pip install process?)
- Still bad TXT input?
I'd appreciate any suggestions, guidance or help. I feel as though I'm quite close to getting it working for hundreds of hours of audio!
Thanks in advance,
Andrew
Willem van der Walt
Feb 15, 2017, 1:23:53 AM2/15/17
to QA Collective, aeneas-forced-alignment
Hi,
Yes, this is a great tool.
I am attaching a script that works for me.
Just change the language code and what else you require.
The parameter that really made the word-alignment work better was
mfcc_mask_nonspeech_l3=True.
I had core dumps when some of the audio was missing from the wav file.
The parameter os_task_file_no_zero I think, should prevent the zero-length
starts/ends from occuring.
HTH, Willem
On Tue, 14 Feb 2017, QA Collective wrote:
> Hello Alberto & All,
>
> First of all, congratulations on a really great piece of software. Even
> though I haven't quite got it working yet ... I can already see that it is
> a very professionally run project with attention to detail and excellent
> documentation and good old fashioned quality design! I'm anticipating that
> aeneas may be able to do a lot of useful work in my pursuit of training
> DNNs for speech recognition. Unfortunately, I've been getting a seg fault
> while running Aeneas on multilevel mode. I'll try to be detailed without
> going too far...
>
> *What I'm doing: *trying to achieve word level alignment from a TV talk
> *What I have tried:* The one thing I have tried that works is running
> - Ensure the input file is clean - removed punctuation and extra \n so
> - Pass the -r="c_extensions=False" parameter
> - Change requested output formats
> - Provide a WAV file (PCM 16khz 1 chan little endian) in stead of the
> - Bad wav file input
> - Possibly a combination of both the above?
> - I need to compile the C extensions manually (but I thought this was
> You received this message because you are subscribed to the Google Groups "aeneas-forced-alignment" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to aeneas-forced-ali...@googlegroups.com.
> To post to this group, send email to aeneas-forc...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/aeneas-forced-alignment/d528429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.
Please consider the environment before printing this email.
Yes, this is a great tool.
I am attaching a script that works for me.
Just change the language code and what else you require.
The parameter that really made the word-alignment work better was
mfcc_mask_nonspeech_l3=True.
I had core dumps when some of the audio was missing from the wav file.
The parameter os_task_file_no_zero I think, should prevent the zero-length
starts/ends from occuring.
HTH, Willem
On Tue, 14 Feb 2017, QA Collective wrote:
> Hello Alberto & All,
>
> First of all, congratulations on a really great piece of software. Even
> though I haven't quite got it working yet ... I can already see that it is
> a very professionally run project with attention to detail and excellent
> documentation and good old fashioned quality design! I'm anticipating that
> aeneas may be able to do a lot of useful work in my pursuit of training
> DNNs for speech recognition. Unfortunately, I've been getting a seg fault
> while running Aeneas on multilevel mode. I'll try to be detailed without
> going too far...
>
> show which provides transcripts which are well written but do not include
> noises and misspeak from the speakers. I'm providing input transcript in
> MPLAIN format, requesting CSV or SMIL output and providing the
> '--presets-word' option.
>
> *What happens:* Aeneas appears to work as expected (from what I can see
> noises and misspeak from the speakers. I'm providing input transcript in
> MPLAIN format, requesting CSV or SMIL output and providing the
> '--presets-word' option.
>
> Aeneas on PLAIN formatted word level input (one word per line) and not
> passing the '--presets-word' parameter. Unfortunately the results returned
> by this have words being badly out of sync after about 60 seconds. This is
> a complete list of the things I've tried to get multilevel alignment
> working before writing this email ... with no success thus far.
>
> - Ensure that all Aeneas diagnostics pass
> passing the '--presets-word' parameter. Unfortunately the results returned
> by this have words being badly out of sync after about 60 seconds. This is
> a complete list of the things I've tried to get multilevel alignment
> working before writing this email ... with no success thus far.
>
> - Ensure the input file is clean - removed punctuation and extra \n so
> that there is only 1 x \n between sentences and \n\n between paragraphs
> - Install and run Aeneas on both Python 3.5 and 2.7
> - Pass the -r="c_extensions=False" parameter
> - Change requested output formats
> - Provide a WAV file (PCM 16khz 1 chan little endian) in stead of the
> MP4 video file
>
> The more options I try, whenever I see the log above I now wonder if the
> problem is:
>
> - A bug in the C extensions
>
> The more options I try, whenever I see the log above I now wonder if the
> problem is:
>
> - Bad wav file input
> - Possibly a combination of both the above?
> - I need to compile the C extensions manually (but I thought this was
> done inside the pip install process?)
> - Still bad TXT input?
>
>
> I'd appreciate any suggestions, guidance or help. I feel as though I'm
> quite close to getting it working for hundreds of hours of audio!
>
>
> Thanks in advance,
>
> Andrew
>
> --
>
> I'd appreciate any suggestions, guidance or help. I feel as though I'm
> quite close to getting it working for hundreds of hours of audio!
>
>
> Thanks in advance,
>
> Andrew
>
> You received this message because you are subscribed to the Google Groups "aeneas-forced-alignment" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to aeneas-forced-ali...@googlegroups.com.
> To post to this group, send email to aeneas-forc...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/aeneas-forced-alignment/d528429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.
Please consider the environment before printing this email.
Alberto Pettarin
Feb 15, 2017, 4:31:27 AM2/15/17
to aeneas-forc...@googlegroups.com
Hello Andrew,
welcome to the aeneas mailing list.
Thank you for providing the input files and parameters, I was indeed
able to reproduce your problem, and thus I acknowledge there is a bug
somewhere in aeneas.
The segfault is generated in the cdtw C extension when it tries to align
a zero-length interval of real audio with a non-zero length synthetic
audio, and this happens at a leaf (in fact, the very first leaf) of the
"tree" representing the text, i.e. at level 3 (word level). [1]
While one can argue that it is not nice that the C code in a C extension
can segfault crashing the whole Python interpreter [0], I think that the
Python code should not have called the cdtw C code in the first place in
such a situation.
After inspecting your WAV file, I tried adding the following parameters
to the task config string [2]:
is_audio_file_head_length=13.400|is_audio_file_tail_length=71.200
to the config string, to exclude the head and the tail of your audio
file, and this happens to prevent the bug to be triggered, so you get an
output file eventually.
Unfortunately, if you also add the "--presets-word" switch at runtime in
addition to the head/tail parameters [3], you will trigger again the
bug, but now at a later leaf (level 3, fragment 372), so it does not
help that much.
For the record, I also converted your input file in plain format, one
word per line, and run with the parameters [4], and it completes,
although probably the alignment quality is worse than with mplain.
I think the fix for this bug should be rather simple, it is just a
matter of finding the place where the check for zero-length is missing
in the recursion tree. I will investigate this later tonight.
Finally, let me just comment that perhaps aeneas is not the right tool
if you need word-level, high-precision alignment while also feeding long
audio files with noise in it (clapping, laughing, overlapping speakers).
If you work with materials in English, you have plenty of alternatives,
not sure if you checked them out (
https://github.com/pettarin/forced-alignment-tools ).
Anyway, thank you for reporting this issue and providing the input
files/parameters to reproduce it, I will keep you posted.
Best regards,
Alberto Pettarin
For Willem: thank you for sharing your script. Just a note: if you want
to prevent zero-length fragments, the parameter in your config string
should be: task_adjust_boundary_no_zero=True as of v1.7.1. Note the
"=True" part, and the fact that it was renamed for consistency since
v1.7.0. Documentation is at:
https://www.readbeyond.it/aeneas/docs/globalconstants.html#aeneas.globalconstants.PPN_TASK_ADJUST_BOUNDARY_NO_ZERO
Also note that this parameter affects the "post-processing" of the sync
map tree computed via DTW, so it would help with this particular issue.
Footnotes:
[0] TODO for me: add a safety net in cdtw to prevent the segfault.
[1] the fact that the execution tree has a node performing an alignment
between a zero-length interval in the real audio and a non-zero-length
synthetic interval is a symptom of a mis-alignment at the previous level
(level 2, sentence) --- however this might happen, given how aeneas works.
[2] command line:
$ python -m aeneas.tools.execute_task qanda_2012_ep99_climate.wav
qanda_2012_ep99_climate.mp4.txt
"task_language=eng|is_text_type=mplain|os_task_file_format=json|is_audio_file_head_length=13.400|is_audio_file_tail_length=71.200"
qanda_2012_ep99_climate.mplain.json -v -l=mplain.log
[3] command line:
$ python -m aeneas.tools.execute_task qanda_2012_ep99_climate.wav
qanda_2012_ep99_climate.mp4.txt
"task_language=eng|is_text_type=mplain|os_task_file_format=json|is_audio_file_head_length=13.400|is_audio_file_tail_length=71.200"
qanda_2012_ep99_climate.mplain.json -v -l=mplain.log --presets-word
[4] command line:
$ python -m aeneas.tools.execute_task qanda_2012_ep99_climate.wav
qanda_2012_ep99_climate.plain.words.txt
"task_language=eng|is_text_type=plain|os_task_file_format=json|is_audio_file_head_length=13.400|is_audio_file_tail_length=71.200|task_adjust_boundary_nonspeech_string=REMOVE"
qanda_2012_ep99_climate.plain.removed.json -v -l=plain.log --presets-word
On 02/15/2017 02:45 AM, QA Collective wrote:
> Hello Alberto & All,
>
> First of all, congratulations on a really great piece of software. Even
> though I haven't quite got it working yet ... I can already see that it
> is a very professionally run project with attention to detail and
> excellent documentation and good old fashioned quality design! I'm
> anticipating that aeneas may be able to do a lot of useful work in my
> pursuit of training DNNs for speech recognition. Unfortunately, I've
> been getting a seg fault while running Aeneas on multilevel mode. I'll
> try to be detailed without going too far...
>
> *What I'm doing: *trying to achieve word level alignment from a TV talk
> *What I have tried:* The one thing I have tried that works is running
> * Ensure the input file is clean - removed punctuation and extra \n so
> * Pass the -r="c_extensions=False" parameter
> * Change requested output formats
> * Provide a WAV file (PCM 16khz 1 chan little endian) in stead of the
> * Bad wav file input
> * Possibly a combination of both the above?
> * I need to compile the C extensions manually (but I thought this was
> <mailto:aeneas-forc...@googlegroups.com>.
Alberto Pettarin
web: http://readbeyond.it/
web: http://www.albertopettarin.it/
twitter: http://twitter.com/acutebit/
skype: alberto_pettarin
mobile: +39 340 82 18 704
welcome to the aeneas mailing list.
Thank you for providing the input files and parameters, I was indeed
able to reproduce your problem, and thus I acknowledge there is a bug
somewhere in aeneas.
The segfault is generated in the cdtw C extension when it tries to align
a zero-length interval of real audio with a non-zero length synthetic
audio, and this happens at a leaf (in fact, the very first leaf) of the
"tree" representing the text, i.e. at level 3 (word level). [1]
While one can argue that it is not nice that the C code in a C extension
can segfault crashing the whole Python interpreter [0], I think that the
Python code should not have called the cdtw C code in the first place in
such a situation.
After inspecting your WAV file, I tried adding the following parameters
to the task config string [2]:
is_audio_file_head_length=13.400|is_audio_file_tail_length=71.200
to the config string, to exclude the head and the tail of your audio
file, and this happens to prevent the bug to be triggered, so you get an
output file eventually.
Unfortunately, if you also add the "--presets-word" switch at runtime in
addition to the head/tail parameters [3], you will trigger again the
bug, but now at a later leaf (level 3, fragment 372), so it does not
help that much.
For the record, I also converted your input file in plain format, one
word per line, and run with the parameters [4], and it completes,
although probably the alignment quality is worse than with mplain.
I think the fix for this bug should be rather simple, it is just a
matter of finding the place where the check for zero-length is missing
in the recursion tree. I will investigate this later tonight.
Finally, let me just comment that perhaps aeneas is not the right tool
if you need word-level, high-precision alignment while also feeding long
audio files with noise in it (clapping, laughing, overlapping speakers).
If you work with materials in English, you have plenty of alternatives,
not sure if you checked them out (
https://github.com/pettarin/forced-alignment-tools ).
Anyway, thank you for reporting this issue and providing the input
files/parameters to reproduce it, I will keep you posted.
Best regards,
Alberto Pettarin
For Willem: thank you for sharing your script. Just a note: if you want
to prevent zero-length fragments, the parameter in your config string
should be: task_adjust_boundary_no_zero=True as of v1.7.1. Note the
"=True" part, and the fact that it was renamed for consistency since
v1.7.0. Documentation is at:
https://www.readbeyond.it/aeneas/docs/globalconstants.html#aeneas.globalconstants.PPN_TASK_ADJUST_BOUNDARY_NO_ZERO
Also note that this parameter affects the "post-processing" of the sync
map tree computed via DTW, so it would help with this particular issue.
Footnotes:
[0] TODO for me: add a safety net in cdtw to prevent the segfault.
[1] the fact that the execution tree has a node performing an alignment
between a zero-length interval in the real audio and a non-zero-length
synthetic interval is a symptom of a mis-alignment at the previous level
(level 2, sentence) --- however this might happen, given how aeneas works.
[2] command line:
$ python -m aeneas.tools.execute_task qanda_2012_ep99_climate.wav
qanda_2012_ep99_climate.mp4.txt
"task_language=eng|is_text_type=mplain|os_task_file_format=json|is_audio_file_head_length=13.400|is_audio_file_tail_length=71.200"
qanda_2012_ep99_climate.mplain.json -v -l=mplain.log
[3] command line:
$ python -m aeneas.tools.execute_task qanda_2012_ep99_climate.wav
qanda_2012_ep99_climate.mp4.txt
"task_language=eng|is_text_type=mplain|os_task_file_format=json|is_audio_file_head_length=13.400|is_audio_file_tail_length=71.200"
qanda_2012_ep99_climate.mplain.json -v -l=mplain.log --presets-word
[4] command line:
$ python -m aeneas.tools.execute_task qanda_2012_ep99_climate.wav
qanda_2012_ep99_climate.plain.words.txt
"task_language=eng|is_text_type=plain|os_task_file_format=json|is_audio_file_head_length=13.400|is_audio_file_tail_length=71.200|task_adjust_boundary_nonspeech_string=REMOVE"
qanda_2012_ep99_climate.plain.removed.json -v -l=plain.log --presets-word
On 02/15/2017 02:45 AM, QA Collective wrote:
> Hello Alberto & All,
>
> First of all, congratulations on a really great piece of software. Even
> though I haven't quite got it working yet ... I can already see that it
> is a very professionally run project with attention to detail and
> excellent documentation and good old fashioned quality design! I'm
> anticipating that aeneas may be able to do a lot of useful work in my
> pursuit of training DNNs for speech recognition. Unfortunately, I've
> been getting a seg fault while running Aeneas on multilevel mode. I'll
> try to be detailed without going too far...
>
> show which provides transcripts which are well written but do not
> include noises and misspeak from the speakers. I'm providing input
> transcript in MPLAIN format, requesting CSV or SMIL output and providing
> the '--presets-word' option.
>
> *What happens:* Aeneas appears to work as expected (from what I can see
> include noises and misspeak from the speakers. I'm providing input
> transcript in MPLAIN format, requesting CSV or SMIL output and providing
> the '--presets-word' option.
>
> Aeneas on PLAIN formatted word level input (one word per line) and not
> passing the '--presets-word' parameter. Unfortunately the results
> returned by this have words being badly out of sync after about 60
> seconds. This is a complete list of the things I've tried to get
> multilevel alignment working before writing this email ... with no
> success thus far.
>
> * Ensure that all Aeneas diagnostics pass
> passing the '--presets-word' parameter. Unfortunately the results
> returned by this have words being badly out of sync after about 60
> seconds. This is a complete list of the things I've tried to get
> multilevel alignment working before writing this email ... with no
> success thus far.
>
> * Ensure the input file is clean - removed punctuation and extra \n so
> that there is only 1 x \n between sentences and \n\n between paragraphs
> * Install and run Aeneas on both Python 3.5 and 2.7
> * Pass the -r="c_extensions=False" parameter
> * Change requested output formats
> * Provide a WAV file (PCM 16khz 1 chan little endian) in stead of the
> MP4 video file
>
> The more options I try, whenever I see the log above I now wonder if the
> problem is:
>
> * A bug in the C extensions
>
> The more options I try, whenever I see the log above I now wonder if the
> problem is:
>
> * Bad wav file input
> * Possibly a combination of both the above?
> * I need to compile the C extensions manually (but I thought this was
> done inside the pip install process?)
> * Still bad TXT input?
>
>
> I'd appreciate any suggestions, guidance or help. I feel as though I'm
> quite close to getting it working for hundreds of hours of audio!
>
>
> Thanks in advance,
>
> Andrew
>
>
> I'd appreciate any suggestions, guidance or help. I feel as though I'm
> quite close to getting it working for hundreds of hours of audio!
>
>
> Thanks in advance,
>
> Andrew
>
> --
> You received this message because you are subscribed to the Google
> Groups "aeneas-forced-alignment" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to aeneas-forced-ali...@googlegroups.com
> <mailto:aeneas-forced-ali...@googlegroups.com>.
> You received this message because you are subscribed to the Google
> Groups "aeneas-forced-alignment" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to aeneas-forced-ali...@googlegroups.com
> <mailto:aeneas-forc...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/aeneas-forced-alignment/d528429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com
> <https://groups.google.com/d/msgid/aeneas-forced-alignment/d528429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com?utm_medium=email&utm_source=footer>.
> https://groups.google.com/d/msgid/aeneas-forced-alignment/d528429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com
Alberto Pettarin
web: http://readbeyond.it/
web: http://www.albertopettarin.it/
twitter: http://twitter.com/acutebit/
skype: alberto_pettarin
mobile: +39 340 82 18 704
Alberto Pettarin
Feb 15, 2017, 4:33:54 AM2/15/17
to aeneas-forc...@googlegroups.com
Ooops, I clearly meant "so it would NOT help with this particular issue".
Apologies,
AP
Apologies,
AP
QA Collective
Feb 15, 2017, 7:26:50 AM2/15/17
to aeneas-forc...@googlegroups.com
Hi Alberto & Willem,
Thank you very much for your efforts to solve my problem. Much appreciated.
I am in the process of evaluating Aeneas for my purposes but unfortunately I quickly got to this seg fault issue. I am also at the moment trying to determine what level of error DNNs may be able to tolerate.
Alberto, are you suggesting that even with a multi-level alignment, given the possible noise in the audio, I'd be likely to get regular misalignments or alignments that are often off by up to a second?
I may just have to wait for the bug fix and see :)
Until then, I may browse some alternative as you've suggested - the page you wrote is again very thorough. Do you have any suggestions on which may be fast to setup and actually run? I've already tried CMU Sphinx and it takes at least 3x the audio length to get the alignments!
Thanks again,
Andrew
To post to this group, send email to
To view this discussion on the web visit
https://groups.google.com/d/msgid/aeneas-forced-alignment/d528429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com
<https://groups.google.com/d/msgid/aeneas-forced-alignment/d528429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
Alberto Pettarin
web: http://readbeyond.it/
web: http://www.albertopettarin.it/
twitter: http://twitter.com/acutebit/
skype: alberto_pettarin
mobile: +39 340 82 18 704
--
You received this message because you are subscribed to a topic in the Google Groups "aeneas-forced-alignment" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/aeneas-forced-alignment/NnpyiejGkHk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to aeneas-forced-alignment+unsubscr...@googlegroups.com.
To post to this group, send email to aeneas-forced-alignment@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aeneas-forced-alignment/a5fc4f9b-1fcb-8361-134b-8c095c2adb4e%40readbeyond.it.
Willem van der Walt
Feb 15, 2017, 7:55:15 AM2/15/17
to aeneas-forc...@googlegroups.com
Hi Andrew,
To give you some idea of the accuracy you can expect, I can make available
to you some wav files and corresponding textgrid files of an English book.
In short, the cleaner the audio, the better the alignment.
You can grab it at hhttp://hlt.meraka.csir.co.za/animal_farm_alignment.zip
Do not redistribute the audio to others as we have permission to use it
for research purposes only.
HTH, Willem
>>> an email to aeneas-forced-ali...@googlegroups.com
>>> <mailto:aeneas-forced-ali...@googlegroups.com>.
>>> <mailto:aeneas-forc...@googlegroups.com>.
>> db4e%40readbeyond.it.
> To post to this group, send email to aeneas-forc...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/aeneas-forced-alignment/CADF5g4doJ_Vs2pk-p31vu11J8BoM8h20G5cRXFb8g%3D1im3Thig%40mail.gmail.com.
To give you some idea of the accuracy you can expect, I can make available
to you some wav files and corresponding textgrid files of an English book.
In short, the cleaner the audio, the better the alignment.
You can grab it at hhttp://hlt.meraka.csir.co.za/animal_farm_alignment.zip
Do not redistribute the audio to others as we have permission to use it
for research purposes only.
HTH, Willem
>>> <mailto:aeneas-forced-ali...@googlegroups.com>.
>>> To post to this group, send email to
>>> aeneas-forc...@googlegroups.com
>>> <mailto:aeneas-forc...@googlegroups.com>.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/aeneas-forced-alignment/d5
>>> 28429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com
>>> <https://groups.google.com/d/msgid/aeneas-forced-alignment/d
>>> 528429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com?utm_
>>> medium=email&utm_source=footer>.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> Alberto Pettarin
>>
>> web: http://readbeyond.it/
>> web: http://www.albertopettarin.it/
>> twitter: http://twitter.com/acutebit/
>> skype: alberto_pettarin
>> mobile: +39 340 82 18 704
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "aeneas-forced-alignment" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>> pic/aeneas-forced-alignment/NnpyiejGkHk/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> aeneas-forced-ali...@googlegroups.com.
>>> https://groups.google.com/d/msgid/aeneas-forced-alignment/d5
>>> 28429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com
>>> <https://groups.google.com/d/msgid/aeneas-forced-alignment/d
>>> 528429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com?utm_
>>> medium=email&utm_source=footer>.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> Alberto Pettarin
>>
>> web: http://readbeyond.it/
>> web: http://www.albertopettarin.it/
>> twitter: http://twitter.com/acutebit/
>> skype: alberto_pettarin
>> mobile: +39 340 82 18 704
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "aeneas-forced-alignment" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>> pic/aeneas-forced-alignment/NnpyiejGkHk/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> To post to this group, send email to aeneas-forced-alignment@google
>> groups.com.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/aeneas-forced-alignment/a5fc4f9b-1fcb-8361-134b-8c095c2a
>> db4e%40readbeyond.it.
> You received this message because you are subscribed to the Google Groups "aeneas-forced-alignment" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to aeneas-forced-ali...@googlegroups.com.
> To post to this group, send email to aeneas-forc...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/aeneas-forced-alignment/CADF5g4doJ_Vs2pk-p31vu11J8BoM8h20G5cRXFb8g%3D1im3Thig%40mail.gmail.com.
Alberto Pettarin
Feb 15, 2017, 9:22:58 AM2/15/17
to aeneas-forc...@googlegroups.com
On 02/15/2017 01:26 PM, QA Collective wrote:
> Alberto, are you suggesting that even with a multi-level alignment,
> given the possible noise in the audio, I'd be likely to get regular
> misalignments or alignments that are often off by up to a second?
aeneas has been designed with different goals in mind: alignment at
> Alberto, are you suggesting that even with a multi-level alignment,
> given the possible noise in the audio, I'd be likely to get regular
> misalignments or alignments that are often off by up to a second?
sentence/phrase level, support for multiple languages (including
"exotic" ones), speed, and "work-out-of-the-box" reasonable defaults for
a majority of users under those assumptions.
Said that, aeneas can be used to compute word-level alignment, but it
might require some tweaking of the parameters, including the rconf ones,
to get it right. For example, I recently helped a researcher getting
word-level alignment in Danish, with short (5-10 minutes) and clear
(quiet-room-like background noise, but no spurious noises) recordings
and the results were almost perfect with 5 ms resolution.
What concerns me the most is that your input file contains several
intervals of clapping/laughing/overlapping speakers and aeneas is not
designed to handle those, you might get a bad alignment around those
intervals. The cleaner the audio, the better you can hope the alignment
would be --- and of course this is true for all automated aligners.
On top of that, if you need to train your DNN with phone-level (do
you?), then the DTW approach taken by aeneas is probably insufficient
and you would need to use ASR-like techniques (HTK/Sphinx/Kaldi or their
children). Note that e.g. Kaldi has DNN/RNN algorithms and also
pre-trained models for English.
> Until then, I may browse some alternative as you've suggested - the page
> you wrote is again very thorough. Do you have any suggestions on which
> may be fast to setup and actually run? I've already tried CMU Sphinx
> and it takes at least 3x the audio length to get the alignments!
Bash script to install Kaldi and all the other dependencies. I do prefer
CLI instead of having to use curl to talk to a localhost server, but it
at least the setup was not painful.
All the other aligners require some manual labor, e.g. to install
dependencies like HTK. In the past I managed to run both
Prosodylab-Aligner (HTK-based) and the one from the University of
Montreal (Kaldi-based), as they have step-by-step installation guides.
HTH,
AP
Alberto Pettarin
Feb 15, 2017, 3:21:24 PM2/15/17
to aeneas-forc...@googlegroups.com
Logged as issue #160: https://github.com/readbeyond/aeneas/issues/160
The fix is slightly deeper than I thought, but I have coded it already.
The fast tests are all green, but I need to run the full tests before
pushing my local code onto devel/.
I will post an update when the latter is done.
AP
On 02/15/2017 02:45 AM, QA Collective wrote:
> *What I have tried:* The one thing I have tried that works is running
> * Ensure the input file is clean - removed punctuation and extra \n so
> * I need to compile the C extensions manually (but I thought this was
The fix is slightly deeper than I thought, but I have coded it already.
The fast tests are all green, but I need to run the full tests before
pushing my local code onto devel/.
I will post an update when the latter is done.
AP
On 02/15/2017 02:45 AM, QA Collective wrote:
> Hello Alberto & All,
>
> First of all, congratulations on a really great piece of software. Even
> though I haven't quite got it working yet ... I can already see that it
> is a very professionally run project with attention to detail and
> excellent documentation and good old fashioned quality design! I'm
> anticipating that aeneas may be able to do a lot of useful work in my
> pursuit of training DNNs for speech recognition. Unfortunately, I've
> been getting a seg fault while running Aeneas on multilevel mode. I'll
> try to be detailed without going too far...
>
> *What I'm doing: *trying to achieve word level alignment from a TV talk
>
> First of all, congratulations on a really great piece of software. Even
> though I haven't quite got it working yet ... I can already see that it
> is a very professionally run project with attention to detail and
> excellent documentation and good old fashioned quality design! I'm
> anticipating that aeneas may be able to do a lot of useful work in my
> pursuit of training DNNs for speech recognition. Unfortunately, I've
> been getting a seg fault while running Aeneas on multilevel mode. I'll
> try to be detailed without going too far...
>
> show which provides transcripts which are well written but do not
> include noises and misspeak from the speakers. I'm providing input
> transcript in MPLAIN format, requesting CSV or SMIL output and providing
> the '--presets-word' option.
>
> *What happens:* Aeneas appears to work as expected (from what I can see
> include noises and misspeak from the speakers. I'm providing input
> transcript in MPLAIN format, requesting CSV or SMIL output and providing
> the '--presets-word' option.
>
> Aeneas on PLAIN formatted word level input (one word per line) and not
> passing the '--presets-word' parameter. Unfortunately the results
> returned by this have words being badly out of sync after about 60
> seconds. This is a complete list of the things I've tried to get
> multilevel alignment working before writing this email ... with no
> success thus far.
>
> * Ensure that all Aeneas diagnostics pass
> passing the '--presets-word' parameter. Unfortunately the results
> returned by this have words being badly out of sync after about 60
> seconds. This is a complete list of the things I've tried to get
> multilevel alignment working before writing this email ... with no
> success thus far.
>
> * Ensure the input file is clean - removed punctuation and extra \n so
> that there is only 1 x \n between sentences and \n\n between paragraphs
> * Install and run Aeneas on both Python 3.5 and 2.7
> * Pass the -r="c_extensions=False" parameter
> * Change requested output formats
> * Provide a WAV file (PCM 16khz 1 chan little endian) in stead of the
> * Pass the -r="c_extensions=False" parameter
> * Change requested output formats
> MP4 video file
>
> The more options I try, whenever I see the log above I now wonder if the
> problem is:
>
>
> The more options I try, whenever I see the log above I now wonder if the
> problem is:
>
> * A bug in the C extensions
> * Bad wav file input
> * Possibly a combination of both the above?
> * Bad wav file input
> * I need to compile the C extensions manually (but I thought this was
> done inside the pip install process?)
> * Still bad TXT input?
>
>
> I'd appreciate any suggestions, guidance or help. I feel as though I'm
> quite close to getting it working for hundreds of hours of audio!
>
>
> Thanks in advance,
>
> Andrew
>
>
> I'd appreciate any suggestions, guidance or help. I feel as though I'm
> quite close to getting it working for hundreds of hours of audio!
>
>
> Thanks in advance,
>
> Andrew
>
> --
> You received this message because you are subscribed to the Google
> Groups "aeneas-forced-alignment" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to aeneas-forced-ali...@googlegroups.com
> <mailto:aeneas-forced-ali...@googlegroups.com>.
> To post to this group, send email to
> aeneas-forc...@googlegroups.com
> <mailto:aeneas-forc...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/aeneas-forced-alignment/d528429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com
> <https://groups.google.com/d/msgid/aeneas-forced-alignment/d528429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com?utm_medium=email&utm_source=footer>.
> You received this message because you are subscribed to the Google
> Groups "aeneas-forced-alignment" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to aeneas-forced-ali...@googlegroups.com
> <mailto:aeneas-forced-ali...@googlegroups.com>.
> To post to this group, send email to
> aeneas-forc...@googlegroups.com
> <mailto:aeneas-forc...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/aeneas-forced-alignment/d528429f-94b3-4d51-80ab-ec517ae121cb%40googlegroups.com
Alberto Pettarin
Feb 16, 2017, 3:55:42 AM2/16/17
to aeneas-forc...@googlegroups.com
I published the (hopefully) fix for the issue in the devel/ branch:
https://github.com/readbeyond/aeneas/tree/devel
If you want to get it immediately:
$ git clone https://github.com/ReadBeyond/aeneas
$ cd aeneas
$ git checkout devel
$ python setup.py build_ext --inplace
I would like to address another issue (#159) and a few other minor
things before releasing as aeneas v1.7.2, but this cannot happen before
the end of the next week (7-10 days from now), as I have other projects
to ship in the next days.
Thank you for reporting the issue,
AP
https://github.com/readbeyond/aeneas/tree/devel
If you want to get it immediately:
$ git clone https://github.com/ReadBeyond/aeneas
$ cd aeneas
$ git checkout devel
$ python setup.py build_ext --inplace
I would like to address another issue (#159) and a few other minor
things before releasing as aeneas v1.7.2, but this cannot happen before
the end of the next week (7-10 days from now), as I have other projects
to ship in the next days.
Thank you for reporting the issue,
AP
QA Collective
Feb 22, 2017, 12:40:06 AM2/22/17
to aeneas-forc...@googlegroups.com
Hi Alberto,
Thanks for that prompt update. I have since pulled down that dev branch and confirmed that it is now working on the same input file where it was formerly crashing.python -m aeneas.tools.execute_task qanda_2012_ep99_climate.wav qanda_2012_ep99_climate.mp4.txt "task_language=eng|is_text_type=mplain|os_task_file_format=srt|PPN_TASK_OS_FILE_LEVELS=3" qanda_2012_ep99_climate.srt --presets-word
I have also tried adding tts=festival, but verbose output still seems to be using ESPEAK.
--
You received this message because you are subscribed to a topic in the Google Groups "aeneas-forced-alignment" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/aeneas-forced-alignment/NnpyiejGkHk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to aeneas-forced-alignment+unsubscr...@googlegroups.com.
To post to this group, send email to aeneas-forced-alignment@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aeneas-forced-alignment/33f77e48-b9d8-8314-c0b0-5ee37c5e5456%40readbeyond.it.
Alberto Pettarin
Feb 22, 2017, 3:12:17 AM2/22/17
to aeneas-forc...@googlegroups.com
PPN_TASK_OS_FILE_LEVELS is the internal name (notice it is all
uppercase, while the external/userlevel parameters are all lowercase).
Instead, you should use:
"os_task_file_levels=3"
Also, please note that:
$ python -m aeneas.tools.execute_task --list-parameters
and
$ python -m aeneas.tools.execute_task --help-rconf
print the list of config parameters (respectively, runtime parameters)
and a brief description of what they mean.
AP
> <https://groups.google.com/d/topic/aeneas-forced-alignment/NnpyiejGkHk/unsubscribe>.
> <mailto:aeneas-forced-alignment%2Bunsu...@googlegroups.com>.
> <mailto:aeneas-forc...@googlegroups.com>.
> <https://groups.google.com/d/optout>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "aeneas-forced-alignment" group.
> <mailto:aeneas-forc...@googlegroups.com>.
> <https://groups.google.com/d/msgid/aeneas-forced-alignment/CADF5g4f%3Dj49s02pFuSjojgOAeFn5jDVVoFu6PapyXP-932WSwg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
uppercase, while the external/userlevel parameters are all lowercase).
Instead, you should use:
"os_task_file_levels=3"
Also, please note that:
$ python -m aeneas.tools.execute_task --list-parameters
and
$ python -m aeneas.tools.execute_task --help-rconf
print the list of config parameters (respectively, runtime parameters)
and a brief description of what they mean.
AP
> To unsubscribe from this group and all its topics, send an email to
> aeneas-forced-ali...@googlegroups.com
> <mailto:aeneas-forced-alignment%2Bunsu...@googlegroups.com>.
> To post to this group, send email to
> aeneas-forc...@googlegroups.com
> <mailto:aeneas-forc...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/aeneas-forced-alignment/33f77e48-b9d8-8314-c0b0-5ee37c5e5456%40readbeyond.it
> <https://groups.google.com/d/msgid/aeneas-forced-alignment/33f77e48-b9d8-8314-c0b0-5ee37c5e5456%40readbeyond.it>.
> https://groups.google.com/d/msgid/aeneas-forced-alignment/33f77e48-b9d8-8314-c0b0-5ee37c5e5456%40readbeyond.it
> <https://groups.google.com/d/optout>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "aeneas-forced-alignment" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to aeneas-forced-ali...@googlegroups.com
> <mailto:aeneas-forced-ali...@googlegroups.com>.
> an email to aeneas-forced-ali...@googlegroups.com
> <mailto:aeneas-forced-ali...@googlegroups.com>.
> To post to this group, send email to
> aeneas-forc...@googlegroups.com
> <mailto:aeneas-forc...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/aeneas-forced-alignment/CADF5g4f%3Dj49s02pFuSjojgOAeFn5jDVVoFu6PapyXP-932WSwg%40mail.gmail.com
> <https://groups.google.com/d/msgid/aeneas-forced-alignment/CADF5g4f%3Dj49s02pFuSjojgOAeFn5jDVVoFu6PapyXP-932WSwg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Reply all
Reply to author
Forward
0 new messages