Kaldi version 5.4

[Daniel Povey]

The master on github now points to Kaldi 5.4.

The main change is the availability of new and improved recipes which are a kind of factorized TDNN, for instance, in

egs/swbd/s5c/local/chain/tuning/run_tdnn_lstm_1n.sh

There are also less-drastically modified versions of the mini_librispeech and WSJ recipes.  (For smaller datasets, only a linear bottleneck just before the output layer was helpful; putting linear bottlenecks in the other layers was not helpful).

We'll be working on applying this architecture to the other recipes.

Actually I am working on a further improvement to these recipes, involving a form of dropout.

Dan

[Rudolf Arseni Braun]

Forgive me for the basic question but is this really a 20+ layer network?

https://github.com/kaldi-asr/kaldi/blob/master/egs/swbd/s5c/local/chain/tuning/run_tdnn_7n.sh

Or is relu_batchnorm_layer just the pointwise relu + batchnorm (so one layer is actually linearcomponent + relu[-batchnorm]) ? Or maybe this can only be explained by talking about the factorized TDNNs you mention? Looking forward to a write-up!

[Daniel Povey]

It's really a 20+ layer network in the sense that there are 20 linear or affine transforms, but there are only about 11 or 12 nonlinearities (ReLUs), and we view each pair of (linear or affine) operations as a kind of factorization of a single layer.  So an alternative way to view it is as a factorization of an 11 or 12-layer network.

Dan

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+unsubscribe@googlegroups.com.
To post to this group, send email to kaldi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/30ae348b-1c11-4080-be1b-07704d232ff0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rémi Francis]

Have you tried having skip connections the way resnets do it (i.e. just an addition instead of concatenating the hidden states)?

On Monday, 19 February 2018 19:33:29 UTC, Dan Povey wrote:

It's really a 20+ layer network in the sense that there are 20 linear or affine transforms, but there are only about 11 or 12 nonlinearities (ReLUs), and we view each pair of (linear or affine) operations as a kind of factorization of a single layer.  So an alternative way to view it is as a factorization of an 11 or 12-layer network.

Dan

On Mon, Feb 19, 2018 at 2:31 PM, Rudolf Arseni Braun <rab...@gmail.com> wrote:

Forgive me for the basic question but is this really a 20+ layer network?

https://github.com/kaldi-asr/kaldi/blob/master/egs/swbd/s5c/local/chain/tuning/run_tdnn_7n.sh

Or is relu_batchnorm_layer just the pointwise relu + batchnorm (so one layer is actually linearcomponent + relu[-batchnorm]) ? Or maybe this can only be explained by talking about the factorized TDNNs you mention? Looking forward to a write-up!

On Saturday, February 17, 2018 at 7:14:44 AM UTC, Dan Povey wrote:

The master on github now points to Kaldi 5.4.

The main change is the availability of new and improved recipes which are a kind of factorized TDNN, for instance, in

egs/swbd/s5c/local/chain/tuning/run_tdnn_lstm_1n.sh

There are also less-drastically modified versions of the mini_librispeech and WSJ recipes.  (For smaller datasets, only a linear bottleneck just before the output layer was helpful; putting linear bottlenecks in the other layers was not helpful).

We'll be working on applying this architecture to the other recipes.

Actually I am working on a further improvement to these recipes, involving a form of dropout.

Dan

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.

[Daniel Povey]

In the past I had tried those resnet-style additions and had not seen a benefit, but it's definitely worth trying again with the current architecture.  I hadn't tried it for this type of model.

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+unsubscribe@googlegroups.com.

To post to this group, send email to kaldi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/c86bc241-e998-42da-b6c6-60666f0e1a61%40googlegroups.com.