Is any difference between "prefinal-chain.batchnorm" and "prefinal-chain-batchnorm" ?

[付嘉懿]

Hi all,

  I'm looking into the final.mdl of the TDNN and I found there are two batchnorm component in the prefinal layer.

  The first one is named "prefinal-chain.batchnorm" and the second one is named "prefinal-chain-batchnorm", both of them are "type=BatchNormComponent".

  At the beginning I thought the difference in their names is used to distinguish two batchnorm. But when I use the same C++ code to read the parameters ("offset_" and "scale_") of these two components, I found I only can read the correct parameter of the  first one(prefinal-chain.batchnorm) and the parameter of the second one("prefinal-chain-batchnorm") is different from the value in final.mdl.

  So I want to know is there any difference between these to batchnorm component ? or I just make some mistakes in my code ? I means the symbol ("." or "-") has any special meaning ?

  Thanks!

[Daniel Povey]

There are two branches at the output, and only one of them is used for decoding (the prefinal-chain branch); the other one is used only in training.

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+unsubscribe@googlegroups.com.
To post to this group, send email to kaldi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/b5e13697-235f-4102-8970-280e5e97f3b4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[付嘉懿]

Thanks Dan.

You means the branch prefinal-chain and the branch prefinal-xnet ? 

In branch prefinal-chain, is there any difference between prefinal-chain-batchnorm and prefinal-chain.batchnorm ?

在 2018年7月11日星期三 UTC+8上午11:36:35，Dan Povey写道：

There are two branches at the output, and only one of them is used for decoding (the prefinal-chain branch); the other one is used only in training.

On Tue, Jul 10, 2018 at 11:26 PM, 付嘉懿 <zxczx...@gmail.com> wrote:

Hi all,

  I'm looking into the final.mdl of the TDNN and I found there are two batchnorm component in the prefinal layer.

  The first one is named "prefinal-chain.batchnorm" and the second one is named "prefinal-chain-batchnorm", both of them are "type=BatchNormComponent".

  At the beginning I thought the difference in their names is used to distinguish two batchnorm. But when I use the same C++ code to read the parameters ("offset_" and "scale_") of these two components, I found I only can read the correct parameter of the  first one(prefinal-chain.batchnorm) and the parameter of the second one("prefinal-chain-batchnorm") is different from the value in final.mdl.

  So I want to know is there any difference between these to batchnorm component ? or I just make some mistakes in my code ? I means the symbol ("." or "-") has any special meaning ?

  Thanks!

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.

[Daniel Povey]

Oh-- I misread it.

No, the . and - have no special meaning.

In some TDNN-F models, there is a layer called prefinal-chain that had a batchnorm component (which would be called prefinal-chain.batchnorm), and also a component specified directly from the xconfig called prefinal-chain-batchnorm.  They are distinct and they are both used in the computation.

Dan

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+unsubscribe@googlegroups.com.

To post to this group, send email to kaldi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/dbee3838-9411-48b0-be10-c79e03b44e1c%40googlegroups.com.

[付嘉懿]

Thanks Dan.

Actually I'm looking into a TDNN-F model. I used the function "BatchNormComponent::Offset" and "BatchNormComponent::Scale" to read the parameters of these two component and compare with final.mdl.

I found the parameter of prefinal-chain.batchnorm is equal to final.mdl but the prefinal-chain-batchnorm is not.

So I want to know is there any difference between these two components ? just like  prefinal-chain.batchnorm is the Class BatchNormComponent and prefinal-chain-batchnorm is another Class ?

Or I just make some mistake in my code.

Thanks again!

在 2018年7月11日星期三 UTC+8下午12:09:37，Dan Povey写道：

[Daniel Povey]

I don't know what you mean by the offset and scale being different from those in final.mdl because those are not actually written in the model, only the stats mean and variance.  What this tells me is that you must have two different methods of figuring out the scale and offsets from the final.mdl file, and those two different methods are giving you two different answers.  But without knowing what those two different methods are, I wouldn't be able to say why the answers differ.

There is no difference in the code.

Dan

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+unsubscribe@googlegroups.com.

To post to this group, send email to kaldi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/d1508e1f-2cd3-4bc0-9934-185a48173688%40googlegroups.com.

[付嘉懿]

Thanks Dan. 

I have another related question about the batchnorm component of prefinal-layer in the TDNN-F model.

I printed the component name with function "nnet_.GetComponentName" in the code nnet-compute.cc.

The component names of tdnn layer are "tdnn11.affine" or "tdnn10.batchnorm" or something and they are easy to understand as a affine operation or a batchnorm operation.

But one component name of prefinal layer is "prefinal-chain-batchnorm.output.affine".

I'm confused about this name. It's a batchnorm operation or a affine operation ? 

在 2018年7月12日星期四 UTC+8上午1:36:54，Dan Povey写道：

[付嘉懿]

I know. It combine the batchnorm of prefinal layer and the affine of output layer.

Thanks Dan.

在 2018年7月13日星期五 UTC+8上午11:11:06，付嘉懿写道：