MobileNet's DepthwiseConv2D vs SeparableConv2D

[trevor....@gmail.com]

Hi all,

I've noticed that MobileNet's implementation defines a new layer DepthwiseConv2D. How is this different from the built in SeparableConv2D? Could the implementation of MobileNet just use that instead? If not, why not?

[alexand...@gmail.com]

Was wondering about this aswell. Comparing the source descriptions theres obviously an extra step included in SeperableConv2D following after the DepthwiseConv2D called 'pointwise convolution'. Im not certain but i think that's just a standard 1x1 conv like those used to make residual blocks

with different channel numbers fit together in ResNets. When I've used DepthwiseConv2D, Ive found them to be much trickier to use when trying to downsample the number

of channels, so I suspect by adding those layers he might be trying to save us dimensional headaches.

to make layers with different channel numbers match). Probably worth noting that Francois Chollet is kindof obsessed with depthwise seperable convs

and wrote to papers and built a full model pretty much entirely based on them, so he probably recreated the original 'purer' SeperableConv layer used in the MobileNet paper

so that the model weights would fit but then for his library's api gave us what he thought we needed.

class DepthwiseConv2D(Conv2D):
	"""Depthwise separable 2D convolution.
	Depthwise Separable convolutions consists in performing
	just the first step in a depthwise spatial convolution
	(which acts on each input channel separately).
	The `depth_multiplier` argument controls how many
	output channels are generated per input channel in the depthwise step.

class SeparableConv2D(Conv2D):
	"""Depthwise separable 2D convolution.
	Separable convolutions consist in first performing
	a depthwise spatial convolution
	(which acts on each input channel separately)
	followed by a pointwise convolution which mixes together the resulting
	output channels. The `depth_multiplier` argument controls how many
	output channels are generated per input channel in the depthwise step.
	Intuitively, separable convolutions can be understood as
	a way to factorize a convolution kernel into two smaller kernels,
	or as an extreme version of an Inception block.

[philipp...@resson.com]

Late reply, but I'm fairly certain the DepthwiseConv2D layer in keras is just the first portion of of the SeparableConv2D layer. In the MobileNet implementation one block consists of DepthwiseConv2D ->BatchNorm->Relu-> PointwiseConv.

The SeparableConv2D is DepthwiseConv2D -> PointwiseConv.

I've also been wondering why they added so much for the mobilenet implementation, but I think it is specifically  to match the mobilenet paper which has the additional intermediate batchnorm and relu.

[Trevor Standley]

Thanks for the reply. I think you're right. It'd be cool if SeparableConv2D had some boolean parameters for including a relu and BN in the middle.

--
You received this message because you are subscribed to a topic in the Google Groups "Keras-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/keras-users/sec8pYjJwwE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to keras-users+unsubscribe@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/keras-users/52bc5294-ef65-4adb-8488-b5dba949d860%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Daπid]

On 24 October 2017 at 03:46, Trevor Standley <trevor....@gmail.com> wrote:

Thanks for the reply. I think you're right. It'd be cool if SeparableConv2D had some boolean parameters for including a relu and BN in the middle.

No need, I wrap this kind of layer groups it in a function:

def add_1d_resnet(input_layer, length, nb_filt, REG):
    tower_1 = Conv1D(nb_filt, length, padding='same', kernel_initializer=INIT, kernel_regularizer=l2(REG))(input_layer)
    tower_1 = ACTIVATION()(tower_1)
    tower_1 = BatchNormalization(**bn_opts)(tower_1)
    tower_1 = Dropout(0.1)(tower_1)

    tower_1 = Conv1D(nb_filt, length, padding='same', kernel_initializer=INIT, kernel_regularizer=l2(REG))(tower_1)
    tower_1 = ACTIVATION()(tower_1)
    tower_1 = BatchNormalization(**bn_opts)(tower_1)
    tower_1 = Dropout(0.1)(tower_1)

    x = add([tower_1, input_layer])
    return x

 

--
You received this message because you are subscribed to the Google Groups "Keras-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keras-users+unsubscribe@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/keras-users/CALSvU-55WMxSQw_wOa2RyjTif9f4YHDZQY_1kG5vG2MAGJx89Q%40mail.gmail.com.

[ihsan...@gmail.com]

Hi David,

Is this code for having 1D signal as input or for 2d images? Can you kindly guide or provide an example how can i use mobilenet for 1D signal? 

Your earliest response in this regards will be highly appreciated. 

Regards

Ihs

To unsubscribe from this group and all its topics, send an email to keras-users...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/keras-users/52bc5294-ef65-4adb-8488-b5dba949d860%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Keras-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to keras-users...@googlegroups.com.