GPU run out of memory when use nn.SpatialDepthWiseConvolution

[Elliot He]

Originally, I have the ResNet code running using nn.spatialconvolution. Once I got the neural network training, GPU memory usage basically did not change.

However, after I replace the nn.spatialconvolution with nn.SpatialDepthWiseConvolution (with some add-on simple layers), I noticed that the GPU memory usage keeps growing after training each mini-batch. How could I solve it?