Autoencoder for multi-channel input images

[i.mar...@gmail.com]

Hi all,

I am trying to make an autoencoder for multi-channel input. 

For example, instead of an input image of shape (1,H,W) my input is of shape (15,H,W).

The autoencoders I found on the internet worked well for 1 channel but (very) bad for more than 1 (15 in my case).

Did someone tried something similar before?

Many Thanks,

Ioannis

[Jan Schlüter]

The autoencoders I found on the internet worked well for 1 channel but (very) bad for more than 1 (15 in my case).

Did someone tried something similar before?

I don't see an inherent reason why it should perform worse. Searching for "color image autoencoder" gives you plenty of examples that work on 3 channels. The DCGAN paper and follow-up work on GANs might also give you some ideas on how to design the network (with the discriminator architecture as the encoder, and the generator architecture as the decoder).

Good luck!
Best, Jan

[i.mar...@gmail.com]

Hi Jan,

Thank you for your reply.

The fusion of the multiple channels is the key I think. For 1 channel there is not need for fusion but for i.e. 15 channels this is not so easy. 

For MNIST dataset ( https://stats.stackexchange.com/questions/176881/cannot-make-this-autoencoder-network-function-properly-with-convolutional-and-m ) with 1x28x28 input images, a dense layer with 2 dimensions in the middle is fine to reconstruct the images but for 15 channels even with 128 dimensions the result is very bad.

In case of VAE (https://github.com/Lasagne/Recipes/blob/master/examples/variational_autoencoder/variational_autoencoder.py) this problem holds.

Do you have a specific AE (preferably written in Lasagne) in your mind for me to try it (except  DCGAN)?

Best,

Ioannis

[Jan Schlüter]

Do you have a specific AE (preferably written in Lasagne) in your mind for me to try it (except  DCGAN)?

No, sorry. Maybe a GAN is not the best source of inspiration anyway, since it only has to produce plausible samples, not reproduce the input. It only serves to show how to get from a vectorized hidden representation to a color image. If you can't get auto-encoders to work, but only need *something* that can encode a multi-channel image and decode it back, also have a look at ALI, BiGAN and BEGAN.

Best, Jan

[i.mar...@gmail.com]

Hi Jan,

Many Thanks,

Ioannis