Same prediction for all inputs

[Eric]

I'm trying to build a simple neural network and train and test with a dummy function; however, the predictions are always around 1.0.

The code can be found at: http://pastebin.com/q6bYbdN1

I'm using the following versions:

Theano==0.7.0

Lasagne==0.1

Nolearn==0.6a

Any help would be greatly appreciated!

Thanks,
Eric

[goo...@jan-schlueter.de]

I'm trying to build a simple neural network and train and test with a dummy function; however, the predictions are always around 1.0.

Your targets are between 0 and 87, but your output nonlinearity is a tanh(). Predicting 1.0 is as close as it can get. Try a linear output layer (nonlinearity=None) and regression with mean squared error. Possibly also try lowering the learning rate. If the learning rate is too high, your hidden layer weights may easily get too big, causing the tanh() hidden units to get stuck (saturate).
 

Any help would be greatly appreciated!

If you're just getting started, have a look at the MNIST example/walkthrough: http://lasagne.readthedocs.org/en/latest/user/tutorial.html
It will give you an experiment that is known to work, so you can start from there. There are similar tutorials for nolearn as well.

Hope this helps!
Best, Jan

[Eric]

Thanks!!

Do you have any suggestions regarding reading material for guidance to setting things like the activation function, learning rate, etc?

[goo...@jan-schlueter.de]

Thanks!!

Do you have any suggestions regarding reading material for guidance to setting things like the activation function, learning rate, etc?

Again, please have a look at the tutorial: http://lasagne.readthedocs.org/en/latest/user/tutorial.html
The "Before we get started" section links two well-written introductions to neural networks (by Michael Nielsen and Andrej Karpathy, respectively).

Best, Jan