View Single Post
Old 03-16-2013, 09:52 PM
yaser's Avatar
yaser yaser is offline
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,478
Default Re: Neural Network predictions converging to one value

Originally Posted by rk1000 View Post

The training data points do have different target outputs.

After training from a mini-batch of the training set and adjusting the weights, when I try to predict the outputs of the same training mini-batch, it produces identical (incorrect) outputs when it has undergone some training. Initially i.e. at the beginning of training, these values are very far wrong and also different but many are the same, but eventually after some training, all the predictions on the training set start to converge, and they get closer to the mean of the outputs of the latest training mini-batch.
OK, so somehow the training phase is not working. There could be different reasons for that. Can I ask you to try to train on only two examples with different outputs and see what happens?
Where everyone thinks alike, no one thinks very much
Reply With Quote