View Single Post
Old 03-16-2013, 11:36 PM
yaser's Avatar
yaser yaser is offline
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,478
Default Re: Neural Network predictions converging to one value

Originally Posted by rk1000 View Post
Sure, I just tried the training that you suggested. With repeated training over the same 2 examples that have different target outputs, it learns correctly, and eventually predicts them both with no training error.
OK. Now, this narrows it down to inability to learn properly on the full data set. Two possibilities

1. Computational: Not enough epochs, or a bad local minimum.

2. Inherent: The target function is almost impossible to capture given the size of the network.

Let's deal with 1 first. Try a very long run with say 100 times the epochs and see if the result is better. Also try 100 different runs (initial random weights seeded differently) with the smaller number of epochs and see how the best result is.
Where everyone thinks alike, no one thinks very much
Reply With Quote