LFD Book Forum - Problem 1.5 Adaline

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)

- Chapter 1 - The Learning Problem (http://book.caltech.edu/bookforum/forumdisplay.php?f=108)

- - Problem 1.5 Adaline (http://book.caltech.edu/bookforum/showthread.php?t=4805)

Problem 1.5 Adaline

I had bad luck with the ALA: for all but the smallest training data sets and with more than 2 dimensions, the weights would go scooting off to infinity.

I modified the algorithm so as to become a regression vs categorization problem, I changed the update criteria to be:

Code:

s = np.dot(x[i,:], w)

if np.abs(y[i] - s) > 0.01:                    

   w = w + eta * (y[i] - s) * x[i,:] 

   n_updates += 1

This worked very well, with eta set to 0.1, for training sets of size N=1000 in d=10 dimensions required only 2.7 +/-1.1 iterations through the data to achieve the tolerance of 0.1 on every training data point. PLA on the same training data required about 750 iterations.

So rather than choosing a plane that separates the data, this chooses the plan that gets the correct distance (within the 0.01) between the plan and the data point for every training data point.

Re: Problem 1.5 Adaline

Though this is interesting, on further thought, it seems to be quite useless. The y associated with each training point is the distance between that point to the separating plane. So you would have to know the plane to begin with..

Re: Problem 1.5 Adaline

This looks like the Adaline algorithm, by the way.

https://en.wikipedia.org/wiki/ADALINE

Re: Problem 1.5 Adaline

Actually, it isn't that useless if the data happens to be that then adaline is a quick way of converging to a plane that fits the data. Yes, thanks for that wikipedia reference.

Re: Problem 1.5 Adaline

Well Said Don!

Wikipedia reference was helpful - Thanks Htlin!