LFD Book Forum Problem 1.5 Adaline
 User Name Remember Me? Password
 FAQ Calendar Mark Forums Read

 Thread Tools Display Modes
#1
12-09-2017, 10:05 AM
 don slowik Member Join Date: Nov 2017 Posts: 11
Problem 1.5 Adaline

I had bad luck with the ALA: for all but the smallest training data sets and with more than 2 dimensions, the weights would go scooting off to infinity.

I modified the algorithm so as to become a regression vs categorization problem, I changed the update criteria to be:
Code:
```s = np.dot(x[i,:], w)
if np.abs(y[i] - s) > 0.01:
w = w + eta * (y[i] - s) * x[i,:]
n_updates += 1```
This worked very well, with eta set to 0.1, for training sets of size N=1000 in d=10 dimensions required only 2.7 +/-1.1 iterations through the data to achieve the tolerance of 0.1 on every training data point. PLA on the same training data required about 750 iterations.

So rather than choosing a plane that separates the data, this chooses the plan that gets the correct distance (within the 0.01) between the plan and the data point for every training data point.
#2
12-10-2017, 06:44 AM
 don slowik Member Join Date: Nov 2017 Posts: 11
Re: Problem 1.5 Adaline

Though this is interesting, on further thought, it seems to be quite useless. The y associated with each training point is the distance between that point to the separating plane. So you would have to know the plane to begin with..
#3
12-13-2017, 01:34 PM
 htlin NTU Join Date: Aug 2009 Location: Taipei, Taiwan Posts: 601
Re: Problem 1.5 Adaline

This looks like the Adaline algorithm, by the way.

https://en.wikipedia.org/wiki/ADALINE
__________________
When one teaches, two learn.
#4
12-18-2017, 05:53 PM
 don slowik Member Join Date: Nov 2017 Posts: 11
Re: Problem 1.5 Adaline

Actually, it isn't that useless if the data happens to be that then adaline is a quick way of converging to a plane that fits the data. Yes, thanks for that wikipedia reference.
#5
12-28-2017, 01:28 AM
 pdsubraa Member Join Date: Aug 2017 Location: Singapore Posts: 20
Re: Problem 1.5 Adaline

Well Said Don!

Wikipedia reference was helpful - Thanks Htlin!

 Tags ala, classify vs regression

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 11:36 PM.

 Contact Us - LFD Book - Top

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.