LFD Book Forum

LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Chapter 1 - The Learning Problem (http://book.caltech.edu/bookforum/forumdisplay.php?f=108)
-   -   what is bias means? (http://book.caltech.edu/bookforum/showthread.php?t=348)

antinucleon 04-15-2012 09:06 PM

what is bias means?
 
I am confused with the bias.
In wikipedia, w0 is called threshold, so I think it may be any value.
But in this lecture, we make this tuple (1, x0, y0). When in perception process, all changes performed on w0 is +1 or -1, it can't reach the degree of 0.1 or more precious.
Where am I wrong?

yaser 04-15-2012 10:01 PM

Re: what is bias means?
 
Quote:

Originally Posted by antinucleon (Post 1313)
I am confused with the bias.
In wikipedia, w0 is called threshold, so I think it may be any value.
But in this lecture, we make this tuple (1, x0, y0). When in perception process, all changes performed on w0 is +1 or -1, it can't reach the degree of 0.1 or more precious.
Where am I wrong?

This may be a case of notational confusion, as the lectures follow the book notation, not wikipedia. The tuple in the lecture was (x_0,x_1,x_2) which is the input vector {\bf x}, and x_0=1 by definition. The weight vector is (w_0,w_1,w_2) and all 3 coordinates can assume different values, with w_0 being the threshold (also referred to as bias) coordinate.

antinucleon 04-16-2012 01:08 AM

Re: what is bias means?
 
Quote:

Originally Posted by yaser (Post 1315)
This may be a case of notational confusion, as the lectures follow the book notation, not wikipedia. The tuple in the lecture was (x_0,x_1,x_2) which is the input vector {\bf x}, and x_0=1 by definition. The weight vector is (w_0,w_1,w_2) and all 3 coordinates can assume different values, with w_0 being the threshold (also referred to as bias) coordinate.


Thanks dear prof first!
Assume we start at init hypothesis w=(0, 0, 0), init target function is w_t=(0.5, 2, 5)
As w = w + yx, y equals 0 or 1,x_n = (1, 0.5, 3) etc
When in iteration, w_0 = w_0 + y*x_0, as y and x_0 can only be 0 or 1, so the hypothesis function's threshold may be quite different from the target function, right?

htlin 04-16-2012 02:47 PM

Re: what is bias means?
 
Quote:

Originally Posted by antinucleon (Post 1323)
Thanks dear prof first!
Assume we start at init hypothesis w=(0, 0, 0), init target function is w_t=(0.5, 2, 5)
As w = w + yx, y equals 0 or 1,x_n = (1, 0.5, 3) etc
When in iteration, w_0 = w_0 + y*x_0, as y and x_0 can only be 0 or 1, so the hypothesis function's threshold may be quite different from the target function, right?

In your case, the target function is equivalently (1, 4, 10), (2, 8, 20), so it can still be implemented with the PLA model even with integer w_0. But of course, learning is not for replicating the target function, but for approximating it. In practice the learned hypothesis is somewhat different from the target function more often than not. Hope this helps.

don slowik 12-05-2017 07:00 PM

Re: what is bias means?
 
Problem 1.3 proves that PLA converges (to give the correct sign for each training data point) and in steps of size x as you point out (and of size +/-1 for the 0 component). As w grows in magnitude the fractional accuracy will improve, and after it has converged, normalizing w[1:] to a unit vector gives the direction vector of the separating plane and w[0] will be its' offset from the origin.


All times are GMT -7. The time now is 12:49 AM.

Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.