 LFD Book Forum HW5 Q8~9: Is the line a straight line?
 Register FAQ Calendar Mark Forums Read

#1
 lucag Member Join Date: Apr 2012 Posts: 44 HW5 Q8~9: Is the line a straight line?

In homework 5, questions 8 and 9, is the random line to be chosen as a target function f(x) a straight line?
I feel it's a rather silly question ... but I am somewhat confused on what target function to pick.
#2
 kkkkk Invited Guest Join Date: Mar 2012 Posts: 71 Re: HW5 Q8~9: Is the line a straight line?

Yes, it is a straight line.
#3
 mathprof Invited Guest Join Date: Apr 2012 Location: Bakersfield, California Posts: 36 Re: HW5 Q8~9: Is the line a straight line?

.. Unless someone left the line out in the rain; then it may warp, and no longer be straight. #4
 ehaussmann Junior Member Join Date: Apr 2012 Posts: 5 Re: HW5 Q8~9: Is the line a straight line?

Am I missing something or is the name f(x) a bit misleading? In the slides for logistic regression f(x) is explicitly referred to as a probability. However in the exercise it is a linear function that may also take negative function values.

It seems the function f(x) corresponds to the line that is learned inside the sigmoid function, but that's not the final g(x) we are learning (which should approximate f(x) ) ?
#5
 mic00 Invited Guest Join Date: Apr 2012 Posts: 49 Re: HW5 Q8~9: Is the line a straight line?

Quote:
 Originally Posted by ehaussmann Am I missing something or is the name f(x) a bit misleading? In the slides for logistic regression f(x) is explicitly referred to as a probability. However in the exercise it is a linear function that may also take negative function values. It seems the function f(x) corresponds to the line that is learned inside the sigmoid function, but that's not the final g(x) we are learning (which should approximate f(x) ) ?
The presence of f in the problem description may be a little confusing, since it hardly plays a role. f(x) is indeed a probability: P(y | x) is written in terms of it. Specifically, P(y=+1 | x) = f(x), and P(y=-1 | x) = 1 - f(x). But the problem description specifies a very simple f(x): it is always 1 on one side of the line, and always 0 on the other side. In other words, if we are on one side of the line, the "probability" that y will be +1 is exactly 1.0, and if we are on the other, the "probability" that y will be -1 is also exactly 1.0. Suppose it were a little different, say, f(x)=0.95 on one side of the line, and f(x) = 0.05 on the other. That would mean that, as you generate sample points, there is a little bit of noisiness -- the data are (probably) not linearly separable. Logistic regression is interesting because it attempts to predict probabilistic targets like this (hence the sigmoid function, which maps x*w to a probability).

In this problem, though, stochastic gradient descent is the focus. The fact that probabilities are involved is only really interesting because of the error function: if the data are linearly separable (as they are here), but E_in is not zero, then it has behaved differently from PLA, even if its goal is the same here.
#6
 ehaussmann Junior Member Join Date: Apr 2012 Posts: 5 Re: HW5 Q8~9: Is the line a straight line?

Thank you for clarifying. Meanwhile I realized that the excercise just utilizes a straight line to define f(x), but f(x) itself is not the straight line...

In hindsight things are (almost) always obvious Tags hw5, logistic regression Thread Tools Show Printable Version Email this Page Display Modes Linear Mode Switch to Hybrid Mode Switch to Threaded Mode Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 06:14 PM. The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.