hw2 q10: noise calculation on Eout?

[learnaholic]

Hi,

I'm trying to figure out where I went wrong on #10, but I'm not getting it.

There's only one place I can think of where I went wrong and I want to see if this is the place because if so, I need an explanation.

So I calculated all my Eout points using the weights that I gathered from problem 9. I then calculated how many points were misclassified. I did this by calculating what the real value should be by using the target function. When I did this, I got approximately .032.

Now....the only thing I could think of as to where I went wrong was to use the target function that had some noise when producing the output. If I do this, I get something closer to .1.

I feel very uncomfortable with this answer, however. When I calculate my Eout, I really only care about what the actual target function should produce, not what a noisy target function produces. Why would I care about what a noisy target function produces? I only care if the value matched what the actual target function should produce without noise, no?

To use a real example: Let's say that the function given was the absolute truth for whether I should approve someone for credit. I then got a data set where 5% of the people that should have gotten credit didn't, and 5% of the people who shouldn't have gotten credit did. If I plugged these people into the target function, I would have found those 10% to be misclassified. Now, when I run my Eout, I don't know what the answer is, but because we preconceived this target function (which we wouldn't know in the real world), I can actually check to see how often my hypothesis was correct.

If this is where I went wrong, I hope I can get an explanation. If it's not where I went wrong, I'll keep digging, but I hope someone can provide an explanation as to what the program should look like.

Thanks!

[dudefromdayton]

I read this to mean that the actual target function is noisy.

[learnaholic]

I hope I can either the Professor or a TA to confirm this. I reread the question, and I still don't read it that way. It does say "consider the target function <blah>" rather than "consider a target function <blah> that has noise".

Can I get confirmation on this? I'm not trying to grade grub here because I'm not taking the class for credit, but I'd like to verify that I am understanding the problem correctly.

[yaser]

Originally Posted by learnaholic

I hope I can either the Professor or a TA to confirm this. I reread the question, and I still don't read it that way. It does say "consider the target function <blah>" rather than "consider a target function <blah> that has noise".

Can I get confirmation on this? I'm not trying to grade grub here because I'm not taking the class for credit, but I'd like to verify that I am understanding the problem correctly.

Hi,

The target is indeed noisy. Question 10 prescibes adding noise when is calculated by specifying "Estimate it by generating a new set of 1000 points and adding noise as before."

In the course and in the book, we either have noiseless target and noiseless data set, or noisy target and noisy data set. I can see where you are coming from since there are treatments elsewhere in the literature that add noise to the data while leaving the target and the test points noiseless.

[learnaholic]

Thanks Professor, I now understand why I was wrong.

In a similar vein....I know other people have asked this, but since I'm here more for the learning as opposed to the grades, I'd find it really, really helpful if I can have more information on the questions I got "right", too. I put "right" in quotes because I do wonder if I got the correct answer because I knew the material or if I got it wrong, but my wrong answer was close enough.

I guess what I'm saying is that I need a better error number on my data set rather than just a +1 or -1?