LFD Book Forum Problem 1.9

#11
06-23-2017, 04:42 PM
 RicLouRiv Junior Member Join Date: Jun 2017 Posts: 7
Re: Problem 1.9

Quick question on Part D...

I agree that s = ln(a / (1-a)) is the value of s that minimizes e^(-sa)U(s), where I'm using "a" to represent "alpha."

I think that, if you plug this value of s in, you get 2^(-b), where I'm using "b" to represent "beta."

So, 2^(-b) < e^(-sa)U(s), by definition of minimization.

Then, raising to the power of N, we get:

2^(-bN) < (e^(-sa)U(s))^N, but this inequality is the wrong way.

Any tips?
#12
06-24-2017, 09:36 AM
 RicLouRiv Junior Member Join Date: Jun 2017 Posts: 7
Re: Problem 1.9

Ah, maybe it's because the inequality holds for any s, it must hold for the min.
#13
06-29-2017, 10:00 PM
 htlin NTU Join Date: Aug 2009 Location: Taipei, Taiwan Posts: 601
Re: Problem 1.9

Quote:
 Originally Posted by RicLouRiv Ah, maybe it's because the inequality holds for any s, it must hold for the min.
You got it. :-)
__________________
When one teaches, two learn.
#14
10-22-2017, 04:19 AM
 mygame182 Junior Member Join Date: Oct 2017 Posts: 2
Re: Problem 1.9

Quote:
 Originally Posted by RicLouRiv Ah, maybe it's because the inequality holds for any s, it must hold for the min.
Excuse me, I can understand your last reply, but confuse in this one.

When 2^(-b) is the minimize of e^(-sa)U(s), i find that 1-a = 1/2 + e (e means epsilon), so P[u>=a] = P[u>= 1/2 - e] , but not P[u>=1/2+e].
#15
10-23-2017, 08:23 AM
 mygame182 Junior Member Join Date: Oct 2017 Posts: 2
Re: Problem 1.9

When 2^(-b) equal to the minimize of e^(-sa)U(s), i try to assume that 1-a=1/2-e, so that a = 1/2 + e, P[u>=a]=P[u>=1/2+e]

According to (b), P[u>=1/2+e] = P[u>=a] <= (e^(-sa)U(s))^N for any s , even if the minimize of e^(-sa)U(s) when s = ln(a / (1-a)).

Hence P[u>=1/2+e] <= 2^(-bN)
#16
11-14-2017, 03:14 PM
 don slowik Member Join Date: Nov 2017 Posts: 11
Re: Problem 1.9

That's correct. But it doesn't get us anywhere.
#17
11-14-2017, 03:18 PM
 don slowik Member Join Date: Nov 2017 Posts: 11
Re: Problem 1.9

Ahhhm, That was a reply to kongweihan's OP.
It is correct as shown by the first line: Since...
But ends up barking up the wrong tree.
There is a nice description on wikipedia of Chernoff bound.
It is similar to maciekleks path.
#18
11-09-2018, 07:02 PM
 Ulyssesyang Junior Member Join Date: Nov 2018 Posts: 3
Re: Problem 1.9

Quote:
 Originally Posted by svend Here's my take on Problem 1.9, part(b), which is following the same lines as the description of MaciekLeks above. We have: Since is monotonically increasing in t. Also, is non negative for all t, implying Markov inequality holds: The last line being true since [math]x_n[\math] are independent. From there it directly follows that
I just think you should note that there are two expectations, one is based on e^su_n, while other is based on u_n. Of course, you can refer the law of unconscious statisticians to prove that both are same.

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 06:28 AM.