LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 6

Reply
 
Thread Tools Display Modes
  #1  
Old 08-21-2012, 09:06 AM
apinde apinde is offline
Member
 
Join Date: Jul 2012
Posts: 12
Default Subset & deterministic noise

Related to Q1 in HW6 I used the following reasoning to say that there is no trend. Whats the falacy?

Construct the hypothesis set H as the union of two hypothesis sets H1 and H2
Let H1 have higher deterministic noise and H2 have lower
H will have deterministic noise between H1 and H2
Now let H' be the set H1. Clearly H' is a subset of H ans H' has higher deterministic noise than H
Similarly H' can be H2, and then H' has lower deterministic noise than H

Therefore there is no trend
Reply With Quote
  #2  
Old 08-21-2012, 09:44 AM
Andrs Andrs is offline
Member
 
Join Date: Jul 2012
Posts: 47
Default Re: Subset & deterministic noise

If H is the union of H1 and H2 and assuming that H1 differs from H2, the hypothesis set H should be at least as complex as H2 therfore it should have a lower deterministic noise.
Related to the question:
The basic charachteristic is that H' is a subset of H therefore H' contains fewer alternatives compared to H (less options less flexibility). That is, H' will not be able to fit into the target function f as well as H would have. That is, H' implies a higher "deterministic noise".
Reply With Quote
  #3  
Old 08-21-2012, 10:42 AM
apinde apinde is offline
Member
 
Join Date: Jul 2012
Posts: 12
Default Re: Subset & deterministic noise

Thanks for your response, Andrs.

Another expression of deterministic noise is the expected value of the squared error between the mean hypothesis and f(x). The mean hypothesis when one has disjoint H1 and H2 should fall somewhere between the more complex and the more simple sets and the deterministic error should be somewhere between the deterministic errors of H1 and H2. It other words, the deterministic error of H is not just the lower of the two.
Reply With Quote
  #4  
Old 08-21-2012, 11:21 AM
MLearning MLearning is offline
Senior Member
 
Join Date: Jul 2012
Posts: 56
Default Re: Subset & deterministic noise

The argument that there is no trend seems to make sense to me. We are not told about the complexity of the hypothesis, what we are told is both H' and H contain a set of hypothesis functions which can be simple or complex.

I disagree with the assertion that fewer alternatives means less options and less complexity. A hypothesis set may have a few number of possible hypothesis functions but one of those functions can have the complexity to provide a good fit. Hence, less deterministic noise.
Reply With Quote
  #5  
Old 08-21-2012, 11:40 AM
MLearning MLearning is offline
Senior Member
 
Join Date: Jul 2012
Posts: 56
Default Re: Subset & deterministic noise

The argument that there is no trend seems to make sense to me. We are not told about the complexity of the hypothesis, what we are told is both H' and H contain a set of hypothesis functions which can be simple or complex.

I disagree with the assertion that fewer alternatives means less options and less complexity. A hypothesis set may have a few number of possible hypothesis functions but one of those functions can have the complexity to provide a good fit. Hence, less deterministic noise.
Reply With Quote
  #6  
Old 08-21-2012, 12:30 PM
JohnH JohnH is offline
Member
 
Join Date: Jul 2012
Posts: 43
Default Re: Subset & deterministic noise

The phrase "deterministic noise" refers to complexity in the target function that cannot be described by the hypothesis set. \mathcal{H} is analogous to a vocabulary with each h being a word within that vocabulary. Removing words from the vocabulary, \mathcal{H}'\subset\mathcal{H}, diminishes expressive capability; i.e., there are fewer things that can be described.
Reply With Quote
  #7  
Old 08-21-2012, 12:39 PM
MLearning MLearning is offline
Senior Member
 
Join Date: Jul 2012
Posts: 56
Default Re: Subset & deterministic noise

@JohnH,

I see your point and your argument makes sense. However, what if those remaining volcabularies in H' are complex enough to completely characterize the target function. Remember the target function is fixed in this case.
Reply With Quote
  #8  
Old 08-21-2012, 03:13 PM
JohnH JohnH is offline
Member
 
Join Date: Jul 2012
Posts: 43
Default Re: Subset & deterministic noise

The question was about trending, not a specific case. The question could be posed in terms of expected outcome. Is \mathcal{H}' more or less likely than \mathcal{H} to be sufficient to describe the target function?
Reply With Quote
  #9  
Old 02-13-2013, 07:00 PM
ilya239 ilya239 is offline
Senior Member
 
Join Date: Jul 2012
Posts: 58
Default Re: Subset & deterministic noise

Quote:
Originally Posted by MLearning View Post
@JohnH,

I see your point and your argument makes sense. However, what if those remaining volcabularies in H' are complex enough to completely characterize the target function. Remember the target function is fixed in this case.
Deterministic noise will either be reduced or stay the same. So the change is zero with some probability and negative with some probability, so the expected change is negative. (What a hand-waving argument )
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 08:39 AM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.