Subset & deterministic noise
Related to Q1 in HW6 I used the following reasoning to say that there is no trend. Whats the falacy?
Construct the hypothesis set H as the union of two hypothesis sets H1 and H2 Let H1 have higher deterministic noise and H2 have lower H will have deterministic noise between H1 and H2 Now let H' be the set H1. Clearly H' is a subset of H ans H' has higher deterministic noise than H Similarly H' can be H2, and then H' has lower deterministic noise than H Therefore there is no trend 
Re: Subset & deterministic noise
If H is the union of H1 and H2 and assuming that H1 differs from H2, the hypothesis set H should be at least as complex as H2 therfore it should have a lower deterministic noise.
Related to the question: The basic charachteristic is that H' is a subset of H therefore H' contains fewer alternatives compared to H (less options less flexibility). That is, H' will not be able to fit into the target function f as well as H would have. That is, H' implies a higher "deterministic noise". 
Re: Subset & deterministic noise
Thanks for your response, Andrs.
Another expression of deterministic noise is the expected value of the squared error between the mean hypothesis and f(x). The mean hypothesis when one has disjoint H1 and H2 should fall somewhere between the more complex and the more simple sets and the deterministic error should be somewhere between the deterministic errors of H1 and H2. It other words, the deterministic error of H is not just the lower of the two. 
Re: Subset & deterministic noise
The argument that there is no trend seems to make sense to me. We are not told about the complexity of the hypothesis, what we are told is both H' and H contain a set of hypothesis functions which can be simple or complex.
I disagree with the assertion that fewer alternatives means less options and less complexity. A hypothesis set may have a few number of possible hypothesis functions but one of those functions can have the complexity to provide a good fit. Hence, less deterministic noise. 
Re: Subset & deterministic noise
The argument that there is no trend seems to make sense to me. We are not told about the complexity of the hypothesis, what we are told is both H' and H contain a set of hypothesis functions which can be simple or complex.
I disagree with the assertion that fewer alternatives means less options and less complexity. A hypothesis set may have a few number of possible hypothesis functions but one of those functions can have the complexity to provide a good fit. Hence, less deterministic noise. 
Re: Subset & deterministic noise
@JohnH,
I see your point and your argument makes sense. However, what if those remaining volcabularies in H' are complex enough to completely characterize the target function. Remember the target function is fixed in this case. 
Re: Subset & deterministic noise

Re: Subset & deterministic noise
Quote:

All times are GMT 7. The time now is 06:05 AM. 
Powered by vBulletin® Version 3.8.3
Copyright ©2000  2021, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. AbuMostafa, Malik MagdonIsmail, and HsuanTien Lin, and participants in the Learning From Data MOOC by Yaser S. AbuMostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.