LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 2 (http://book.caltech.edu/bookforum/forumdisplay.php?f=131)
-   -   How to do the homework? (http://book.caltech.edu/bookforum/showthread.php?t=362)

 ShengrongYIN 04-17-2012 07:45 PM

How to do the homework?

It would be very appreciated for your help! I was totally exhausted with the homework. I want to know whether it is necessary to do the homework with some computer tools(like Octove or Matlab), if so ,how can I solve these problems using these tools. I may not understand the problem well. Is there any online tutorial to solve the problems step by step?

 dudefromdayton 04-17-2012 09:05 PM

Re: How to do the homework?

You ask some very good questions. If you're exhausted, I might suggest making an educated guess at some of the answers. Alternatively, you might be able to eliminate some of the alternatives. For the first two homework sets, I found a strong correlation between the estimates I could make without a computer, and the experimental results obtained after a lot more work.

 magdon 04-17-2012 09:17 PM

Re: How to do the homework?

Indeed, if you are doing the homework manually, I am not surprised at all that you are exhausted. I would be dead. You are expected to build computer programs to solve the problems by implementing the algorithms. Convenient languages to work with are matlab, octave, R, python.

Quote:
 Originally Posted by ShengrongYIN (Post 1374) It would be very appreciated for your help! I was totally exhausted with the homework. I want to know whether it is necessary to do the homework with some computer tools(like Octove or Matlab), if so ,how can I solve these problems using these tools. I may not understand the problem well. Is there any online tutorial to solve the problems step by step?

 Hillbilly 04-18-2012 07:39 PM

Re: How to do the homework?

@dudefromdayton, your remarks stir curiosity. Could you be more specific please, as with an example, of how to make an "educated guess" on these homeworks (that's correct!)? Does doing so presume prior education on the subject of the questions (beyond the basic prerequisites stated)? How can one make an educated guess when not yet educated on the subject? I'm not trying to be snarky, just curious about what you mean. :clueless:

I, too, found the homeworks time-consuming and somewhat tiring, and would love to know any shortcuts! I did OK in the end, but it took a lot of work, suggesting that my understanding is shallow.

Even more curious, how did you make reasonable estimates for any of the problems requiring simulations? Did you use relationships, equations, etc. known to practitioners, but not mentioned in class yet? For example, results reported from the Homework 1 PLA simulation suggest that iterations to convergence is close to N, the training set size (for randomly choosing a misclassification to update). If that hypothesis is correct, and was known beforehand, then of course, answering correctly was trivial without doing the programming.

Would appreciate any helpful hints. Thanks!

Quote:
 Originally Posted by dudefromdayton (Post 1379) If you're exhausted, I might suggest making an educated guess at some of the answers. ..... For the first two homework sets, I found a strong correlation between the estimates I could make without a computer, and the experimental results obtained after a lot more work.

 dudefromdayton 04-19-2012 10:30 AM

Re: How to do the homework?

I would like to, and Yaser knows what I have in mind. Let me make sure it's appropriate here before I post!

 dudefromdayton 04-26-2012 09:39 PM

Re: How to do the homework?

@Hillbilly, I don't want to overstep on this topic. But have a look at problem 9 (in the second homework set). Start from the target function:

f(x1, x2) = sign(x1^2 + x2^2 - 0.6)

Notice that it's symmetric in x1 and x2? What's that suggest to you about choices b, c, and d? Why would the two strongest coefficients in the polynomial differ by tenfold between these variables? And choice e, how does the x1x2 term become so significant compared to the others? In fact, what would the geometric significance of the x1x2 term be in terms of classification?

Let's also look at problem 10, which uses the same target function and illustrates a similar point. The target function wasn't originally linearly separable, but your transformation definitely made it so. This means you're running linear regression on a linearly separable set of data, and you have a fairly large set of it. How much error will this result in? A lot? Medium? Very little? But the target function has 10% noise, which depending on what you decided on my last question, E[out] will be 10% + a lot, 10% + somewhat, or 10% + negligible.

In general, it is not easy to write multiple-choice problem sets on advanced topics. Just ask the folks who write questions for the SAT! Not long ago, it was possible to answer reading comprehension questions without reading the passage -- just pick the answer which wouldn't cast any group of people in a bad light. It was also possible to identify (and skip) entire sections of the exam which would not be graded.

 Hillbilly 05-02-2012 07:14 AM

Re: How to do the homework?

@dudefromdayton, thank you for those examples of "off-the-cuff" reasoning (I assume that's an appropriate term). I ended up doing something similar on Homework 4, for lack of time. For example, I intended to try both analytical and brute-force simulation approaches to solving #7 (which learning model has least out-of-sample error), but some other problems used up my time quota. So I employed rough-and-ready reasoning, and got the right answer (I claim no victory, however, given the 20% chance of a random guess being right!).

I think it's a good idea to employ multiple methods, if enough time -- mathematical analysis, numerical examples, simulation, graphical methods, even intuition. If they all produce the same answer, the problem is mastered; if not, then something needs fixing!

 All times are GMT -7. The time now is 04:35 PM.