 LFD Book Forum *ANSWER* Hw6 q10
 Register FAQ Calendar Mark Forums Read

#1
 GB449 Member Join Date: Aug 2013 Posts: 20 *ANSWER* Hw6 q10

I get 522. If we assume two hidden layers with 18 units each, we have a total of four layers. From L0 to L1, there will be 180 weights, from L1 to L2 there will be 18*18 = 324 weights and from L2 to L3, 18 weights. Total = 522

I went with two layers of equal units as an educated guess using this reasoning from numbers: given any two positive integers whose sum is a constant, the product is maximum if the two numbers are equal (e.g. x + y = 10, max of x*y is 5*5 = 25).

Since the answer is 510, I seem to be making a mistake but not sure what that mistake is.
#2
 foruhar Junior Member Join Date: Jun 2014 Posts: 2 Re: *ANSWER* Hw6 q10

got the same which is not in the provided options... #3 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,478 Re: *ANSWER* Hw6 q10

Maybe take a look at

Post 14 seems to have resolved the issue for a number of participants.
__________________
Where everyone thinks alike, no one thinks very much
#4
 foruhar Junior Member Join Date: Jun 2014 Posts: 2 Re: *ANSWER* Hw6 q10

Thanks a lot professor! I got the right answer.

The error is introduced by the fact that the constant node does not receive any input weights.

Tackling the general problem: Assuming two hidden layers, with m and n nodes, and i input nodes, the goal is to

maximize: (i*(m-1)) + (m*(n-1)) + (n*1)

given m + n = s (s is the total number of hidden nodes allowed - in this case 36)

I think it it is intuitive to have 2 layers vs more when maximizing number of weights:
- First adding more layers creates more wasted constant nodes.
- Secondly even if there were no wasted nodes we would be creating smaller multipliers and then summing them up vs multiplying them directly. i.e. adding layers cannot increase the weights among the hidden layers (it only deprives some nodes from connecting to others by putting them more than one layer apart) and cannot increase the connections to the input and output nodes.

In general, (using my dusty calculus), Setting n = s - m, differentiating over n and setting to zero, I get:

n = (s + 2 - i)/2

Where n is the number of nodes in the second layer, s is the sum total of hidden nodes and i is the number of input nodes.

It also seems to indicates that if s + 2 is less than or equal to i, having one layer instead of two is the way to go.
#5
 allenyin Junior Member Join Date: Aug 2015 Posts: 2 Re: *ANSWER* Hw6 q10

The answer still doesn't make sense to me...even when I agree with your approach.

If the arrangement that gives the most weights is (10)-18-18-(1), where the hidden nodes are distributed evenly. Then according to your formula (which I agree is correct), would give:

10*(18-1)+18*(18-1)+18=494

This is still not the answer given (510). What am I doing wrong?

 Tags homework6, question10 Thread Tools Show Printable Version Email this Page Display Modes Linear Mode Switch to Hybrid Mode Switch to Threaded Mode Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 12:04 PM. The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.