LFD Book Forum *ANSWER* Hw6 q10

#1
09-21-2013, 01:09 AM
 GB449 Member Join Date: Aug 2013 Posts: 20

I get 522. If we assume two hidden layers with 18 units each, we have a total of four layers. From L0 to L1, there will be 180 weights, from L1 to L2 there will be 18*18 = 324 weights and from L2 to L3, 18 weights. Total = 522

I went with two layers of equal units as an educated guess using this reasoning from numbers: given any two positive integers whose sum is a constant, the product is maximum if the two numbers are equal (e.g. x + y = 10, max of x*y is 5*5 = 25).

Since the answer is 510, I seem to be making a mistake but not sure what that mistake is.
#2
08-09-2014, 07:05 PM
 foruhar Junior Member Join Date: Jun 2014 Posts: 2

got the same which is not in the provided options...
#3
08-10-2014, 07:53 PM
 yaser Caltech Join Date: Aug 2009 Location: Pasadena, California, USA Posts: 1,477

Maybe take a look at

Post 14 seems to have resolved the issue for a number of participants.
__________________
Where everyone thinks alike, no one thinks very much
#4
08-17-2014, 12:30 PM
 foruhar Junior Member Join Date: Jun 2014 Posts: 2

Thanks a lot professor! I got the right answer.

The error is introduced by the fact that the constant node does not receive any input weights.

Tackling the general problem: Assuming two hidden layers, with m and n nodes, and i input nodes, the goal is to

maximize: (i*(m-1)) + (m*(n-1)) + (n*1)

given m + n = s (s is the total number of hidden nodes allowed - in this case 36)

I think it it is intuitive to have 2 layers vs more when maximizing number of weights:
- First adding more layers creates more wasted constant nodes.
- Secondly even if there were no wasted nodes we would be creating smaller multipliers and then summing them up vs multiplying them directly. i.e. adding layers cannot increase the weights among the hidden layers (it only deprives some nodes from connecting to others by putting them more than one layer apart) and cannot increase the connections to the input and output nodes.

In general, (using my dusty calculus), Setting n = s - m, differentiating over n and setting to zero, I get:

n = (s + 2 - i)/2

Where n is the number of nodes in the second layer, s is the sum total of hidden nodes and i is the number of input nodes.

It also seems to indicates that if s + 2 is less than or equal to i, having one layer instead of two is the way to go.
#5
08-25-2015, 05:27 PM
 allenyin Junior Member Join Date: Aug 2015 Posts: 2

The answer still doesn't make sense to me...even when I agree with your approach.

If the arrangement that gives the most weights is (10)-18-18-(1), where the hidden nodes are distributed evenly. Then according to your formula (which I agree is correct), would give:

10*(18-1)+18*(18-1)+18=494

This is still not the answer given (510). What am I doing wrong?

 Tags homework6, question10

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home General     General Discussion of Machine Learning     Free Additional Material         Dynamic e-Chapters         Dynamic e-Appendices Course Discussions     Online LFD course         General comments on the course         Homework 1         Homework 2         Homework 3         Homework 4         Homework 5         Homework 6         Homework 7         Homework 8         The Final         Create New Homework Problems Book Feedback - Learning From Data     General comments on the book     Chapter 1 - The Learning Problem     Chapter 2 - Training versus Testing     Chapter 3 - The Linear Model     Chapter 4 - Overfitting     Chapter 5 - Three Learning Principles     e-Chapter 6 - Similarity Based Methods     e-Chapter 7 - Neural Networks     e-Chapter 8 - Support Vector Machines     e-Chapter 9 - Learning Aides     Appendix and Notation     e-Appendices

All times are GMT -7. The time now is 04:04 PM.