LFD Book Forum  

Go Back   LFD Book Forum > Course Discussions > Online LFD course > Homework 6

Reply
 
Thread Tools Display Modes
  #1  
Old 09-21-2013, 12:09 AM
GB449 GB449 is offline
Member
 
Join Date: Aug 2013
Posts: 20
Default *ANSWER* Hw6 q10

I get 522. If we assume two hidden layers with 18 units each, we have a total of four layers. From L0 to L1, there will be 180 weights, from L1 to L2 there will be 18*18 = 324 weights and from L2 to L3, 18 weights. Total = 522

I went with two layers of equal units as an educated guess using this reasoning from numbers: given any two positive integers whose sum is a constant, the product is maximum if the two numbers are equal (e.g. x + y = 10, max of x*y is 5*5 = 25).

Since the answer is 510, I seem to be making a mistake but not sure what that mistake is.
Reply With Quote
  #2  
Old 08-09-2014, 06:05 PM
foruhar foruhar is offline
Junior Member
 
Join Date: Jun 2014
Posts: 2
Default Re: *ANSWER* Hw6 q10

got the same which is not in the provided options...
Reply With Quote
  #3  
Old 08-10-2014, 06:53 PM
yaser's Avatar
yaser yaser is offline
Caltech
 
Join Date: Aug 2009
Location: Pasadena, California, USA
Posts: 1,474
Default Re: *ANSWER* Hw6 q10

Maybe take a look at

http://book.caltech.edu/bookforum/showthread.php?t=494

Post 14 seems to have resolved the issue for a number of participants.
__________________
Where everyone thinks alike, no one thinks very much
Reply With Quote
  #4  
Old 08-17-2014, 11:30 AM
foruhar foruhar is offline
Junior Member
 
Join Date: Jun 2014
Posts: 2
Default Re: *ANSWER* Hw6 q10

Thanks a lot professor! I got the right answer.

The error is introduced by the fact that the constant node does not receive any input weights.

Tackling the general problem: Assuming two hidden layers, with m and n nodes, and i input nodes, the goal is to

maximize: (i*(m-1)) + (m*(n-1)) + (n*1)

given m + n = s (s is the total number of hidden nodes allowed - in this case 36)

I think it it is intuitive to have 2 layers vs more when maximizing number of weights:
- First adding more layers creates more wasted constant nodes.
- Secondly even if there were no wasted nodes we would be creating smaller multipliers and then summing them up vs multiplying them directly. i.e. adding layers cannot increase the weights among the hidden layers (it only deprives some nodes from connecting to others by putting them more than one layer apart) and cannot increase the connections to the input and output nodes.

In general, (using my dusty calculus), Setting n = s - m, differentiating over n and setting to zero, I get:

n = (s + 2 - i)/2

Where n is the number of nodes in the second layer, s is the sum total of hidden nodes and i is the number of input nodes.

It also seems to indicates that if s + 2 is less than or equal to i, having one layer instead of two is the way to go.
Reply With Quote
  #5  
Old 08-25-2015, 04:27 PM
allenyin allenyin is offline
Junior Member
 
Join Date: Aug 2015
Posts: 2
Default Re: *ANSWER* Hw6 q10

The answer still doesn't make sense to me...even when I agree with your approach.

If the arrangement that gives the most weights is (10)-18-18-(1), where the hidden nodes are distributed evenly. Then according to your formula (which I agree is correct), would give:

10*(18-1)+18*(18-1)+18=494

This is still not the answer given (510). What am I doing wrong?
Reply With Quote
Reply

Tags
homework6, question10

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 11:18 PM.


Powered by vBulletin® Version 3.8.3
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
The contents of this forum are to be used ONLY by readers of the Learning From Data book by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, and participants in the Learning From Data MOOC by Yaser S. Abu-Mostafa. No part of these contents is to be communicated or made accessible to ANY other person or entity.