LFD Book Forum (http://book.caltech.edu/bookforum/index.php)
-   Homework 6 (http://book.caltech.edu/bookforum/forumdisplay.php?f=135)
-   -   Homework 6 #10 (http://book.caltech.edu/bookforum/showthread.php?t=494)

 mjbeeson 05-14-2012 10:48 AM

Homework 6 #10

Counting weights. Just to clarify if I am counting right: suppose we have 2 input nodes and one hidden layer with 2 nodes and one output node. So then we have 4 weights, right? According to the problem statement the constant input nodes are counted in the "2". Similarly if every input or hidden layer has an even number of nodes then we must get an even number for the total number of weights. Do you agree? Or do I have an off-by-one error in this count somehow?

 cassio 05-14-2012 10:59 AM

Re: Homework 6 #10

Quote:
 Originally Posted by mjbeeson (Post 2108) Counting weights. Just to clarify if I am counting right: suppose we have 2 input nodes and one hidden layer with 2 nodes and one output node. So then we have 4 weights, right?
Maybe I'm not right, but I think that in your example we have 6 weights. 4 to fully connect the input layer with the hidden layer, plus 2 for the hidden layer with the output layer

 rohanag 05-14-2012 11:05 AM

Re: Homework 6 #10

Quote:
 Originally Posted by cassio (Post 2110) Maybe I'm not right, but I think that in your example we have 6 weights. 4 to fully connect the input layer with the hidden layer, plus 2 for the hidden layer with the output layer
4 nodes are not needed to connect the input layer with the hidden one, if he wants one of the 2 hidden layer nodes to be constant.

 rohanag 05-14-2012 11:08 AM

Re: Homework 6 #10

so if you did count the constant nodes in your specification, I think 4 is correct for the final answer

 cassio 05-14-2012 11:09 AM

Re: Homework 6 #10

Quote:
 Originally Posted by rohanag (Post 2111) Not 4 nodes to connect the input layer with the hidden one, if he wants one of the 2 hidden layer nodes to be constant. Do you mjbeeson?
But even the constant is multiplied by a weight that connects it to the next layer, does not?

 rohanag 05-14-2012 11:10 AM

Re: Homework 6 #10

Quote:
 Originally Posted by cassio (Post 2113) But even the constant is multiplied by a weight that connects it to the next layer, does not?
Yes so, that's why there are 2 weights from the input to the hidden layer, and 2 nodes from the hidden layer to the final output node.

 cassio 05-14-2012 11:17 AM

Re: Homework 6 #10

Quote:
 Originally Posted by rohanag (Post 2114) Yes so, that's why there are 2 weights from the input to the hidden layer, and 2 nodes from the hidden layer to the final output node.
Yes yes, now I understood what you meant. The constant unit does not receive connections from back layers.

 rohanag 05-14-2012 11:19 AM

Re: Homework 6 #10

Quote:
 Originally Posted by cassio (Post 2115) Yes yes, now I understood what you meant. The constant unit does not receive connections from back layers.
exactly :)

 cassio 05-14-2012 11:23 AM

Re: Homework 6 #10

@mjbeeson, I am sorry for misled you in my first post.

 Yellin 05-14-2012 11:46 AM

Re: Homework 6 #10

Quote:
 Originally Posted by mjbeeson (Post 2108) Counting weights. Just to clarify if I am counting right: suppose we have 2 input nodes and one hidden layer with 2 nodes and one output node. So then we have 4 weights, right? According to the problem statement the constant input nodes are counted in the "2". Similarly if every input or hidden layer has an even number of nodes then we must get an even number for the total number of weights. Do you agree? Or do I have an off-by-one error in this count somehow?
There may be some confusion here between "nodes", which are the little circles in the neural net diagrams, and "units", which are the x's that enter and leave the circles. So mibeeson's example may mean there are 2 input units, of which one is constant, with two nodes, each receiving the two input units and producing one unit each. Adding one constant unit to the two units from those two nodes makes for 3 hidden units; the three go to one output node, which produces the output unit. Each node needs weights for each of its incoming units, so there are 4 weights from the first layer of nodes plus 3 from the three units entering the output node. Is this reasonable, or am I the one who is confused?

 cassio 05-14-2012 12:23 PM

Re: Homework 6 #10

Quote:
 Originally Posted by Yellin (Post 2119) There may be some confusion here between "nodes", which are the little circles in the neural net diagrams, and "units", which are the x's that enter and leave the circles. So mibeeson's example may mean there are 2 input units, of which one is constant, with two nodes, each receiving the two input units and producing one unit each. Adding one constant unit to the two units from those two nodes makes for 3 hidden units; the three go to one output node, which produces the output unit. Each node needs weights for each of its incoming units, so there are 4 weights from the first layer of nodes plus 3 from the three units entering the output node. Is this reasonable, or am I the one who is confused?
Yellin, in my understanding you are right in the example you gave. And I would like to add the other interpretation of the problem proposed by mjbeeson where rohanag opened my eyes: if I have 2 input units (one of them is the constant) and one node that receive both connections from the input units. Now to next layer, I have the unit value from the node and one constant unit (what result in two units in this hidden layer), both connecting with the output node. Thus, I have two weights in first part and more two in the last. Am I right?

 mjbeeson 05-14-2012 12:48 PM

Re: Homework 6 #10

I don't think that's the way we are to read the homework. Notice problem 8, where it says,
"10 input units (including the constant value)...and 36 hidden units (include the constant inputs of each hidden layer).
The "including" part important in the counting.

 dvs79 08-16-2012 05:00 AM

Re: Homework 6 #10

Has anybody got the correct answer? My answer was wrong and I can't see where is a mistake in my reasoning.
The maximum number of weights should be for the network looking like 10-18-18-1 (including constant nodes in the first three layers), right? But number of weights for it is not one in the answer key.

 Keith 08-17-2012 08:01 AM

Re: Homework 6 #10

I got this one right. I used Excel (of all things) to explore all arrangements with one and two hidden layers, and also looked at portions of arrangements with more hidden layers. The spreadsheet does the arithmetic with simple copied formulas and gives the correct answer without too much work.

 dvs79 08-17-2012 12:39 PM

Re: Homework 6 #10

Oh, I see. My stupid blunder.

Thnx! :)

 DavidNJ 08-21-2012 11:33 AM

Re: Homework 6 #10

Now I feel like an idiot! :(

You could also have used goal seek in Excel.

 tzs29970 08-21-2012 01:08 PM

Re: Homework 6 #10

In case anyone is curious, there are 9227465 possible networks that satisfy the constraints of this problem.

 ilya239 02-13-2013 02:02 PM

Re: Homework 6 #10

Quote:
 Originally Posted by mjbeeson (Post 2108) Counting weights. Just to clarify if I am counting right: suppose we have 2 input nodes and one hidden layer with 2 nodes and one output node. So then we have 4 weights, right? According to the problem statement the constant input nodes are counted in the "2". Similarly if every input or hidden layer has an even number of nodes then we must get an even number for the total number of weights. Do you agree? Or do I have an off-by-one error in this count somehow?
"2 input nodes", does that include the constant node or not? I.e. does layer 0 look like or like ? What's the right standard terminology to use here?

 yaser 02-13-2013 05:02 PM

Re: Homework 6 #10

Quote:
 Originally Posted by ilya239 (Post 9388) What's the right standard terminology to use here?
In the statement of this problem (preamble before Problem 9), the constant variables are counted as units.

 spramod 02-17-2013 08:15 AM

Re: Homework 6 #10

Professor, after the deadline, could you please post an explanation to this and Question 9, if there's a general case analytical solution, or if there's any other straightforward method?

 yaser 02-17-2013 09:54 AM

Re: Homework 6 #10

Quote:
 Originally Posted by spramod (Post 9415) Professor, after the deadline, could you please post an explanation to this and Question 9, if there's a general case analytical solution, or if there's any other straightforward method?
Please start an *ANSWER* thread after the deadline, where people can discuss different approaches. No general methodology really, but mostly looking at different combinations to seek out the two extremes.

 All times are GMT -7. The time now is 02:20 PM.