Homework 6 #10

[mjbeeson]

Counting weights. Just to clarify if I am counting right: suppose we have 2 input nodes and one hidden layer with 2 nodes and one output node. So then we have 4 weights, right? According to the problem statement the constant input nodes are counted in the "2". Similarly if every input or hidden layer has an even number of nodes then we must get an even number for the total number of weights. Do you agree? Or do I have an off-by-one error in this count somehow?

[cassio]

Originally Posted by mjbeeson

Counting weights. Just to clarify if I am counting right: suppose we have 2 input nodes and one hidden layer with 2 nodes and one output node. So then we have 4 weights, right?

Maybe I'm not right, but I think that in your example we have 6 weights. 4 to fully connect the input layer with the hidden layer, plus 2 for the hidden layer with the output layer

[rohanag]

Originally Posted by cassio

Maybe I'm not right, but I think that in your example we have 6 weights. 4 to fully connect the input layer with the hidden layer, plus 2 for the hidden layer with the output layer

4 nodes are not needed to connect the input layer with the hidden one, if he wants one of the 2 hidden layer nodes to be constant.

[rohanag]

so if you did count the constant nodes in your specification, I think 4 is correct for the final answer

[cassio]

Originally Posted by rohanag

Not 4 nodes to connect the input layer with the hidden one, if he wants one of the 2 hidden layer nodes to be constant. Do you mjbeeson?

But even the constant is multiplied by a weight that connects it to the next layer, does not?

[rohanag]

Originally Posted by cassio

But even the constant is multiplied by a weight that connects it to the next layer, does not?

Yes so, that's why there are 2 weights from the input to the hidden layer, and 2 nodes from the hidden layer to the final output node.

[cassio]

Originally Posted by rohanag

Yes so, that's why there are 2 weights from the input to the hidden layer, and 2 nodes from the hidden layer to the final output node.

Yes yes, now I understood what you meant. The constant unit does not receive connections from back layers.

[rohanag]

Originally Posted by cassio

Yes yes, now I understood what you meant. The constant unit does not receive connections from back layers.

exactly

[cassio]

@mjbeeson, I am sorry for misled you in my first post.

[Yellin]

Originally Posted by mjbeeson

Counting weights. Just to clarify if I am counting right: suppose we have 2 input nodes and one hidden layer with 2 nodes and one output node. So then we have 4 weights, right? According to the problem statement the constant input nodes are counted in the "2". Similarly if every input or hidden layer has an even number of nodes then we must get an even number for the total number of weights. Do you agree? Or do I have an off-by-one error in this count somehow?

There may be some confusion here between "nodes", which are the little circles in the neural net diagrams, and "units", which are the x's that enter and leave the circles. So mibeeson's example may mean there are 2 input units, of which one is constant, with two nodes, each receiving the two input units and producing one unit each. Adding one constant unit to the two units from those two nodes makes for 3 hidden units; the three go to one output node, which produces the output unit. Each node needs weights for each of its incoming units, so there are 4 weights from the first layer of nodes plus 3 from the three units entering the output node. Is this reasonable, or am I the one who is confused?