Number of layers vs list(net.parameters()) - neural-network

Question

New to convolutional neural nets so sorry if this doesn't make much sense. I have this code:
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(3, 6, 5)
self.pool = nn.MaxPool2d(2, 2)
self.conv2 = nn.Conv2d(6, 16, 5)
self.fc1 = nn.Linear(16 * 5 * 5, 120)
self.fc2 = nn.Linear(120, 84)
self.fc3 = nn.Linear(84, 10)
def forward(self, x):
x = self.pool(F.relu(self.conv1(x)))
x = self.pool(F.relu(self.conv2(x)))
x = x.view(-1, 16 * 5 * 5)
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = self.fc3(x)
return x
net = Net()
and I think this has 5 layers. However, when I print len((list(net.parameters())) I get 10. Shouldn't this be a list of size 5 with parameters for each layer?

Answer 1

Quick answer : You get an extra parameter array for each layer containing the bias vector associated to the layer.
Detailled answer:
I will try to guide you in my process of investigating your questions.
It seems like a good idea to see what our 10 parameters are :
for param in net.parameters():
print(type(param), param.size())
<class 'torch.nn.parameter.Parameter'> torch.Size([6, 3, 5, 5])
<class 'torch.nn.parameter.Parameter'> torch.Size([6])
<class 'torch.nn.parameter.Parameter'> torch.Size([16, 6, 5, 5])
<class 'torch.nn.parameter.Parameter'> torch.Size([16])
<class 'torch.nn.parameter.Parameter'> torch.Size([120, 400])
<class 'torch.nn.parameter.Parameter'> torch.Size([120])
<class 'torch.nn.parameter.Parameter'> torch.Size([84, 120])
<class 'torch.nn.parameter.Parameter'> torch.Size([84])
<class 'torch.nn.parameter.Parameter'> torch.Size([10, 84])
<class 'torch.nn.parameter.Parameter'> torch.Size([10])
We can recognize our 5 layers and an extra line for each layer. For instance, if we look at a specific layer for instance the first conv layer, we get :
for param in net.conv1.parameters():
print(type(param), param.size())
<class 'torch.nn.parameter.Parameter'> torch.Size([6, 3, 5, 5])
<class 'torch.nn.parameter.Parameter'> torch.Size([6])
So now that we know we have two array of parameters per layer, the question is why. The first 6*3*5*5 array corresponds to your 6 kernels of size 5*5 with 3 channels, the second one corresponds the the bias associated to each of your kernel. Mathematically speaking, to compute the value at the next layer associated to a given kernel, you make the convolution between the area under your desired pixel and the kernel and you add a real number. That number is called the bias and it is empirically proven that using a bias gives better results.
Now you can also create a layer without bias, and then you will only get one parameter array :
layer = nn.Conv2d(3,6,5, bias= False)
for param in layer.parameters():
print(type(param), param.size())
<class 'torch.nn.parameter.Parameter'> torch.Size([6, 3, 5, 5])