Related
I've got this segment of code in a discriminator network for MNIST:
nn.Conv2d(1, 64, 4, 2, 1),
From my understanding, there is 1 input channel (the MNIST image), then we apply a 4x4 kernel to the image in strides of 2 to produce 64 feature maps. Does this mean that we actually have 64 kernels at this layer? Because in order to get 64 different feature maps, we would need 64 separate kernels to convolve over the image?
Then after some ReLu, we have another convolution:
nn.Conv2d(64, 128, 4, 2, 1),
How do we get from 64 to 128? From my understanding of the first example, we have 64 seperate kernels that can produce 64 seperate feature maps. But here we go from 64 feature maps to 128 feature maps? Does that mean that we only have two kernels?
I hope someone can shine some light on whether my understanding is correct!
Your understanding in the first example is correct, you have 64 different kernels to produce 64 different feature maps.
In case of the second example, so the number of input channels not beeing one, you still have as "many" kernels as the number of output feature maps (so 128), which each are trained on a linear combination of the input feature maps. So in your case each of these kernels would have 4x4x64 trainable weights.
All the input channels are connected to each output channel (if group = 1, as by default) by convolution with filters (kernels) -- one for each output channel. Each kernel though has sub-kernels for each input channel.
So in the first layer you have in_channels = 1 and out_channels = 64 meaning that there are 64 kernels (and sub-kernels). In the second layer you have in_channels = 64 and out_channels = 128 meaning that there are 128 kernels each having 64 * 128 sub-kernels.
Here's a simple example of one conv layer taken from cs231n for clarification:
And my implementation in Pytorch:
import torch
from torch import nn
cnn = nn.Conv2d(in_channels=3, out_channels=2, kernel_size=3,
stride=2, padding=1, bias=True, groups=1)
w0 = torch.FloatTensor([[[-1, -1, 0],
[ 1, 1, 1],
[ 1, 1, 0]],
[[ 1, 1, -1],
[ 0, 0, 0],
[ 1, 1, -1]],
[[ 0, -1, 0],
[-1, 0, -1],
[ 1, 0, 1]]])
b0 = torch.FloatTensor([1])
w1 = torch.FloatTensor([[[-1, 0, 0],
[ 1, 1, 1],
[-1, -1, 0]],
[[ 1, -1, -1],
[-1, 1, -1],
[ 1, -1, 0]],
[[ 1, -1, 0],
[ 0, 1, 1],
[ 1, 0, 1]]])
b1 = torch.FloatTensor([0])
cnn.weight = torch.nn.Parameter(torch.stack((w0, w1), 0))
cnn.bias = torch.nn.Parameter(torch.cat((b0, b1), 0))
inpt = torch.FloatTensor([[[ 1, 2, 0, 1, 2],
[ 1, 0, 2, 2, 0],
[ 2, 0, 0, 2, 2],
[ 0, 0, 2, 2, 0],
[ 2, 2, 2, 1, 2]],
[[ 2, 0, 0, 1, 1],
[ 1, 0, 2, 1, 2],
[ 2, 0, 2, 2, 1],
[ 0, 2, 0, 0, 1],
[ 1, 2, 1, 2, 0]],
[[ 0, 0, 2, 1, 2],
[ 0, 1, 0, 2, 0],
[ 1, 1, 0, 0, 2],
[ 0, 0, 0, 1, 1],
[ 0, 1, 2, 0, 2]]])
cnn(inpt.unsqueeze(0))
Output:
tensor([[[[ 7., 9., 10.],
[ 0., 6., 10.],
[ 2., 5., 2.]],
[[ 4., 4., 4.],
[ 5., 1., 2.],
[ 2., 6., 0.]]]])
I have a rankingMat which contains the rankings of equity tickers, where every column represents one ticker and every row represents a point in time. The ranking has been performed in a descending manner, i.e. a 1 in rankingMat identifies the highest rank for that time period (read: row). Ticker/column 4 represents a cash security. This will be important further down the road.
Now, I would like to identify in which equity tickers I am supposed to invest. There exists two conditions:
I only invest in tickers that have a rank less or equal to 3.
In addition, I only invest in tickers that are ranked higher or equal compared to cash (ticker/column 4).
I'm able to get a result that fulfills the first condition:
rankingMat = ...
[NaN, NaN, NaN, NaN, NaN, NaN; ...
1, 5, 2, 3, 6, 4; ...
4, 5, 2, 3, 6, 1; ...
4, 1, 2, 5, 6, 3; ...
6, 4, 5, 2, 1, 3; ...
2, 3, 4, 6, 1, 5; ...
3, 6, 4, 1, 2, 5; ...
2, 5, 6, 1, 4, 3];
portfolio = rankingMat <= 3;
The result looks like this:
portfolio = ...
[0, 0, 0, 0, 0, 0; ...
1, 0, 1, 1, 0, 0; ...
0, 0, 1, 1, 0, 1; ...
0, 1, 1, 0, 0, 1; ...
0, 0, 0, 1, 1, 1; ...
1, 1, 0, 0, 1, 0; ...
1, 0, 0, 1, 1, 0; ...
1, 0, 0, 1, 0, 1]
My problem is condition 2. In every row, I need to compare not only if the integer is less or equal to 3, I also need to make sure that it is less than the integer in column 4 in that particular row. I am looking for a solution that avoids a for-loop. I guess it is possible with indexing. So, any hint is highly appreciated.
The final result should look like this:
portfolio = ...
[0, 0, 0, 0, 0, 0; ...
1, 0, 1, 1, 0, 0; ...
0, 0, 1, 1, 0, 1; ...
0, 1, 1, 0, 0, 1; ...
0, 0, 0, 1, 1, 0; ...
1, 1, 0, 0, 1, 0; ...
0, 0, 0, 1, 0, 0; ...
0, 0, 0, 1, 0, 0]
% Prior to R2016b
portfolio = rankingMat <= 3 & ...
bsxfun(#lt, rankingMat, rankingMat(:,4));
% On or after R2016b
portfolio = rankingMat <= 3 & ...
rankingMat < rankingMat(:,4);
So the formula that needs to be implemented is:
P = ((1 - delta)/n) + ((delta)*(A)ij / (Sigma(k=1 to n)(A)ik))
where delta = 0.85
n = 8
and A = the adjacency matrix for web pages being surfed
The mathematica code for it is:
A = {{1, 1, 1, 0, 0, 1, 0, 1}, {0, 0, 1, 0, 0, 1, 0, 1}, {1, 1, 0, 1,
0, 1, 1, 0}, {0, 1, 1, 0, 1, 0, 1, 0}, {1, 1, 1, 1, 0, 1, 1,
1}, {1, 1, 1, 0, 0, 1, 1, 0}, {1, 0, 1, 0, 1, 0, 1, 0}, {0, 0, 0,
0, 1, 0, 0, 1}};
n = 8;
\[Delta] = 0.85;
P = Transpose[Table[(1 - \[Delta])/n + (\[Delta]*A[[i, j]])/(Sum[A[[i, k]], {k, 1, n}]), {i, 1, n}, {j, 1, n}]];
Everything else is just plugging in numbers.
Now the main problem I seem to have is getting the A[[i,j]]/Sum[A[i,k]] to work on matlab.
On matlab: when I input A[[i,j]] as A, and sum[A[i,k]] as either (sum(A,2))' or sum(A,1), the P that gets output on matlab becomes a column vector rather than an 8 x 8 matrix.
What am I missing?
There are many ways of doing it. I'll show you one way of using native vectorized MATLAB code, so no need for for-loops or arrayfun or anything like that.
A = [1, 1, 1, 0, 0, 1, 0, 1; 0, 0, 1, 0, 0, 1, 0, 1; 1, 1, 0, 1, 0, 1, 1, 0; 0, 1, 1, 0, 1, 0, 1, 0; 1, 1, 1, 1, 0, 1, 1, 1; 1, 1, 1, 0, 0, 1, 1, 0; 1, 0, 1, 0, 1, 0, 1, 0; 0, 0, 0, 0, 1, 0, 0, 1];
n = size(A, 1);
delta = 0.85;
% First, sum up all columns in each row
rowSums = sum(A,2);
% Then replicate them because the entries in each row of the vector are valid for every column in that specific row. So replicate them so that the output-matrix matches the size of A so we can work element-wise division later
rowSums2 = repmat(rowSums, 1, n);
% Use vectorized code, ./ will yield element-wise division as A and rowSums2 are of the same size now
P = (1 - delta)/n + delta*A ./ rowSums2
I hope I got the desired output right.
the problem:
>> syms('a', 'b', 'c');
>>A =
[ -2, 3, 1, a]
[ 1, 1, -1, b]
[ 0, 5, -1, c]
>>rref(A)
ans =
[ 1, 0, -4/5, 0]
[ 0, 1, -1/5, 0]
[ 0, 0, 0, 1]
The problem I'm having is that I need the answer to be in terms of a b and c, so I need something like:
ans =
[1, 0, 0, a+2b-c]
[0, 1, 0, 3a-c]
[0, 0, 1, a+b+c]
Is there any way to get Matlab to accomplish this?
You just need to convert the problem to a relevant form in order to use rref properly, that is, instead of :
[1, 0, 0, a+2b-c]
[0, 1, 0, 3a-c]
[0, 0, 1, a+b+c]
where the x,y,z form the unity matrix block,
write:
syms x y z
D =
[ 1, 2, -1, x]
[ 3, 0, -1, y]
[ 1, 1, 1, z]
rref(D)
ans =
[ 1, 0, 0, (3*y)/10 - x/10 + z/5]
[ 0, 1, 0, (2*x)/5 - y/5 + z/5]
[ 0, 0, 1, (3*z)/5 - y/10 - (3*x)/10]
Matlab actually gave the correct answer for your example, which may seem surprising as it does not depend on a, b or c; if you use another example, e.g.
B = [1, 2, a; 2, 1, b];
rref(B)
then the answer will depend on a and b as expected.
Here I just explain why the reduced row echelon form of your A is indeed
[ 1, 0, -4/5, 0]
[ 0, 1, -1/5, 0]
[ 0, 0, 0, 1]
by showing some of the important steps of the row reduction. Matlab starts with A, and reduces it to
[ 1, 0, -4/5, (3*b)/5 - a/5]
[ 0, 1, -1/5, a/5 + (2*b)/5]
[ 0, 0, 0, a + 2*b - c]
it then divides the last row by (a + 2*b - c) to get
[ 1, 0, -4/5, (3*b)/5 - a/5]
[ 0, 1, -1/5, a/5 + (2*b)/5]
[ 0, 0, 0, 1]
and then it proceeds to subtract that row from the rows above it to give
[ 1, 0, -4/5, 0]
[ 0, 1, -1/5, 0]
[ 0, 0, 0, 1]
Matlab assumed that a + 2*b - c is not equal to zero (try replacing c with a + 2*b, then rref(A) will give you a different answer). I know of no way to force Matlab not to divide by a + 2*b - c, i.e. to make Matlab assume that a + 2*b + c may be equal to zero.
You want the elimination matrix that gives you rref(A).
But What is rref(A)?
UpperTri=rref(A)=[LastStep]...[Step3][Step2][Step1]*A.
That is a series of steps that reduces A to an upper triangle or the best one possible.
Matlab had,
[ 1, 0, -4/5, (3*b)/5 - a/5]
[ 0, 1, -1/5, a/5 + (2*b)/5]
[ 0, 0, 0, 1]
Then Matlab just rolled it up and you no longer have the steps. So it appears as:
[ 1, 0, -4/5, 0]
[ 0, 1, -1/5, 0]
[ 0, 0, 0, 1]
If you want the series of steps you want the elimination matrix with the series of steps so recorded it using an Identity matrix. Instead of:
A =
[ -2, 3, 1, a]
[ 1, 1, -1, b]
[ 0, 5, -1, c]
Use
>> A = [ -2, 3, 1, 1 0 0; 1, 1, -1, 0 1 0; 0, 5, -1, 0 0 1]
A =
-2 3 1 1 0 0
1 1 -1 0 1 0
0 5 -1 0 0 1
>> E=rref(A)
E =
1.0000 0 -0.8000 0 1.0000 -0.2000
0 1.0000 -0.2000 0 0 0.2000
0 0 0 1.0000 2.0000 -1.0000
This is like [A][I], Now [rref Steps matrix]*[A][I]=[rref(A)][rref Steps matrix]
Now E=[rref(A)][rref Steps matrix] =
E =
1.0000 0 -0.8000 0 1.0000 -0.2000
0 1.0000 -0.2000 0 0 0.2000
0 0 0 1.0000 2.0000 -1.0000
By inspection E1=[rref Steps matrix]=
E1 =
0 1.0000 -0.2000
0 0 0.2000
1.0000 2.0000 -1.0000
so now:
E1 =
0 1.0000 -0.2000
0 0 0.2000
1.0000 2.0000 -1.0000
>> B=[a;b;c;]
B =
a
b
c
>> B1=E1*B
B1 =
b - c/5
c/5
a + 2*b - c
To check:
A1 = [ -2, 3, 1, a; 1, 1, -1, b; 0, 5, -1, c]
A1 =
[ -2, 3, 1, a]
[ 1, 1, -1, b]
[ 0, 5, -1, c]
>> A2=E1*A1
A2 =
[ 1, 0, -4/5, b - c/5]
[ 0, 1, -1/5, c/5]
[ 0, 0, 0, a + 2*b - c]
Note a + 2b - c =0, therefore c=a+2b, therefore sub c, (c/5)= ((a+2b)/5), and (b-c/5)=(5b-(a+2b))/5
therefore
[(-a+3*b)/5; (a+2*b)/5; a + 2*b - c ]
I was trying to solve a matrix with parameters in matlab. So i defined a and b with syms a b and then my matrix D = [-1 -2 2 1 -3; 1 1 a 1 2; 2 1 -1 b 3; 1 1 -1 1 2; -1 1 a -1-b 0]
D =
[ -1, -2, 2, 1, -3]
[ 1, 1, a, 1, 2]
[ 2, 1, -1, b, 3]
[ 1, 1, -1, 1, 2]
[ -1, 1, a, - b - 1, 0]
When i do rref(D) it gives me the following:
>> rref(D)
ans =
[ 1, 0, 0, 0, 1]
[ 0, 1, 0, 0, 1]
[ 0, 0, 1, 0, 0]
[ 0, 0, 0, 1, 0]
[ 0, 0, 0, 0, 0]
This actually useless for me, because i need the system solved with the resulting parameters in it.
Anyone an idea? Thanks in advance!
well, it is pretty obvious that both a and b can be eliminated using rref, since a appears twice only in the 2nd col, and b appears twice only in the 4th col. For example, (row2+row3)/(row5+row1) and bingo, no a and b ...