This is the boolean function I try to fit.
[boolean function description][1]
Theoretically we need a neural network with 1 hidden layer which has 3 neurons at least.
And that's actually how I built the neural network in Pytorch.
However, despite the prediction of NN usually correct, the parameters (I mean the weights and bias) not as expected.
I expect the parameters to be like this way(A perceptron operation is equivalent to a Boolean gate):
[perceptron equivalent to a boolean gate][2]
Here the key code:
class NeuralNetwork(nn.Module):
def __init__(self):
super(NeuralNetwork, self).__init__()
self.linear_relu_stack = nn.Sequential(
nn.Linear(4, 3),
nn.ReLU(),
nn.Linear(3, 1),
)
def forward(self, x):
logits = self.linear_relu_stack(x)
return logits
base_lr = 0.001
optimizer = torch.optim.Adam(model.parameters(), base_lr)
criterion = nn.MSELoss().to(device)
Here is the key output:
First, the prediction not so bad, The categories are correct, but some of the numbers are not precise enough
w x y z pred
0 0 0 0 0 [tensor(0.9992, grad_fn=<UnbindBackward0>)]
1 0 0 0 1 [tensor(0.2459, grad_fn=<UnbindBackward0>)]
2 0 0 1 0 [tensor(0.9992, grad_fn=<UnbindBackward0>)]
3 0 0 1 1 [tensor(0.0040, grad_fn=<UnbindBackward0>)]
4 0 1 0 0 [tensor(0.9992, grad_fn=<UnbindBackward0>)]
5 0 1 0 1 [tensor(0.7707, grad_fn=<UnbindBackward0>)]
6 0 1 1 0 [tensor(-0.0015, grad_fn=<UnbindBackward0>)]
7 0 1 1 1 [tensor(-0.0025, grad_fn=<UnbindBackward0>)]
8 1 0 0 0 [tensor(0.9992, grad_fn=<UnbindBackward0>)]
9 1 0 0 1 [tensor(-0.2525, grad_fn=<UnbindBackward0>)]
10 1 0 1 0 [tensor(0.9992, grad_fn=<UnbindBackward0>)]
11 1 0 1 1 [tensor(-0.0077, grad_fn=<UnbindBackward0>)]
12 1 1 0 0 [tensor(0.9992, grad_fn=<UnbindBackward0>)]
13 1 1 0 1 [tensor(0.2722, grad_fn=<UnbindBackward0>)]
14 1 1 1 0 [tensor(-0.0066, grad_fn=<UnbindBackward0>)]
15 1 1 1 1 [tensor(0.0033, grad_fn=<UnbindBackward0>)]
Second, the parameters are not as expected.
linear_relu_stack.0.weight tensor([[-0.3637, 0.3838, 0.7624, 0.3661],
[ 0.2857, 0.5719, 0.5721, -0.5846],
[ 0.4782, -0.5035, -0.2349, 1.2070]])
linear_relu_stack.0.bias tensor([-0.7657, -0.8599, -0.4842])
linear_relu_stack.2.weight tensor([[-1.3418, -1.7255, -1.0422]])
linear_relu_stack.2.bias tensor([0.9992])
My question is why the NN doesn't convege to my expected position?
What' the problem?
[1]: https://i.stack.imgur.com/WqaXi.png
[2]: https://i.stack.imgur.com/Z9cQb.png
Generally you can't expect a neural network to solve a problem the same way you do.
Firstly, the network starts from a random state and proceeds by optimizing that state (e.g. you have used Adam in your code) which means there is no guarantee of what state will the code eventually land in.
Secondly, if your results are [as you stated in your question] more or less correct, then your network has found a reduction for your function, it's just that this reduction might not make sense to you, specially if you're trying to understand it in terms of logical functions which is not very close to how Neural Networks works.
Not being interpretable is a known down-side of Neural Networks and this is case is no exception.
Related
Say that we have the following random matrix:
1 2 3 4
5 6 7 8
9 8 7 6
5 4 3 2
I'd like to transform it into the following:
1 0 2 0 3 0 4 0
0 0 0 0 0 0 0 0
5 0 6 0 7 0 8 0
0 0 0 0 0 0 0 0
9 0 8 0 7 0 6 0
0 0 0 0 0 0 0 0
5 0 4 0 3 0 2 0
0 0 0 0 0 0 0 0
For some reason I cannot use mathjax format so it looks a bit awful, sorry for this. Point, is, that I want to add row and columns of zeros in between of my current rows and columns so that I increase its size 2x.
I came up with the following code, but it only works for very small matrixes if i use it on a a big image it cannot finish due to memory limitation problems.
clear all
I=imread('image.png');
I=rgb2gray(I);
B=zeros(2*size(I));
[x, y]=find(-inf<I<inf);
xy=[x,y];
nxy=xy;
%coord change
nxy=2*xy-1;
B(nxy(:,1),nxy(:,2))=I(xy(:,1),xy(:,2));
I expected to be fast because it is fully vectorised with maltlab functions but it fails miserably. Is there some other elegant way to do this?
If you take a look at your indexing vectors, this is something like I([1 1 2 2] ,[1 2 1 2] ); for a 2x2 matrix which means you index each row and column twice. The right solution is B(1:2:end,1:2:end)=I; which indexes every second row and every second column.
This can be also done via the one liner, say your original matrix is called A, then
kron(A,[1,0;0,0])
I used selforgmap for pattern recognition. After training finished i calculated the network's output of the whole data and I got a logical matrix.
I want know how selforgmap:
1- numbers the neurons (i mean from 1 to N, while N equals the total number of neurons)
2-
Here is my map
1 O------O
/ /
0 O------O
0 0.5 1 1.5
the output looks like this (after transpose)
1 0 0 0
0 1 0 0
1 0 0 0
1 0 0 0
0 0 1 0
0 1 0 0
0 0 1 0
0 0 1 0
i want know which column in output corresponds to which neuron of the map
Selforgmap in MATLAB starts the numbering from the bottom left. For your example, the neurons are labeled:
3 - 4
/ /
1 2
You can use the
vec2ind(output)
command to associate the output with the neuron to which the corresponding input has been assigned.
Using MATLAB, I have a matrix such as:
1 1 0
1 0 1
1 1 1
The aim is to represent the zero's as a mine in a minesweeper program and the values around the 0's should reflect how many mines are adjacent to it.
Therefore creating a vector like this:
1 2 0
1 0 2
1 1 1
I have thought to take elements around the zero as a sub matrix and then add 1, but then it will turn 0's into 1's.
How would I program such a task?
I think this can be achieved by simple convolution plus some post-processing on the resultant matrix as follows:
% Defining a 6x6 matrix of zeros and ones
mineMat=randi(2,6,6)-1;
numberOfMines=conv2(double(~mineMat),ones(3,3),'same').*mineMat;
% Result:
mineMat=
1 0 1 1 0 0
0 0 0 1 0 0
1 1 1 1 1 0
1 1 1 1 0 1
0 1 0 0 0 0
0 1 1 0 0 0
numberOfMines=
3 0 3 3 0 0
0 0 0 3 0 0
2 3 2 3 4 0
1 2 2 4 0 4
0 3 0 0 0 0
0 3 3 0 0 0
Parag's answer would be my first option. Another approach is to use blockproc (Image Processing Toolbox):
blockproc(~M, [1 1], #(x)sum(x.data(:)), 'Bordersize', [1 1], 'TrimBorder', 0).*M
Sounds like you are looking to apply a (two dimensional) filter:
M = [1 1 0; 1 0 1; 1 1 1]==0;
F = filter2(ones(3),M);
F(M)=0
The middle line basically does the work (applying the filter) to create the count. The last line ensures that the mines stay at value 0.
The question is about feature detection concept.
I'm stuck after I finding the corner of image and I want to know how to finding the feature point within the computed corners.
Suppose I have grayscale image that have data like this
A = [ 1 1 1 1 1 1 1 1;
1 3 3 3 1 1 4 1;
1 3 5 3 1 4 4 4;
1 3 3 3 1 4 4 4;
1 1 1 1 1 4 6 4;
1 1 1 1 1 4 4 4]
if I use
B = imregionalmax(A);
the result would be like this
B = [ 0 0 0 0 0 0 0 0;
0 1 1 1 0 0 1 0;
0 1 1 1 0 1 1 1;
0 1 1 1 0 1 1 1;
0 0 0 0 0 1 1 1;
0 0 0 0 0 1 1 1]
The question is how do I pick the highest peak inside max local region (in sample how did I chose 5 from 3 and 6 from 4)?
My idea was using B to detect each region and use imregionalmax() again but I'm not good at coding and I need some advice or other ideas.
There are a couple of other easy ways to implement a 2D peak finder: ordfilt2 or imdilate.
ordfilt2
The most direct method is to use ordfilt2, which sorts values in local neighborhoods and picks the n-th value. (The MathWorks example demonstrates how to implemented a max filter.) You can also implement a 3x3 peak finder with ordfilt2 by, (1) using a 3x3 domain that does not include the center pixel, (2) selecting the largest (8th) value and (3) comparing to the center value:
>> mask = ones(3); mask(5) = 0 % 3x3 max
mask =
1 1 1
1 0 1
1 1 1
There are 8 values considered in this mask, so the 8-th value is the max. The filter output:
>> B = ordfilt2(A,8,mask)
B =
3 3 3 3 3 4 4 4
3 5 5 5 4 4 4 4
3 5 3 5 4 4 4 4
3 5 5 5 4 6 6 6
3 3 3 3 4 6 4 6
1 1 1 1 4 6 6 6
The trick is compare this to A, the center value of each neighborhood:
>> peaks = A > B
peaks =
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0
imdilate
Image dilation is usually done on binary images, but grayscale image dilation is simply a max filter (see Definitions section of imdilate docs). The same trick used with ordfilt2 applies here: define a neighborhood that does not include the center neighborhood pixel, apply the filter and compare to the unfiltered image:
B = imdilate(A, mask);
peaks = A > B;
NOTE: These methods only find a single pixel peak. If any neighbors have the same value, it will not be a peak.
The function imregionalmax gives you the 8-connected region containing the maximum and its 8 neighbours (i.e. the 3x3-regions you are seeing). You could then use morphological operations with the same 3x3 structural element to thin out those regions to their centers. E.g.
B = imregionalmax(A);
C = imerode(B, ones(3));
or equivalently
B = imregionalmax(A);
D = bwmorph(B, 'erode');
Alternatively you could write your own maximum finding function using block-processing:
fun = #(block) % your code working on 'block' goes here ...
B = blockproc(A, ones(3), fun)
But most likely this will be slower than the built-in functions.
(I don't have the toolbox available right now, so I can't try that out.)
Also have a look here and here.
I am working with MATLAB, and I have an adjacency matrix:
mat =
0 1 0 0 0 0
1 0 0 0 1 0
0 0 0 1 0 0
0 0 1 0 0 1
0 1 0 0 0 0
0 0 0 1 0 0
which is not fully connected. Nodes {1,2,5} are connected, and {3,4,6} are connected (the edges are directed).
I would like to see the separate clusters in a dendrogram on a single plot. Since there is not path from one cluster to the next, I would like to see separate trees with separate roots for each cluster. I am using the commands:
mat=zeros(6,6)
mat(1,2)=1;mat(2,1)=1;mat(5,2)=1;mat(2,5)=1;
mat(6,4)=1;mat(4,6)=1;mat(3,4)=1;mat(4,3)=1;
Y=pdist(mat)
squareform(Y)
Z=linkage(Y)
figure()
dendrogram(Z)
These commands are advised from Hierarchical Clustering. And the result is attached: imageDendrogram. Other than that the labels don't make sense, the whole tree is connected, and I connect figure out how to have several disconnected trees which reflect the disconnected nature of the data. I would like to avoid multiple plots as I wish to work with larger datasets that may have many disjoint clusters.
I see this was asked a while ago, but in case you're still interested, here's something to try:
First extract the values above the diagonal from the adjacency matrix, like so:
>> matY = [];
>> for n = 2:6
for m = n:6
matY = [matY mat(n,m)];
end
end
>> matY
matY =
Columns 1 through 13
0 0 0 1 0 0 1 0 0 0 0 1 0
Columns 14 through 15
0 0
Now you have something that looks like the Y vector pdist would have produced. But the values here are the opposite of what you probably want; the unconnected vertices have a "distance" of zero and the connected ones are one apart. Let's fix that:
>> matY(matY == 0) = 10
matY =
Columns 1 through 13
10 10 10 1 10 10 1 10 10 10 10 1 10
Columns 14 through 15
10 10
Better. Now we can compute a regular cluster tree, which will represent connected vertices as "close together" and non-connected ones as "far apart":
>> linkage(matY)
ans =
3 6 1
1 5 1
2 4 1
7 8 10
9 10 10
>> dendrogram(ans)
The resulting diagram:
Hope this is a decent approximation of what you're looking for.