Say, I have a PyTorch 2x2 tensor, and I also generated a boolean tensor of the same dimension (2x2). I want to use this as a mask.
For example, if I have a tensor:
tensor([[1, 3],
[4, 7]])
And if my mask is:
tensor([[ True, False],
[False, True]])
I want to use that mask to get a tensor where elements corresponding to True from my original tensor are retained, whereas elements corresponding to False are set to zero in the output tensor.
Expected Output:
tensor([[1, 0],
[0, 7]])
Any help is appreciated. Thanks!
Assume you have :
t = torch.Tensor([[1,2], [3,4]])
mask = torch.Tensor([[True,False], [False,True]])
You can use the mask by:
masked_t = t * mask
and the output will be:
tensor([[1., 0.],
[0., 4.]])
Related
I am randomly rotating 3D images in pytorch using torch.rot90 but this rotates all the images in the batch in the same way. I would like to find a differentiable way to randomly rotate each image in a different axis.
here is the code which rotates each image to the same orientation:
#x = next batch
k = torch.randint(0, 4, (1,)).item()
dims = [0,0]
dims[0] = dims[1] = torch.randint(2, 5, (1,))
while dims[0] == dims[1]:#this makes sure the two axes aren't the same
dims[1] = torch.randint(2, 5, (1,))
x = torch.rot90(x, k, dims)
# x is now a batch of 3D images that have all been rotated in the same random orientation
You could split the data in the batch randomly into 3 subsets, and apply each dimensional rotation respectively.
Let me expand on iacob's answer. Firstly, let me go over the parameters of rot90 function. Other than the input tensor, it expects k and dims where k is the number of rotations to be done, and dims is a list or tuple containing two dimensions on how the tensor to be rotated. If a tensor is 4D for example, dims could be [0, 3] or (1,2) or [2,3] etc. They have to be valid axes and it should contain two numbers. You don't really need to create tensors for this parameter or k. It is important to note that, depending on the given dims, output shape can drastically change:
x = torch.rand(15, 3, 4,6)
y1 = torch.rot90(x[0:5], 1, [1,3])
y2 = torch.rot90(x[5:10], 1, [1,2])
y3 = torch.rot90(x[10:15], 1, [2,3])
print(y1.shape) # torch.Size([5, 6, 4, 3])
print(y2.shape) # torch.Size([5, 4, 3, 6])
print(y3.shape) # torch.Size([5, 3, 6, 4])
Similar to iacob's answer, here we apply 3 different rotations to slices of the input. Note that how the output dimensions are all different, due to nature of rotations over different dimensions. You can't really join these results into one tensor, unless you have a really specific input size, for example Batch x 10 x 10 x 10 where rotating over combinations of 1,2,3 axes will always return same dimensions. You can however use each of these different sized output separately as inputs to different modules, layers etc.
I personally can't think of a use case where your random axes rotation can be used. If you can elaborate on why you are trying to do this, I can try to give some better solutions.
I'm trying to identify overlapping/intersecting polygons. The techniques I have found only compare two polygons at a time. I have tens-of-thousands of cells in a dataset, and in each one there are 2-20 polygons, each described by x-y coordinates. I want to find the overlapping polygons in each cell. Looping between every pair to check for an intersection is very slow, so I want to ask...
Is there a way to compare all polygons at the same time and extract the IDs of those that are overlapping?
Here is a simple example of a single entry from the dataset:
shapes = cell(4,2);
shapes{1,1} = 'poly1';
shapes{2,1} = 'poly2';
shapes{3,1} = 'poly3';
shapes{4,1} = 'poly4';
shapes{1,2} = [1, 3, 3; 1, 1, 3]';
shapes{2,2} = [2, 4, 2; 2, 2, 5]';
shapes{3,2} = [4, 5, 5, 4; 3, 3, 5, 5]';
shapes{4,2} = [1, 3, 3, 1; 4, 4, 6, 6]';
This example contains these 4 polygons:
This plot was made with separate 'polyshape' objects, but that doesn't mean I need to use this kind of object in the solution.
The output I would like is a record of each overlapping pair:
result =
2×2 cell array
{'poly1'} {'poly2'}
{'poly2'} {'poly4'}
P.S. My current method is to loop through each pair and use the poly2mask function on each polygon of the pair. Then use the & operator to add the binary masks together. This produces a logical array of 1's where there is any overlap.
P.P.S. The actual polygons I am looking at are all annular sectors, therefore they are not all convex
Here is a solution that makes use of 'polyshape' vectors and avoids making all those pairwise comparisons in extra loops (although I don't know how the 'overlap' function works).
% Set up empty vector to hold the different shapes
polyvec = [];
% Loop all shapes and combine into polyshape vector
for ii = 1 : size(shapes, 1)
poly = polyshape(shapes{ii,2}(:,1), shapes{ii,2}(:,2));
% When you combine polyshape objects together the you get
% a vector that is of the polyshape object type
polyvec = [polyvec, poly];
end
% Use the overlap function to compute a symmetric binary matrix
% of which polygons in the polygon vector overlap.
interMatSym = overlaps(polyvec);
% I only need the upper triangle of the symmetric interaction
% matrix and all polygons overlap with themselves so use 'triu'
interMat = triu(overlaps(polyvec), 1);
% Find the coordinates of the overlap in the interaction matrix
[x, y] = find(interMat);
% Save the result
result = [shapes(x,1), shapes(y,1)];
result =
2×2 cell array
{'poly1'} {'poly2'}
{'poly2'} {'poly4'}
If there is a way to create a polyshpae vector any more efficiently then I'd love to know!
I tried to padarray more than 1000 images. However when I time my code, this specific line take the highest amount of time to complete
I=abs(padarray(I, [2, 2], 'replicate', 'both'));
Mainly because of the line 35 of the padarray algorithm (inside profiler): images\private\padarray_algo
b = a(aIdx{:});
Any way to improve the efficiency? Perhaps using another method? Thanks!
You can use repmat and matrix concatenation to get the same result:
r=#repmat;
pad=#(I,d)[r(I(1),d) r(I(1,:),d(1),1) r(I(1,end),d)
r(I(:,1),1,d(2)) I r(I(:,end),1,d(2))
r(I(end,1),d) r(I(end,:),d(1),1) r(I(end),d)];
Usage:
pad(I,[2 2])
If all images are of the same size you can create a matrix of linear indices of the image and apply padarray to it then use padded index array to pad images:
%create matrix of indices
Idx = reshape(1:numel(I),size(I));
%pad the index
Idx_padded = padarray(Idx, [2, 2], 'replicate', 'both');
%use the padded index to pad images
result = I(Idx_padded);
result2 = I2(Idx_padded);
Assume that we have a Convolutional Neural Network trained to classify (w.l.o.g. grayscale) images, in Tensor-Flow.
Given the trained net and a test image one can trace which pixels of it are salient, or "equivalently" which pixels are most responsible for the output classification of the image. A nice, explanation and implementation details in Theano, are given in this article.
Assume that for the first layer of convolutions that is directly linked with the input image, we do have the gradient for the parameters of every convolutional kernel-wrt. the classification function.
How can one propagate the gradient back to the Input layer, so to compute a partial derivative on every pixel of the image?
Propagating and accumulating back the gradient, would give us the salient pixels (they are those with big in-magnitude derivative).
To find the gradient wrt. the kernels of the first layer, so far I did:
Replaced the usual loss operator with the output layer operator.
Used the "compute_gradient" function,
All in all, it looks like:
opt = tf.train.GradientDescentOptimizer(1)
grads = opt.compute_gradients(output)
grad_var = [(grad1) for grad in grads]
g1 = sess.run([grad_var[0]])
Where, the "output" is the max of the output layer of the NN.
And g1, is a (k, k, 1, M) tensor, since I used M: k x k convolutional kernels on the first layer.
Now, I need to find the correct way to propagate g1 on every input pixel, as to compute their derivative wrt. the output.
To compute the gradients, you don't need to use an optimizer, and you can directly use tf.gradients.
With this function, you can directly compute the gradient of output with respect to the image input, whereas the optimizer compute_gradients method can only compute gradients with respect to Variables.
The other advantage of tf.gradients is that you can specify the gradients of the output you want to backpropagate.
So here is how to get the gradients of an input image with respect to output[1, 1]:
we have to set the output gradients to 0 everywhere except at indice [1, 1]
input = tf.ones([1, 4, 4, 1])
filter = tf.ones([3, 3, 1, 1])
output = tf.nn.conv2d(input, filter, [1, 1, 1, 1], 'SAME')
grad_output = np.zeros((1, 4, 4, 1), dtype=np.float32)
grad_output[0, 1, 1, 0] = 1.
grads = tf.gradients(output, input, grad_output)
sess = tf.Session()
print sess.run(grads[0]).reshape((4, 4))
# prints [[ 1. 1. 1. 0.]
# [ 1. 1. 1. 0.]
# [ 1. 1. 1. 0.]
# [ 0. 0. 0. 0.]]
Supposing I have the linear indices of the pixels which I would like to 'color in'.
If I wanted to set all of those pixels to some specific value in my grayscale image sequence, I could do this:
gryscl_imSeq(LinearPixIndx) = 0.7;
edit: where 'LinearPixIndx' is a p-element vector containing the linear indices of the pixels in the image sequence.
If however, I want to introduce some color, I would first need to convert my grayscale image sequence into an m x n x 3 x p matrix, which I could do like this:
RGBvideo = reshape(repmat(reshape(gryscl_imSeq,[],size(gryscl_imSeq,3)),3,1),...
[size(gryscl_imSeq,1) size(gryscl_imSeq,2) 3 size(gryscl_imSeq,3)]);
If somebody could tell me how to convert my grayscale linear pixels indices into corresponding indices for the RGBvideo I would really appreciate it.
Or if there is a better way to go about what I'm trying to do, I would be very grateful for any suggestions.
Thanks in Advance,
N
The question isn't really clear, but I guess I understand what you want. Here's one way to do it.
First I think there's a slightly more elegant way to convert from grayscale to RGB:
% determine image sequence dimensions
[m, n, N] = size(gryscl_imSeq);
% expand to RGB
rgb_imSeq = reshape(gryscl_imSeq, [m, n, 1, N]);
rgb_imSeq = repmat(rgb_imSeq, [1, 1, 3, 1]);
After that, in order to be able to apply your linear indices to all color channels and all frames, you need to explicitly linearize the pixel index by reshaping. Then you can color the pixels, and transform back into the form width x height x color channels x frames:
% linearize image dimensions
rgb_imSeq = reshape(rgb_imSeq, [m * n, 3, N]);
% color pixels
rgb_imSeq(LinearPixIndx, :, :) = repmat([1 0 0], [numel(LinearPixIndx), 1, N]);
% original shape
rgb_imSeq = reshape(rgb_imSeq, [m, n, 3, N]);
Here [1 0 0] is the RGB representation of the color you want to use, in this case red.