I have two matrices, A (dimensions M x N) and B (N x P). In fact, they are collections of vectors - row vectors in A, column vectors in B. I want to get cosine similarity scores for every pair a and b, where a is a vector (row) from matrix A and b is a vector (column) from matrix B.
I have started by multiplying the matrices, which results in matrix C (dimensions M x P).
C = A*B
However, to obtain cosine similarity scores, I need to divide each value C(i,j) by the norm of the two corresponding vectors. Could you suggest the easiest way to do this in Matlab?
The simplest solution would be computing the norms first using element-wise multiplication and summation along the desired dimensions:
normA = sqrt(sum(A .^ 2, 2));
normB = sqrt(sum(B .^ 2, 1));
normA and normB are now a column vector and row vector, respectively. To divide corresponding elements in A * B by normA and normB, use bsxfun like so:
C = bsxfun(#rdivide, bsxfun(#rdivide, A * B, normA), normB);
You can use scipy to compute it very easily.
from scipy.spatial import distance
cosine_sim = 1 - sp.distance.cdist(A, B, 'cosine')
All you need to do is pass your 2D matrices in above formula and spicy will return you numpy array.
Refer doc here: https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.cdist.html
Related
I have an expression such as s=aU+bV, where a and b are scalars and U and V 3-component vectors. The output s is clearly a 3-component vector. Let's assume I want to plot the first component of s find out how this varies when I change a and b.
In order to plot I have to use surf, which takes matrices for the variables a b. So I attempt to create matrices with meshgrid:
A=0:10;
B=1:10;
[a,b]=meshgrid(A,B);
U=[1,1,0];
V=[1,0,1];
s = a*U + b*V;
This clearly doesn't work, because nor the matrix product nor the element-wise product are well defined in this case. How do I actually make the matrices which represent the grid a b multiply element-by-element the vectors U and V?
You want to use element-wise multiplication (.*) because you still want to treat a and b as scalars (i.e. use each element individually).
You can make a 3D output, where each 2D slice corresponds to your meshgrid output, with one slice per component of U and V. Therefore in this example getting a 10*11*3 matrix.
To do this, just reshape the U and V vectors to be 1*1*3 in size
U = reshape( [1,1,0], 1, 1, [] ); % Or equivalently U(1,1,:) = [1,1,0]
V = reshape( [1,0,1], 1, 1, [] ); % Or equivalently U(1,1,:) = [1,0,1]
Then do element-wise multiplication
s = a.*U + b.*V;
Note: before MATLAB R2016b (when implicit expansion was introduced) you'll have to use bsxfun to get the equivalent:
s = bsxfun( #times, a, U ) + bsxfun( #times, b, V );
You can then plot the ith element of S changing with A and B by plotting s(:,:,i).
You could do it using a 3D matrix:
[A,B] = meshgrid(0:10,1:10);
U(1,1,:) = [1,1,0];
V(1,1,:) = [1,0,1];
s = A.*U + B.*V;
% s is now a NxMx3 matrix, where N = length(A) and M = length(B)
% We can plot how s varies with a and b as follows
surf(A,B,s(:,:,1)); % first component
surf(A,B,s(:,:,2)); % second component
surf(A,B,s(:,:,3)); % third component
I try to figure out the best way to perform a kind of convolution.
I have a 3D matrix I = [N x M x P] and a 2D matrix S = [1 x 1 x K x P]. For each pth frame (third dimension) of my 3D matrix I want to return the valid convolution between I(:, :, p-K/2:p+K/2) and S(1, 1, :, p). Do you see a way to do this ?
In fact, in terms of computation the numbers of operation in very close to a standard convolution, the difference is that I need to change the second matrix for each frame...
This is the method I currently use:
% I = 3D matrix [N x M x P]
% S = Filter [1 x 1 x K x P] (K is an odd number)
% OUT = Result
[N, M, P] = size(I); % Data size
K = size(S, 3); % Filter length
win = (K-1)/2 ; % Window
OUT = zeros(size(I)); % Pre-allocation
for p = win+1:P-win
OUT(:, :, p) = convn(I(:, :, p-win:p+win), S(1, 1, :, p), 'valid'); % Perform convolution
end
At the end we have the same number of operations than the standard convolution, the only difference is that the filter is different for each frame...
Any idea ?
Thanks ;)
So you want to convolve a NxMxK sub-image with a 1x1xKx1 kernel, and then only take the valid part, which is an NxM image.
Let's look at this operation for a single (x,y) location. This 1D convolution, of which you only keep 1 value, is equivalent to the dot product of the sub-image and your kernel:
OUT(x,y,p) = squeeze(I(x,y,p-win:p+win))' * squeeze(S(1,1,:,p))
This you can vectorize across all (x,y) by reshaping the sub-image of I to a (N*M)xK matrix (the K is horizontal, S is a column vector).
Repeating this across all p is easiest to implement with a loop, as you do now. The alternative is to create a larger S where each column is shifted by one, so you can do a single dot product between tge two matrices. But that S is also espensive to create, presumably requires a loop too. I don't think that avoiding loops is that pressing any more in MATLAB (it's gotten a lot faster over the years) and the product itself is probably the most expensive part of the algorithm anyway.
I have a n x m x d matrix A (i.e. A is like d n x m matrices). I would like to convert this into one n x m matrix B where each element B(i,j) is function of A(i,j,1), ..., A(i,j,d), more specifically the L2 norm of these values:
B(i,j) = sqrt[A(i,j,1)^2 + ... + A(i,j,d)^2]
Meaning I would like to condens or "flatten" the information in matrix A. How can I achieve this without resorting to a nested for loop?
Do elementwise squaring and sum along the third dimension to produce a N x M matrix and then apply square-root for a vectorized implementation, like so -
B = sqrt(sum(A.^2,3))
Is there any built-in or efficient way to calculate only certain elements in a matrix multiplication A*B = C in MATLAB?
For example, calculate only the elements of C, (i,j) such that D(i,j) = 1, for some other matrix.
This is one approach:
[ii, jj] = find(D==1);
result = sum(A(ii,:).'.*B(:,jj), 1);
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 1 year ago.
Improve this question
I know that, in the 1D case, the convolution between two vectors, a and b, can be computed as conv(a, b), but also as the product between the T_a and b, where T_a is the corresponding Toeplitz matrix for a.
Is it possible to extend this idea to 2D?
Given a = [5 1 3; 1 1 2; 2 1 3] and b=[4 3; 1 2], is it possible to convert a in a Toeplitz matrix and compute the matrix-matrix product between T_a and b as in the 1-D case?
Yes, it is possible and you should also use a doubly block circulant matrix (which is a special case of Toeplitz matrix). I will give you an example with a small size of kernel and the input, but it is possible to construct Toeplitz matrix for any kernel. So you have a 2d input x and 2d kernel k and you want to calculate the convolution x * k. Also let's assume that k is already flipped. Let's also assume that x is of size n×n and k is m×m.
So you unroll k into a sparse matrix of size (n-m+1)^2 × n^2, and unroll x into a long vector n^2 × 1. You compute a multiplication of this sparse matrix with a vector and convert the resulting vector (which will have a size (n-m+1)^2 × 1) into a n-m+1 square matrix.
I am pretty sure this is hard to understand just from reading. So here is an example for 2×2 kernel and 3×3 input.
*
Here is a constructed matrix with a vector:
which is equal to .
And this is the same result you would have got by doing a sliding window of k over x.
1- Define Input and Filter
Let I be the input signal and F be the filter or kernel.
2- Calculate the final output size
If the I is m1 x n1 and F is m2 x n2 the size of the output will be:
3- Zero-pad the filter matrix
Zero pad the filter to make it the same size as the output.
4- Create Toeplitz matrix for each row of the zero-padded filter
5- Create a doubly blocked Toeplitz matrix
Now all these small Toeplitz matrices should be arranged in a big doubly blocked Toeplitz matrix.
6- Convert the input matrix to a column vector
7- Multiply doubly blocked toeplitz matrix with vectorized input signal
This multiplication gives the convolution result.
8- Last step: reshape the result to a matrix form
For more details and python code take a look at my github repository:
Step by step explanation of 2D convolution implemented as matrix multiplication using toeplitz matrices in python
If you unravel k to a m^2 vector and unroll X, you would then get:
a m**2 vectork
a ((n-m)**2, m**2) matrix for unrolled_X
where unrolled_X could be obtained by the following Python code:
from numpy import zeros
def unroll_matrix(X, m):
flat_X = X.flatten()
n = X.shape[0]
unrolled_X = zeros(((n - m) ** 2, m**2))
skipped = 0
for i in range(n ** 2):
if (i % n) < n - m and ((i / n) % n) < n - m:
for j in range(m):
for l in range(m):
unrolled_X[i - skipped, j * m + l] = flat_X[i + j * n + l]
else:
skipped += 1
return unrolled_X
Unrolling X and not k allows a more compact representation (smaller matrices) than the other way around for each X - but you need to unroll each X. You could prefer unrolling k depending on what you want to do.
Here, the unrolled_X is not sparse, whereas unrolled_k would be sparse, but of size ((n-m+1)^2,n^2) as #Salvador Dali mentioned.
Unrolling k could be done like this:
from scipy.sparse import lil_matrix
from numpy import zeros
import scipy
def unroll_kernel(kernel, n, sparse=True):
m = kernel.shape[0]
if sparse:
unrolled_K = lil_matrix(((n - m)**2, n**2))
else:
unrolled_K = zeros(((n - m)**2, n**2))
skipped = 0
for i in range(n ** 2):
if (i % n) < n - m and((i / n) % n) < n - m:
for j in range(m):
for l in range(m):
unrolled_K[i - skipped, i + j * n + l] = kernel[j, l]
else:
skipped += 1
return unrolled_K
The code shown above doesn't produce the unrolled matrix of the right dimensions. The dimension should be (n-k+1)*(m-k+1), (k)(k). k: filter dimension, n: num rows in input matrix, m: num columns.
def unfold_matrix(X, k):
n, m = X.shape[0:2]
xx = zeros(((n - k + 1) * (m - k + 1), k**2))
row_num = 0
def make_row(x):
return x.flatten()
for i in range(n- k+ 1):
for j in range(m - k + 1):
#collect block of m*m elements and convert to row
xx[row_num,:] = make_row(X[i:i+k, j:j+k])
row_num = row_num + 1
return xx
For more details, see my blog post:
http://www.telesens.co/2018/04/09/initializing-weights-for-the-convolutional-and-fully-connected-layers/