I need to emulate the MATLAB function find, which returns the linear indices for the nonzero elements of an array. For example:
>> a = zeros(4,4)
a =
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
>> a(1,1) = 1
>> a(4,4) = 1
>> find(a)
ans =
1
16
numpy has the similar function nonzero, but it returns a tuple of index arrays. For example:
In [1]: from numpy import *
In [2]: a = zeros((4,4))
In [3]: a[0,0] = 1
In [4]: a[3,3] = 1
In [5]: a
Out[5]:
array([[ 1., 0., 0., 0.],
[ 0., 0., 0., 0.],
[ 0., 0., 0., 0.],
[ 0., 0., 0., 1.]])
In [6]: nonzero(a)
Out[6]: (array([0, 3]), array([0, 3]))
Is there a function that gives me the linear indices without calculating them myself?
numpy has you covered:
>>> np.flatnonzero(a)
array([ 0, 15])
Internally it's doing exactly what Sven Marnach suggested.
>>> print inspect.getsource(np.flatnonzero)
def flatnonzero(a):
"""
Return indices that are non-zero in the flattened version of a.
This is equivalent to a.ravel().nonzero()[0].
[more documentation]
"""
return a.ravel().nonzero()[0]
The easiest solution is to flatten the array before calling nonzero():
>>> a.ravel().nonzero()
(array([ 0, 15]),)
If you have matplotlib installed it's probably already there (find that is) in matplotlib.mlab module, as well as some other functions intended for compatibility with matlab. And yes it's implemented the same way as flatnonzero.
Related
How to display a matrix in a chart? I can do this in Python using:
import numpy as np
import matplotlib.pyplot as plt
def plot_filter(filters,tex):
fig, ax = plt.subplots()
for i in range(filters.shape[0]):
for j in range(filters.shape[1]):
c = hori[i][j]
ax.text(i+0.5, j+0.5, str(c), va='center', ha='center')
ax.set_xlim(min_val, filters.shape[0])
ax.set_ylim(min_val, filters.shape[1])
ax.set_xticks(np.arange(filters.shape[0]))
ax.set_yticks(np.arange(filters.shape[1]))
ax.set_xticklabels([])
ax.set_yticklabels([])
ax.grid()
plt.title(tex)
hori = np.array([[0, 0, 0], [1, 2, 1], [0, 0, 0]])
plot_filter(hori, 'Horizontal')
How to do this in MATLAB?
The result should look like this:
You can use cellplot for this purpose.
h = cellplot({0 1 0; 0 2 0; 0 1 0});
title('Horizontal');
which gives:
and to remove the red boxes, use:
set(h(2:2:end),'EdgeColor', [1 1 1], 'FaceColor', [1 1 1]);
I am a bit stumped on this one. I have used spline to smooth my data successfully, however it is just not working this time. Here is the snippet of the code that is not working. Any pointers would be highly appreciated.
In [46]: x
Out[46]:
array([ 600., 650., 700., 750., 800., 850., 900., 950.,
1000., 1050., 1100., 1150., 1200., 1250.])
In [47]: y
Out[47]:
array([ 2.68530481, 3.715443 , 4.11270841, 2.91720571, 1.49194971,
0.24770035, -0.64713611, -1.40938122, -2.24634466, -3.04577225,
-3.73914759, -4.35097303, -4.94702689, -5.56523414])
In [48]: x2=numpy.linspace(x.min(),x.max(),20)
In [49]: spline(x,y,x2)
Out[49]:
array([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0.])
Try using interp1d instead of spline which is deprecated(*):
import numpy as np
from matplotlib import pyplot as plt
from scipy.interpolate import interp1d
plt.ion()
x = np.array([600., 650., 700., 750., 800., 850., 900., 950.,
1000., 1050., 1100., 1150., 1200., 1250.])
y = np.array([2.68530481, 3.715443, 4.11270841, 2.91720571, 1.49194971,
0.24770035, -0.64713611, -1.40938122, -2.24634466,
-3.04577225, -3.73914759, -4.35097303, -4.94702689,
-5.56523414])
plt.plot(x,y)
x2 = np.linspace(x.min(), x.max(), 20)
f = interp1d(x, y, kind='cubic')
y2 = f(x2)
plt.plot(x2,y2)
Output
In [20]: x2
Out[20]:
array([ 600. , 634.21052632, 668.42105263, 702.63157895,
736.84210526, 771.05263158, 805.26315789, 839.47368421,
873.68421053, 907.89473684, 942.10526316, 976.31578947,
1010.52631579, 1044.73684211, 1078.94736842, 1113.15789474,
1147.36842105, 1181.57894737, 1215.78947368, 1250. ])
In [21]: y2
Out[21]:
array([ 2.68530481, 3.35699957, 4.03277746, 4.08420565, 3.31233485,
2.29896296, 1.34965136, 0.48288214, -0.21322503, -0.76839036,
-1.28566315, -1.84433723, -2.42194321, -2.96633554, -3.45993064,
-3.90553288, -4.31968149, -4.7262301 , -5.13883472, -5.56523414])
(*) Under additional tools, scipy lists spline as:
Functions existing for backward compatibility (should not be used in new code):
I want to compare two image chunks such that if they are exactly the same, the result must be 1, and if they match 60 percent, the answer must be 0.6.
In Matlab, I can do this using corr2 command, but in python I couldn't find a way. I have tried numpy.corrcoef but it returns a matrix and scipy.signal.correlate2d returns the same.
This is what I have tried:
import numpy as np
import matplotlib.pyplot as plt
from skimage.filter import threshold_otsu
import matplotlib.cm as cm
import Image
import scipy
from PIL import Image as im
fname = 'testi.jpg'
image = Image.open(fname).convert("L")
arr = np.asarray(image)
global_thresh = threshold_otsu(arr)
global_otsu = arr >= global_thresh
global_otsu = np.invert(global_otsu).astype(int)
a1 = global_otsu[80:150,1350:1350+160]
fname1 = 'testi2.jpg'
image1 = Image.open(fname1).convert("L")
arr1 = np.asarray(image1)
global_thresh1 = threshold_otsu(arr1)
global_otsu1 = arr1 >= global_thresh1
global_otsu1 = np.invert(global_otsu1).astype(int)
a2 = global_otsu1[80:150,1350:1350+160]
co = scipy.signal.correlate2d(a1,a2)
plt.gray()
plt.subplot(121)
plt.imshow(a1)
plt.subplot(122)
plt.imshow(a2)
plt.show()
and the result is:
[[0 0 0 ..., 0 0 0]
[0 0 0 ..., 0 0 0]
[0 0 0 ..., 0 0 0]
...,
[0 0 0 ..., 0 0 0]
[0 0 0 ..., 0 0 0]
[0 0 0 ..., 0 0 0]]
These are the images that I want to compare:
Since you want to compare pixel by pixel you can perform correlation on the flattened images, :
cm = np.corrcoef(a1.flat, a2.flat)
cmcontains the symmetric correlation matrix where the off-diagonal element is the correlation coefficient. You get it by
r = cm[0, 1]
Edit:
There is a problem with using correlation for comparing images. If any of them is completely flat (all pixel the same value) the correlation is undefined.
If the images are binary you could instead simply calculate the percantage of equal pixels:
agreement = np.sum(a == b) / a.size
I want to do something like the following.
Equivalent code in NumPy
a = np.zeros(5)
a[np.array([1, 2, 4])] += [1, 2, 3]
a
array([ 0., 1., 2., 0., 3.])
I tried the following but it does not work.
val v = DenseVector.zeros[Double](5)
v(1, 2, 4) :+= DenseVector(1, 2, 3)
<console>:18: error: could not find implicit value for parameter op:breeze.linalg.operators.OpAdd.InPlaceImpl2[breeze.linalg.Vector[Double],breeze.linalg.DenseVector[Int]]
v(1, 2, 4) += DenseVector(1, 2, 3)
Any help would be appreciated
I have large but sparse arrays and I want to rearrange them by swapping rows an columns. What is a good way to do this in scipy.sparse?
Some issues
I don't think that permutation matrices are well suited for this task, as they like randomly change the sparsity structure. And a manipulation will always 'multiply' all columns or rows, even if there are only a few swaps necessary.
What is the best sparse matrix representation in scipy.sparse for this task?
Suggestions for implementation are very welcome.
I have tagged this with Matlab as well, since this question might find an answer that is not necessarily scipy specific.
CSC format keeps a list of the row indices of all non-zero entries, CSR format keeps a list of the column indices of all non-zero entries. I think you can take advantage of that to swap things around as follows, and I think there shouldn't be any side-effects to it:
def swap_rows(mat, a, b) :
mat_csc = scipy.sparse.csc_matrix(mat)
a_idx = np.where(mat_csc.indices == a)
b_idx = np.where(mat_csc.indices == b)
mat_csc.indices[a_idx] = b
mat_csc.indices[b_idx] = a
return mat_csc.asformat(mat.format)
def swap_cols(mat, a, b) :
mat_csr = scipy.sparse.csr_matrix(mat)
a_idx = np.where(mat_csr.indices == a)
b_idx = np.where(mat_csr.indices == b)
mat_csr.indices[a_idx] = b
mat_csr.indices[b_idx] = a
return mat_csr.asformat(mat.format)
You could now do something like this:
>>> mat = np.zeros((5,5))
>>> mat[[1, 2, 3, 3], [0, 2, 2, 4]] = 1
>>> mat = scipy.sparse.lil_matrix(mat)
>>> mat.todense()
matrix([[ 0., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.],
[ 0., 0., 1., 0., 0.],
[ 0., 0., 1., 0., 1.],
[ 0., 0., 0., 0., 0.]])
>>> swap_rows(mat, 1, 3)
<5x5 sparse matrix of type '<type 'numpy.float64'>'
with 4 stored elements in LInked List format>
>>> swap_rows(mat, 1, 3).todense()
matrix([[ 0., 0., 0., 0., 0.],
[ 0., 0., 1., 0., 1.],
[ 0., 0., 1., 0., 0.],
[ 1., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])
>>> swap_cols(mat, 0, 4)
<5x5 sparse matrix of type '<type 'numpy.float64'>'
with 4 stored elements in LInked List format>
>>> swap_cols(mat, 0, 4).todense()
matrix([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 1.],
[ 0., 0., 1., 0., 0.],
[ 1., 0., 1., 0., 0.],
[ 0., 0., 0., 0., 0.]])
I have used a LIL matrix to show how you could preserve the type of your output. In your application you probably want to already be in CSC or CSR format, and select whether to swap rows or columns first based on it, to minimize conversions.
I've found using matrix operations to be the most efficient. Here's a function which will permute the rows and/or columns to a specified order. It can be modified to swap two specific rows/columns if you would like.
from scipy import sparse
def permute_sparse_matrix(M, row_order=None, col_order=None):
"""
Reorders the rows and/or columns in a scipy sparse matrix to the specified order.
"""
if row_order is None and col_order is None:
return M
new_M = M
if row_order is not None:
I = sparse.eye(M.shape[0]).tocoo()
I.row = I.row[row_order]
new_M = I.dot(new_M)
if col_order is not None:
I = sparse.eye(M.shape[1]).tocoo()
I.col = I.col[col_order]
new_M = new_M.dot(I)
return new_M
In Matlab you can just index the columns and rows the way you like:
Matrix = speye(10);
mycolumnorder = [1 2 3 4 5 6 10 9 8 7];
myroworder = [4 3 2 1 5 6 7 8 9 10];
Myorderedmatrix = Matrix(myroworder,mycolumnorder);
I think this preserves sparsity... Don't know about scipy though...