How would I replace values less than 4 with 0 in this array without triggering a SparseEfficiencyWarning and without reducing its sparsity?
from scipy import sparse
x = sparse.csr_matrix(
[[0, 1, 2, 3, 4],
[1, 2, 3, 4, 5],
[0, 0, 0, 2, 5]])
x[x < 4] = 0
x.toarray() # verifies that this works
Note also that the sparsity between the initial version of x is 11 stored elements, which rises to 15 stored elements after doing the masking.
Manipulate the data array directly
from scipy import sparse
x = sparse.csr_matrix(
[[0, 1, 2, 3, 4],
[1, 2, 3, 4, 5],
[0, 0, 0, 2, 5]])
x.data[x.data < 4] = 0
>>> x.toarray()
array([[0, 0, 0, 0, 4],
[0, 0, 0, 4, 5],
[0, 0, 0, 0, 5]])
>>> x.data
array([0, 0, 0, 4, 0, 0, 0, 4, 5, 0, 5])
Note that the sparsity is unchanged and there are zero values unless you run x.eliminate_zeros().
x.eliminate_zeros()
>>> x.data
array([4, 4, 5, 5])
If for some reason you don't want to use a boolean mask & fancy indexing in numpy, you can loop over the array with numba:
import numba
#numba.jit(nopython=True)
def _set_array_less_than_to_zero(array, value):
for i in range(len(array)):
if array[i] < value:
array[i] = 0
This should also be faster than the numpy indexing by a fairly substantial degree.
array = np.arange(10)
_set_array_less_than_to_zero(array, 5)
>>> array
array([0, 0, 0, 0, 0, 5, 6, 7, 8, 9])
Related
I want to compare each element against all others like following. The number of variables like a, b, c is dynamic. However, each variable's array size is uniform.
let a = [1, 2, 3]
let b = [3, 4, 5]
let c = [4, 5, 6]
for i in a {
for j in b {
for k in c {
/// comparison
}
}
}
Instead looping from start to finish at once, what would be a way to make each comparison on call? For example:
compare(iteration: 0)
/// compares a[0], b[0], c[0]
compare(iteration: 1)
/// compares a[0], b[0], c[1]
/// all the way to
/// compares a[2], b[2], c[2]
Or it could even be like following:
next()
/// compares a[0], b[0], c[0]
next()
/// compares a[0], b[0], c[1]
almost like an iterator stepping through each cycle dictated by my invocation.
Let the number of arrays be n. And let the number of elements in each array, which is guaranteed the same for all of them, be k.
Then create an array consisting of the integers 0 through k-1, repeated n times. For example, in your case, n is 3, and k is 3, so generate the array
[0, 1, 2, 0, 1, 2, 0, 1, 2]
Now obtain all combinations of n elements of that array. You can do this using the algorithm at https://github.com/apple/swift-algorithms/blob/main/Guides/Combinations.md. Unique the result (by, for example, coercing to a Set and then back to an Array). This will give you a result equivalent, in some order or other, to
[[0, 1, 2], [0, 1, 0], [0, 1, 1], [0, 2, 0], [0, 2, 1], [0, 2, 2], [0, 0, 1], [0, 0, 2], [0, 0, 0], [1, 2, 0], [1, 2, 1], [1, 2, 2], [1, 0, 1], [1, 0, 2], [1, 0, 0], [1, 1, 2], [1, 1, 0], [1, 1, 1], [2, 0, 1], [2, 0, 2], [2, 0, 0], [2, 1, 2], [2, 1, 0], [2, 1, 1], [2, 2, 0], [2, 2, 1], [2, 2, 2]]
You can readily see that those are all 27 possible combinations of the numbers 0, 1, and 2. But that is exactly what you were doing with your for loops! So now, use those subarrays as indexes into each of your original arrays respectively.
So for instance, using my result and your original example, the first subarray [0, 1, 2] yields [1, 4, 6] — the first value from the first array, the second value from the second array, and the third value from the third array. And so on.
In this way you will have generated all possible n-tuples by choosing one value from each of your original arrays, which is the desired result; and we are in no way bound to fixed values of n and k, which was what you wanted to achieve. You will then be able to "compare" the elements of each n-tuple, whatever that may mean to you (you did not say in your question what it means).
In the case of your original values, we will get these n-tuples (expressed as arrays):
[1, 4, 6]
[1, 4, 4]
[1, 4, 5]
[1, 5, 4]
[1, 5, 5]
[1, 5, 6]
[1, 3, 5]
[1, 3, 6]
[1, 3, 4]
[2, 5, 4]
[2, 5, 5]
[2, 5, 6]
[2, 3, 5]
[2, 3, 6]
[2, 3, 4]
[2, 4, 6]
[2, 4, 4]
[2, 4, 5]
[3, 3, 5]
[3, 3, 6]
[3, 3, 4]
[3, 4, 6]
[3, 4, 4]
[3, 4, 5]
[3, 5, 4]
[3, 5, 5]
[3, 5, 6]
Those are precisely the triples of values you are after.
Actual code:
// your original conditions
let a = [1, 2, 3]
let b = [3, 4, 5]
let c = [4, 5, 6]
let originals = [a, b, c]
// The actual solution starts here. Note that I never use any hard
// coded numbers.
let n = originals.count
let k = originals[0].count
var indices = [Int]()
for _ in 0..<n {
for i in 0..<k {
indices.append(i)
}
}
let combos = Array(indices.combinations(ofCount: n))
var combosUniq = [[Int]]()
var combosSet = Set<[Int]>()
for combo in combos {
let success = combosSet.insert(combo)
if success.inserted {
combosUniq.append(combo)
}
}
// And here's how to generate your actual desired values.
for combo in combosUniq {
var tuple = [Int]()
for (outerIndex, innerIndex) in combo.enumerated() {
tuple.append(originals[outerIndex][innerIndex])
}
print(tuple) // in real life, do something useful here
}
}
I am trying to use the LAPACK banded symmetric matrix solver dpbsv. I am testing the matrix:
4, 2, 0, 0, 0
2 4, 3, 0, 0
0, 3, 11, 7, 0
0, 0, 7, 11, 5
0, 0, 0, 5, 13
Mathematica tells me that this matrix is positive definite, with a determinate of 3684
I am using swift and have constructed the array
var a: [Double] = [ 0, 2, 3, 7, 5,
4, 4, 11, 11, 13]
var b: [Double] = [1, 2, 3, 4, 5]
And I am calling dpbsv as
var uplo = Int8("U".utf8.first!) // set to 'U'
var n = __CLPK_integer(5)
var kd = __CLPK_integer(1)
var ldab = kd + 1
var nrhs = __CLPK_integer(1)
var ldb = __CLPK_integer(5)
var info: __CLPK_integer = 0
dpbsv_(&uplo,
&n,
&kd,
&nrhs,
&a,
&ldab,
&b,
&ldb,
&info)
if info != 0 {
// here info is 3, indicating non-positive definite.
NSLog("error \(info)")
}
Any idea what the issue is here? Am I interpreting the parameters to dpbsv_ correctly? I've tried other matrices that Mathematica claims are pos-def with the same result.
So, apparently, what LAPACK documents as rows need to be coded as columns in swift. So, if the array is changed to
[ 0, 4,
2, 4,
3, 11,
7, 11,
5, 13]
everything works fine.
This is my code for sobel filter:
def init_f(shape, dtype=None):
sobel_x = tf.constant([[-5, -4, 0, 4, 5], [-8, -10, 0, 10, 8], [-10, -20, 0, 20, 10], [-8, -10, 0, 10, 8], [-5, -4, 0, 4, 5]])
ker = np.zeros(shape, dtype)
ker_shape = tf.shape(ker)
kernel = tf.tile(sobel_x, ker_shape)//*Is this correct?*
return kernel
model.add(Conv2D(filters=30, kernel_size=(5,5), kernel_initializer=init_f, strides=(1,1), activation='relu'))
So far I have managed to do this.
But, this gives me error:
Shape must be rank 2 but is rank 4 for 'conv2d_17/Tile' (op: 'Tile') with input shapes: [5,5], [4].
Tensorflow Version: 2.1.0
You're close, but the args to tile don't appear to be correct. That is why you're getting the error "Shape must be rank 2 but is rank 4 for..." You're sobel_x must be a rank 4 tensor, so you need to add two more dimensions. I used reshape in this example.
from tensorflow import keras
import tensorflow as tf
import numpy
def kernelInitializer(shape, dtype=None):
print(shape)
sobel_x = tf.constant(
[
[-5, -4, 0, 4, 5],
[-8, -10, 0, 10, 8],
[-10, -20, 0, 20, 10],
[-8, -10, 0, 10, 8],
[-5, -4, 0, 4, 5]
], dtype=dtype )
#create the missing dims.
sobel_x = tf.reshape(sobel_x, (5, 5, 1, 1))
print(tf.shape(sobel_x))
#tile the last 2 axis to get the expected dims.
sobel_x = tf.tile(sobel_x, (1, 1, shape[-2],shape[-1]))
print(tf.shape(sobel_x))
return sobel_x
x1 = keras.layers.Input((128, 128, 3))
cvl = keras.layers.Conv2D(30, kernel_size=(5,5), kernel_initializer=kernelInitializer, strides=(2,2), activation='relu')
model = keras.Sequential();
model.add(x1)
model.add(cvl)
data = numpy.ones((1, 128, 128, 3))
data[:, 0:64, 0:64, :] = 0
pd = model.predict(data)
print(pd.shape)
d = pd[0, :, :, 0]
for row in d:
for col in row:
m = '0'
if col != 0:
m = 'X'
print(m, end="")
print("")
I looked at using expand_dims instead of reshape but there didn't appear any advantage. broadcast_to seems ideal, but you still have to add the dimensions, so I don't think it was better than tile.
Why 30 filters of the same filter though? Are they going to be changed afterwards?
i have written the code below,it works for shortest distance but not for shortest path,
import math
def floyd(dist_mat):
n=len(dist_mat)
p=[[0]*n]*n
for k in range(n):
for i in range(n):
for j in range(n):
if dist_mat[i][j]>dist_mat[i][k]+dist_mat[k][j]:
dist_mat[i][j] = dist_mat[i][k] + dist_mat[k][j]
p[i][j] = k+1
return p
if __name__ == '__main__':
print(floyd([[0,5,9999,9999],
[50,0,15,5],
[30,9999,0,15],
[15,9999,5,0]]))
result of this code is: [[4, 1, 4, 2], [4, 1, 4, 2], [4, 1, 4, 2], [4, 1, 4, 2]]
true result is: [[0, 0, 4, 2], [4, 0, 4, 0], [0, 1, 0, 0], [0, 1, 0, 0]],
I will be happy to receive your ideas about why it works wrong soon
I'm taking Princeton's algorithms-divide-conquer course - 3rd week, and trying to implement the quicksort.
Here's my current implementation with some tests ready to run:
import unittest
def quicksort(x):
if len(x) <= 1:
return x
pivot = x[0]
xLeft, xRight = partition(x)
print(xLeft, xRight)
quicksort(xLeft)
quicksort(xRight)
return x
def partition(x):
j = 0
print('partition', x)
for i in range(0, len(x)):
if x[i] < x[0]:
n = x[j + 1]
x[j + 1] = x[i]
x[i] = n
j += 1
p = x[0]
x[0] = x[j]
x[j] = p
return x[:j + 1], x[j + 1:]
class Test(unittest.TestCase):
def test_partition_pivot_first(self):
arrays = [
[3, 1, 2, 5],
[3, 8, 2, 5, 1, 4, 7, 6],
[10, 100, 3, 4, 2, 101]
]
expected = [
[[2, 1, 3], [5]],
[[1, 2, 3], [5, 8, 4, 7, 6]],
[[2, 3, 4, 10], [100, 101]]
]
for i in range(0, len(arrays)):
xLeft, xRight = partition(arrays[i])
self.assertEqual(xLeft, expected[i][0])
self.assertEqual(xRight, expected[i][1])
def test_quicksort(self):
arrays = [
[1, 2, 3, 4, 5, 6],
[3, 5, 6, 10, 2, 4]
]
expected = [
[1, 2, 3, 4, 5, 6],
[1, 2, 3, 4, 6, 10]
]
for i in range(0, len(arrays)):
arr = arrays[i]
quicksort(arr)
self.assertEqual(arr, expected[i])
if __name__ == "__main__":
unittest.main()
so for array = [3, 5, 6, 10, 2, 4] I get [2, 3, 6, 10, 5, 4] as a result... I can't figure what's wrong with my code. It partitions just fine, but the results are off...
Can anyone chip in? :) Thank you!
it's actually so minor problem that you'd be laughing
the problem resides with quicksort function
the correct one is:
def quicksort(x):
if len(x) <= 1:
return x
pivot = x[0]
xLeft, xRight = partition(x)
print(xLeft, xRight)
quicksort(xLeft)
quicksort(xRight)
x=xLeft+xRight #this one!
return x
what happens is python created a new object out of these xleft and xright they were never an in place-sort
so this is one solution(which is not in place)
the other one is to pass the list,the start_index,end_index
and do it in place
well done fella!
edit:
and actually if you'd print xleft and xright you'd see it performed perfectly:)