Traceback (most recent call last):
File "N:\NCI Cyber Security\Sem 3\keycalculatingValue.py", line 37, in
finalData['key'] = chr(keyList[j])
TypeError: integer argument expected, got float
It is hard to say without seeing more code, but it looks like keyList[j] is returning a float while the function chr() requires and integer in the range 0 through 1,114,111. If the integer is out of that range it will return a ValueError; it seems if the input type is incorrect it will return a TypeError, although that does not seem to be documented.
The following code gives the same error:
(j := 1.0) and chr(j)
Related
I have a function that takes 2 spark dataframes, some other arguments and outputs a scalar value.
Would like to bootstrap (to fill missing values in the dataframes above) n times with the whole process above and return the output with n rows.
I tried the below for a simple problem:
def sum_fn(a,b):
return a+b
rdd=spark.sparkContext.parallelize(list(range(1, 9+1)))
df = rdd.map(lambda x: (x,sum_fn(1,x))).toDF()
display(df)
This works fine, however when I input my function with sdf as input instead of sum_fn
I get an error :
Traceback (most recent call last):
File "/databricks/spark/python/pyspark/serializers.py", line 476, in
dumps
return cloudpickle.dumps(obj, pickle_protocol)
File "/databricks/spark/python/pyspark/cloudpickle/cloudpickle_fast.py",
line 72, in dumps
cp.dump(obj)
File "/databricks/spark/python/pyspark/cloudpickle/cloudpickle_fast.py", line 540, in dump
return Pickler.dump(self, obj)
TypeError: cannot pickle '_thread.RLock' object
PicklingError: Could not serialize object: TypeError: cannot pickle '_thread.RLock' object
Could someone please suggest on how I could do the same
Thanks
I am having trouble counting every element in a list that I have created in PySpark.
Here is what I am working with:
test2 = words.filter(lambda line: re.match(r'^[AEIOU]', line)).take(10)
test2
[u'EBook', u'Author:', u'English', u'OF', u'EBOOK', u'Inc.,', u'Etext', u'Inc.,', u'Etexts', u'Etext']
Now I want to confirm the count of test2 is 10. But everytime I use test2.count(), it's giving me an error:
Traceback (most recent call last):
File "", line 1, in
TypeError: count() takes exactly one argument (0 given)
Can someone help me learn how to count the elements properly?
Thank you!
test2 is a list, so you should be doing len(test2) to find the number of elements. The function count(), when called on a list, will return the number of occurrences of whatever you pass as a parameter.
I have the following class and method that should convolve an array with a kernel.
import numpy as np
from numpy.fft import fft2 as FFT, ifft2 as IFFT
from PIL import Image
from tqdm import trange, tqdm
from numba import jit
from time import sleep
import _kernel
class convolve(object):
""" contains methods to convolve two images """
def __init__(self, image_array, kernel):
self.array = image_array
self.kernel = kernel
self.__rangeX_ = self.array.shape[0]
self.__rangeY_ = self.array.shape[1]
self.__rangeKX_ = self.kernel.shape[0]
self.__rangeKY_ = self.kernel.shape[1]
if (self.__rangeKX_ >= self.__rangeX_ or \
self.__rangeKY_ >= self.__rangeY_):
raise ValueError('Must submit suitable sizes for convolution.')
#jit(nopython=True)
def spaceConv(self):
""" normal convolution, O(N^2*n^2). This is usually too slow """
# pad array for convolution
offsetX = self.__rangeKX_ // 2
offsetY = self.__rangeKY_ // 2
self.array = np.pad(self.array, \
[(offsetY, offsetY), (offsetX, offsetX)], \
mode='constant', constant_values=0)
# this is the O(N^2) part of this algorithm
for i in xrange(self.__rangeX_ - 2*offsetX):
for j in xrange(self.__rangeY_ - 2*offsetY):
# Now O(n^2) portion
total = 0.0
for k in xrange(2*offsetX+1):
for t in xrange(2*offsetY+1):
total += self.kernel[k][t] * self.array[i+k][j+t]
self.array[i+offsetX][j+offsetY] = total
return self.array
As an additional note (in case anyone asks), _kernel just generates specific kernels one may want to convolve the image with (e.g. Gaussian, Moffat, etc.), so it has nothing to do with this class.
When I call the above class on an image and kernel, I get the following error:
Traceback (most recent call last):
File "fftconv.py", line 147, in <module>
plt.imshow(conv.spaceConv(), interpolation='none', cmap='gray')
File "/root/anaconda2/lib/python2.7/site-packages/numba/dispatcher.py", line 304, in _compile_for_args
raise e
numba.errors.UntypedAttributeError: Caused By:
Traceback (most recent call last):
File "/root/anaconda2/lib/python2.7/site-packages/numba/compiler.py", line 249, in run
stage()
File "/root/anaconda2/lib/python2.7/site-packages/numba/compiler.py", line 465, in stage_nopython_frontend
self.locals)
File "/root/anaconda2/lib/python2.7/site-packages/numba/compiler.py", line 789, in type_inference_stage
infer.propagate()
File "/root/anaconda2/lib/python2.7/site-packages/numba/typeinfer.py", line 717, in propagate
raise errors[0]
UntypedAttributeError: Unknown attribute "rangeKX" of type pyobject
File "fftconv.py", line 45
[1] During: typing of get attribute at fftconv.py (45)
Failed at nopython (nopython frontend)
Unknown attribute "rangeKX" of type pyobject
File "fftconv.py", line 45
[1] During: typing of get attribute at fftconv.py (45)
This error may have been caused by the following argument(s):
- argument 0: cannot determine Numba type of value <__main__.convolve object at 0xaff5628c>
Usually I'm pretty good at tracing through Python errors to the cause, but because I'm not familiar with the inner-works of Numba, I'm not sure why it doesn't know what type offsetX is. Any suggestions?
One step performed by numba is type-inference. This assigns types to the different values present in the function so that it can compile (in a way that it works fast).
The error means that numba doesn't understand the first input argument on the function (self in this case). Numba works best in plain functions where the arguments are scalars or array (all numeric). One option would be to move the O(n^2) loop into a function of its own and have that function receive the arrays and any other value explicitly, and decorate that function with numba.njit (or numba.jit(nopython=True), which are equivalent
Also worth a try is just trying the code "as is" removing the "nopython=True". If the performance is good enough then leave it alone :). That may happen, as numba.jit is able to detect loops inside the code that can be compiled in "no python" mode and automatically do what is needed so that the loop itself is compiled in full speed mode. The explicit "nopython=True" keyword disables that mode though.
I try to check if input argument is of specific type and throw error message like:
function test(input)
if ~ischar(input)
error('%s is invalid input type.', class(input));
end
end
But Matlab shows error message with backtrace-information:
>> test(1)
Error using test (line 3)
double is invalid input type.
How can I turn off the line Error using test (line 3)?
I'm looking for something similar to off backtrace with warning: warning off backtrace;.
I'm not sure you can. The closest I got was by defining my own error structure:
testerr.message = 'test';
testerr.identifier = '';
testerr.stack.file = '';
testerr.stack.name = 'Test Thing';
testerr.stack.line = 1;
error(testerr)
Which returns:
Error using Test Thing
test
As long as you keep the file field blank it will not display the line specified in the stack.
One potential workaround could be a combination of fprintf and return, courtesy of Undocumented MATLAB:
function test(input)
if ~ischar(input)
fprintf(2, '%s is invalid input type.\n', class(input));
return
end
end
Depending on where this check resides in your real function you might need to get creative with how it exits, since return only kicks you back to the invoking function. Probably have it output a True/False flag?
I am very new to postgresql and writing functions so bear with me. I need to transform a Python script into a postgresql function and I intend to use PL/Python for the purpose. However I am having some problems in doing so. When executing the function I receive an error:
ERROR: TypeError: unsupported operand type(s) for +: 'int' and 'dict'
SQL state: XX000
Context: Traceback (most recent call last):
PL/Python function "ellipse", line 5, in
meanX=float(sum(Xarray))/len(Xarray) if len(Xarray) > 0 else float('nan')
PL/Python function "ellipse"
As to my knowledge, the query stores the result in dictionary which then results in this error (since I am trying to operate with list in the script). At least I think this can be the problem. So my question would be - is there a way to store the query result in a list variable?
CREATE OR REPLACE FUNCTION ellipse()
returns setof ellipse_param as $$
Xarray=plpy.execute("select laius from proov")
Yarray=plpy.execute("select pikkus from proov")
meanX=float(sum(Xarray))/len(Xarray) if len(Xarray) > 0 else float('nan')
meanY=float(sum(Yarray))/len(Yarray) if len(Yarray) > 0 else float('nan')
Xdevs=[]
Ydevs=[]
for x in Xarray:
dev=x-meanX
Xdevs.append(dev)
dev=0
for y in Yarray:
dev=y-meanY
Ydevs.append(dev)
dev=0
sumX=0
sumY=0
for x in Xdevs:
sumX+=x**2
for y in Ydevs:
sumY+=y**2
Xaxes=sqrt(sumX/len(Xdevs))
Yaxes=sqrt(sumY/len(Ydevs))
A=sumX-sumY
B=sqrt(A**2+(((float(sum([a*b for a,b in zip(Xdevs,Ydevs)])))**2)*4))
C=float(sum([a*b for a,b in zip(Xdevs,Ydevs)]))*2
rotation=(atan(((A+B)/C)))
Sx=sqrt(((float(sum([(a*cos(rotation)-b*sin(rotation))**2 for a,b in zip(Xdevs,Ydevs)])))/(len(Xdevs)-2))*2)
Sy=sqrt(((float(sum([(c*sin(rotation)+d*cos(rotation))**2 for c,d in zip(Xdevs,Ydevs)])))/(len(Xdevs)-2))*2)
return meanX, meanY, rotation, Xaxes, Yaxes
$$ LANGUAGE plpython3u;
plpy.execute will give you a list of dict, so you want something like
sum([x['laius'] for x in Xarray])
More info in the docs here http://www.postgresql.org/docs/devel/static/plpython-database.html
Edit: I read too quickly and skimmed over your entire function - you may want to put the list constructor higher up, probably right after executing your queries, so that you have a list of values to use later on (I didn't notice how much of the later code assumes the data are in a simple list).