Sympy .coeff_all() returned list is not readable by scipy - scipy

I have question about the data type of the result returned by Sympy Poly.all_coeffs(). I have started to use Sympy just recently.
My Sympy transfer function is following:
Then I run this code:
n,d = fraction(Gs)
num = Poly(n,s)
den = Poly(d,s)
num_c = num.all_coeffs()
den_c = den.all_coeffs()
I get:
Then I run this code:
from scipy import signal
#nu = [5000000.0]
#de = [4.99, 509000.0]
nu = num_c
de = den_c
sys = signal.lti(nu, de)
w,mag,phase = signal.bode(sys)
plt.plot(w/(2*np.pi), mag)
and the result is:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-131-fb960684259c> in <module>
4 nu = num_c
5 de = den_c
----> 6 sys = signal.lti(nu, de)
But if I use those commented line 'nu' and 'de' straight python lists instead, the program works. So what is wrong here?

Why did you just show a bit the error? Why not the full message, maybe even the full traceback!
In [60]: sys = signal.lti(num_c, den_c)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-60-21f71ecd8884> in <module>
----> 1 sys = signal.lti(num_c, den_c)
/usr/local/lib/python3.6/dist-packages/scipy/signal/ltisys.py in __init__(self, *system, **kwargs)
590 self._den = None
591
--> 592 self.num, self.den = normalize(*system)
593
594 def __repr__(self):
/usr/local/lib/python3.6/dist-packages/scipy/signal/filter_design.py in normalize(b, a)
1609 leading_zeros = 0
1610 for col in num.T:
-> 1611 if np.allclose(col, 0, atol=1e-14):
1612 leading_zeros += 1
1613 else:
<__array_function__ internals> in allclose(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/numpy/core/numeric.py in allclose(a, b, rtol, atol, equal_nan)
2169
2170 """
-> 2171 res = all(isclose(a, b, rtol=rtol, atol=atol, equal_nan=equal_nan))
2172 return bool(res)
2173
<__array_function__ internals> in isclose(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/numpy/core/numeric.py in isclose(a, b, rtol, atol, equal_nan)
2267 y = array(y, dtype=dt, copy=False, subok=True)
2268
-> 2269 xfin = isfinite(x)
2270 yfin = isfinite(y)
2271 if all(xfin) and all(yfin):
TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
Now look at the elements of the num_c list (same for den_c):
In [55]: num_c[0]
Out[55]: 500000.000000000
In [56]: type(_)
Out[56]: sympy.core.numbers.Float
The scipy code is doing numpy testing on the inputs. So it's first turned the lists into arrays:
In [61]: np.array(num_c)
Out[61]: array([500000.000000000], dtype=object)
This array contains sympy object(s). It can't cast that to numpy float with 'safe'. But an explicit astype uses unsafe as the default:
In [63]: np.array(num_c).astype(float)
Out[63]: array([500000.])
So lets convert both lists into valid numpy float arrays:
In [64]: sys = signal.lti(np.array(num_c).astype(float), np.array(den_c).astype(float))
In [65]: sys
Out[65]:
TransferFunctionContinuous(
array([100200.4008016]),
array([1.00000000e+00, 1.02004008e+05]),
dt: None
)
Conversion in a list comprehension also works:
sys = signal.lti([float(i) for i in num_c],[float(i) for i in den_c])

You likely need to conver sympy objects to floats / lists of floats.

Related

Python Jupyter Notebook scipy

For a long time I was able to add data and fit, then plot the curve with data. But recently I get this:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-6-6f645a2744bc> in <module>
1 poland = prepare_data(europe_data, 'Poland')
----> 2 plot_all(poland, max_y=400000)
3 poland
~/Pulpit/library.py in plot_all(country, max_x, max_y)
43 def plot_all(country, max_x = 1000, max_y = 500000):
44
---> 45 parameters_logistic = scipy.optimize.curve_fit(func_logistic, country['n'], country['all'])[0]
46 parameters_expo = scipy.optimize.curve_fit(func_expo, country['n'], country['all'])[0]
47
/usr/local/lib64/python3.6/site-packages/scipy/optimize/minpack.py in curve_fit(f, xdata, ydata, p0, sigma, absolute_sigma, check_finite, bounds, method, jac, **kwargs)
787 cost = np.sum(infodict['fvec'] ** 2)
788 if ier not in [1, 2, 3, 4]:
--> 789 raise RuntimeError("Optimal parameters not found: " + errmsg)
790 else:
791 # Rename maxfev (leastsq) to max_nfev (least_squares), if specified.
RuntimeError: Optimal parameters not found: Number of calls to function has reached maxfev = 800.
Here are all Python Jupyter Notebook files: https://files.fm/u/zj7cc6ne#sign_up
How to solve this?
scipy.optimize.curve_fit takes a keyword argument p0.
Initial guess for the parameters (length N). If None, then the initial
values will all be 1 (if the number of parameters for the function can
be determined using introspection, otherwise a ValueError is raised).
If the defaults 1 are too far of from the result the algorithm may not converge. Try to put some values that make sense for your problem.

PySpark TypeError: object of type 'ParamGridBuilder' has no len()

I am trying to tune my model on Databricks using Pyspark.
I receive the following error:
TypeError: object of type 'ParamGridBuilder' has no len()
My code has been listed below.
from pyspark.ml.recommendation import ALS
from pyspark.ml.evaluation import RegressionEvaluator
als = ALS(userCol = "userId",itemCol="movieId", ratingCol="rating", coldStartStrategy="drop", nonnegative = True, implicitPrefs = False)
# Imports ParamGridBuilder package
from pyspark.ml.tuning import ParamGridBuilder
# Creates a ParamGridBuilder, and adds hyperparameters
param_grid = ParamGridBuilder().addGrid(als.rank, [5,10,20,40]).addGrid(als.maxIter, [5,10,15,20]).addGrid(als.regParam,[0.01,0.001,0.0001,0.02])
evaluator = RegressionEvaluator(metricName="rmse", labelCol="rating",predictionCol="prediction")
# Imports CrossValidator package
from pyspark.ml.tuning import CrossValidator
# Creates cross validator and tells Spark what to use when training and evaluates
cv = CrossValidator(estimator = als,
estimatorParamMaps = param_grid,
evaluator = evaluator,
numFolds = 5)
model = cv.fit(training)
TypeError: object of type 'ParamGridBuilder' has no len()
Full Error Log:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<command-1952169986445972> in <module>()
----> 1 model = cv.fit(training)
2
3 # Extract best combination of values from cross validation
4
5 best_model = model.bestModel
/databricks/spark/python/pyspark/ml/base.py in fit(self, dataset, params)
130 return self.copy(params)._fit(dataset)
131 else:
--> 132 return self._fit(dataset)
133 else:
134 raise ValueError("Params must be either a param map or a list/tuple of param maps, "
/databricks/spark/python/pyspark/ml/tuning.py in _fit(self, dataset)
279 est = self.getOrDefault(self.estimator)
280 epm = self.getOrDefault(self.estimatorParamMaps)
--> 281 numModels = len(epm)
It simple means that your object does not have a length property (unlike lists). Thus, In your line
param_grid = ParamGridBuilder()
.addGrid(als.rank, [5,10,20,40])
.addGrid(als.maxIter, [5,10,15,20])
.addGrid(als.regParam, [0.01,0.001,0.0001,0.02])
You should add .build() in the end to actually construct a grid.

Print proper mathematical formatting

When I use sympy to get the square root of 8, the output is ugly:
2*2**(1/2)
import sympy
In [2]: sympy.sqrt(8)
Out[2]: 2*2**(1/2)
Is there any way to make sympy print output in proper mathematical notation (i.e. using the proper symbol for square root) ?
UPDATE:
when I follow the suggestions from #pqnet:
from sympy import *
x, y, z = symbols('x y z')
init_printing()
init_session()
I get following error:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-23-21d886bf3e54> in <module>()
2 x, y, z = symbols('x y z')
3 init_printing()
----> 4 init_session()
/usr/lib/python2.7/dist-packages/sympy/interactive/session.pyc in init_session(ipython, pretty_print, order, use_unicode, quiet, argv)
154 # and False means don't add the line to IPython's history.
155 ip.runsource = lambda src, symbol='exec': ip.run_cell(src, False)
--> 156 mainloop = ip.mainloop
157 else:
158 mainloop = ip.interact
AttributeError: 'ZMQInteractiveShell' object has no attribute 'mainloop'
In an ipython notebook you can enable Sympy's graphical math typesetting with the init_printing function:
import sympy
sympy.init_printing(use_latex='mathjax')
After that, sympy will intercept the output of each cell and format it using math fonts and symbols. Try:
sympy.sqrt(8)
See also:
Printing section in the Sympy Tutorial.
The simplest way to do it is this:
sympy.pprint(sympy.sqrt(8))
For me (using rxvt-unicode and ipython) it gives
___
2⋅╲╱ 2

scipy curve_fit error: Result from function call is not a proper array of floats

I have an [x,y] dataset and I would like to fit a function to it.
This are x and y
parang = np.array([ 61.1725 , 62.140625, 62.93275 , 63.701625, 65.89225 ,
66.476875, 68.33525 , 68.902375, 72.03975 , 72.590375,
73.144125, 73.670625, 80.36525 , 80.80275 , 87.505375,
87.90375 , 100.557875, 100.8915 ])
q = np.array([-0.03699417, -0.03451252, -0.03851238, -0.0393034 , -0.04059193,
-0.03941371, -0.04206476, -0.04153004, -0.04721763, -0.04667099,
-0.03996427, -0.03872865, -0.05054322, -0.0466561 , -0.05476921,
-0.05274144, -0.0474299 , -0.04974607])
and then I want to fit a function to the data that goes as follows:
def fq(x,bq,cuq):
qval = bq*stndqu[0]*np.cos(np.radians(2*x))+cuq*stndqu[1]*np.sin(np.radians(2*x))
print qval
print qval.dtype
return qval
where 'bq,cuq' are the parameter I need to fit and stndqu are global parameters I obtain as:
stnd = input(r'P ($\%$) and $\theta$ of pol. standard? (as tuple)')
p = stnd[0]/100.
ang = np.radians(stnd[1])
x,y = sympy.symbols('x y')
stndqu = sympy.solve([sympy.sqrt(x**2+y**2)-p,(0.5*sympy.atan(y/x))-ang],[x,y])[1]
and P and theta are 2.73 and 95. The stndqu[0] and stndqu[1] I get out from that block are
0.0272334985720932 and 0.00190435173321445
To find the parameters 'bq' and 'cuq' of the function that fit my data I do:
qpopt,pconv = scio.curve_fit(fq, parang, q)
and here's the result:
[-0.0129614827538107 -0.0137658898997091 -0.0144124082012406
-0.0150294169782742 -0.0167265263727253 -0.0171633151430064
-0.0185034265676582 -0.0188971421096823 -0.0209373417940197
-0.0212701779430718 -0.0215969783128203 -0.0219002154908251
-0.0250793309165333 -0.0252411052388773 -0.0269646924974054
-0.0270214005655701 -0.0260909416985902 -0.0259956074319825]
object
[-0.0129614827538107 -0.0137658898997091 -0.0144124082012406
-0.0150294169782742 -0.0167265263727253 -0.0171633151430064
-0.0185034265676582 -0.0188971421096823 -0.0209373417940197
-0.0212701779430718 -0.0215969783128203 -0.0219002154908251
-0.0250793309165333 -0.0252411052388773 -0.0269646924974054
-0.0270214005655701 -0.0260909416985902 -0.0259956074319825]
object
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
TypeError: array cannot be safely cast to required type
---------------------------------------------------------------------------
error Traceback (most recent call last)
/Users/mj/Documents/NACO/VLT/DataReduction/<ipython-input-57-cac353117232> in <module>()
----> 1 qpopt,pconv = scio.curve_fit(fq, parang, q)
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/optimize/minpack.pyc in curve_fit(f, xdata, ydata, p0, sigma, **kw)
408 # Remove full_output from kw, otherwise we're passing it in twice.
409 return_full = kw.pop('full_output', False)
--> 410 res = leastsq(func, p0, args=args, full_output=1, **kw)
411 (popt, pcov, infodict, errmsg, ier) = res
412
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/optimize/minpack.pyc in leastsq(func, x0, args, Dfun, full_output, col_deriv, ftol, xtol, gtol, maxfev, epsfcn, factor, diag, warning)
268 if (maxfev == 0):
269 maxfev = 200*(n+1)
--> 270 retval = _minpack._lmdif(func,x0,args,full_output,ftol,xtol,gtol,maxfev,epsfcn,factor,diag)
271 else:
272 if col_deriv:
error: Result from function call is not a proper array of floats.
I tried specifying the type of the qval element making it
def fq(x,bq,cuq):
qval = np.array(
bq*stndqu[0]*np.cos(np.radians(2*x))+cuq*stndqu[1]*np.sin(np.radians(2*x)),
dtype=float)
and then the result changes to:
qpopt = scio.curve_fit(fq, parang, q)
[-0.01296148 -0.01376589 -0.01441241 -0.01502942 -0.01672653 -0.01716332
-0.01850343 -0.01889714 -0.02093734 -0.02127018 -0.02159698 -0.02190022
-0.02507933 -0.02524111 -0.02696469 -0.0270214 -0.02609094 -0.02599561]
float64
[-0.01296148 -0.01376589 -0.01441241 -0.01502942 -0.01672653 -0.01716332
-0.01850343 -0.01889714 -0.02093734 -0.02127018 -0.02159698 -0.02190022
-0.02507933 -0.02524111 -0.02696469 -0.0270214 -0.02609094 -0.02599561]
float64
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
TypeError: array cannot be safely cast to required type
---------------------------------------------------------------------------
error Traceback (most recent call last)
/Users/mj/Documents/NACO/VLT/DataReduction/<ipython-input-50-1f4d3764f7ae> in <module>()
----> 1 qpopt = scio.curve_fit(fq, parang, q)
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/optimize/minpack.pyc in curve_fit(f, xdata, ydata, p0, sigma, **kw)
408 # Remove full_output from kw, otherwise we're passing it in twice.
409 return_full = kw.pop('full_output', False)
--> 410 res = leastsq(func, p0, args=args, full_output=1, **kw)
411 (popt, pcov, infodict, errmsg, ier) = res
412
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/optimize/minpack.pyc in leastsq(func, x0, args, Dfun, full_output, col_deriv, ftol, xtol, gtol, maxfev, epsfcn, factor, diag, warning)
268 if (maxfev == 0):
269 maxfev = 200*(n+1)
--> 270 retval = _minpack._lmdif(func,x0,args,full_output,ftol,xtol,gtol,maxfev,epsfcn,factor,diag)
271 else:
272 if col_deriv:
error: Result from function call is not a proper array of floats.
So no progress...
Can someone tell me where is this going wrong?
Thank you very much in advance!
M.
Since stndqu is the result of a call to sympy.solve, it is a symbolic object still. The numbers that you see when printing qval from within your function are probably sympy floats (and thus generic objects to numpy). You should convert stndqu into a numpy array before using it with scipy.curve_fit:
stndqun = numpy.array([sympy.N(i) for i in stndqu])

"LapackError: Parameter a has non-native byte order in lapack_lite.dgesdd" when importing from Matlab files

After importing this data file from Matlab with scipy.io.loadmat, things appeared to work fine until we tried to calculate the conditioning number of one of the matrixes within.
Here's the minimum amount of code that reproduces for us:
import scipy
import numpy
stuff = scipy.io.loadmat("dati-esercizio1.mat")
numpy.linalg.cond(stuff["A"])
Here's the extended stacktrace courtesy of iPython:
In [3]: numpy.linalg.cond(A)
---------------------------------------------------------------------------
LapackError Traceback (most recent call last)
/snip/<ipython-input-3-15d9ef00a605> in <module>()
----> 1 numpy.linalg.cond(A)
/snip/python2.7/site-packages/numpy/linalg/linalg.py in cond(x, p)
1409 x = asarray(x) # in case we have a matrix
1410 if p is None:
-> 1411 s = svd(x,compute_uv=False)
1412 return s[0]/s[-1]
1413 else:
/snip/python2.7/site-packages/numpy/linalg/linalg.py in svd(a, full_matrices, compute_uv)
1313 work = zeros((lwork,), t)
1314 results = lapack_routine(option, m, n, a, m, s, u, m, vt, nvt,
-> 1315 work, -1, iwork, 0)
1316 lwork = int(work[0])
1317 work = zeros((lwork,), t)
LapackError: Parameter a has non-native byte order in lapack_lite.dgesdd
All obvious ideas (like flattening and reshaping the matrix or recreating the matrix from scratch reassigning it element by element) failed. How can I want to massage the data, then, in order to make it more agreeable with numpy?
It's a bug, fixed some time ago: https://github.com/numpy/numpy/pull/235
Workaround:
np.linalg.cond(stuff['A'].newbyteorder('='))
This works for me:
In [33]: stuff = loadmat('dati-esercizio1.mat')
In [34]: a = stuff['A']
In [35]: try: np.linalg.cond(a)
....: except: print "Fail!"
Fail!
In [36]: b = np.array(a, dtype='>d')
In [37]: np.linalg.cond(b)
Out[37]: 62493201976.673141
In [38]: np.all(a == b) # Verify they hold the same data.
Out[38]: True
Apparently it's something wrong with the byte order (endianness?) of each number in the resulting ndarray and not just with the ndarray object itself.
Something like this but more elegant should do the trick:
n, m = A.shape()
B = numpy.empty_like(A)
for i in xrange(n):
for j in xrange(m):
B[i,j] = float(A[i,j])
del A
B = A
print numpy.linalg.cond(A) # 62493210091.354507
(For some reason an in-place replacement still gives that error - so there's something wrong with the byte order of the whole object, too.)