I have been trying to solve the following system of equations using a scipy solver:
from scipy.optimize import fsolve
import math
import numpy as np
S0 = 1000
u = 1
d = 1
delta_t = 1
r = 0
psi_r = (np.exp(-r*delta_t))**(-1)
fi_r = np.exp(-r*delta_t)
fi_2r = np.exp(-2*r*delta_t)
def equations(p):
p_u,p_m,p_d = p
return (p_u+p_m+p_d - 1, fi_r*(p_u*(S0+u) + p_m*S0 + p_u*(S0-d)) -S0,fi_2r*
(p_u*p_u*(S0+2*u) + 2*p_m*p_u*(S0+u) + (2*p_u*p_d+p_m*p_m)*S0 + 2*p_m*p_d*(S0-d) + p_d*p_d*(S0-2*d)) - S0)
p_u,p_m,p_d = fsolve(equations,(0.3,0.5,0.4))
print(equations((p_u,p_m,p_d)))
The problem is that, despite the first equation stating that the sum of my unknowns should be 1, it never gives a result that would satisfy this. What I get is unexpected numbers on the order of 10 to the -12, or sometimes even negative numbers, which I know cannot be the correct solution.
I know I have to try several initial guesses, but what concerns me is that none of the guesses so far has given me probabilities which sum up to 1.
In general, when a system has multiple solutions, the solution returned by a numerical solver depends on the initial solution estimate as well as the algorithm implemented. However, in general, it is difficult to "control" the solution properties returned by the solver.
In your case, you require a solution where all its elements lie within the interval [0,1]. However, the fsolve algorithm converges to a solution that does not satisfy this property. Unfortunately, fsolve does not allow for imposing any constraints on the solution it returns (as is also the case for any other numerical equation solver, to the best of my knowledge).
A workaround for imposing constraints on the solution is to formulate the equation solving problem as a constrained optimization problem. One straightforward approach is the following.
Suppose you need to solve the two equations [f(x)==0, g(x)==0] with respect to the vector x. Note that any solution of the system is also a minimizer of the function f(x)**2+g(x)**2, which may be obtained by calling scipy.optimize.least_squares. Now, scipy.optimize.least_squares accepts bounds for the solution it returns, which essentially allow you to find a solution of the original system of equation with the desired bound properties.
from scipy.optimize import least_squares
sol = least_squares(equations,(0.3,0.5,0.4), bounds = (0,1))
print('solution: ', sol.x, '\n', 'equations:', equations(sol.x))
solution: [ 0.295 0.4101 0.295 ]
equations: (0.0, 0.0, 0.0)
Note that the returned solution is indeed a probability distribution and a root for the system of equations.
Related
I am trying to minimize a highly non-linear function by optimizing three unknown parameters a, b, and c0. I'm attempting to replicate some governing equations of a casino roulette ball in Python 3.
Here is the link to the research paper:
http://www.dewtronics.com/tutorials/roulette/documents/Roulette_Physik.pdf
I will be referencing equations (35) and (40) in the paper.
Basically, I take stopwatch lap measurements of the roulette ball spinning on the wheel. For each successive lap, the lap time will increase because of losses of momentum to non-conservative forces of friction. Then I take these time measurements and fit equation (35) using a Levenberg-Marquardt least squares method in equation (40).
My question is twofold:
(1) I'm using the scipy.optimize.least_squares() method='lm', and I'm not sure how to write the objective function! Right now I have the function written exactly as is in the paper:
def fall_time(k,a,b,c0):
F = (1 / (a * b)) * (c0 - np.arcsinh(c0) * np.exp(a * k * 2 * np.pi))
return F
def parameter_estimation_function(x0,tk):
a = x0[0]
b = x0[1]
c0 = x0[2]
S = 0
for i,t in enumerate(tk):
k = i + 1
S += (t - fall_time(k,a,b,c0))**2
return [S,1,1]
sol = least_squares(parameter_estimation_function,[0.1,0.8,-0.1],args=([tk1]),method='lm',jac='2-point',max_nfev=2000)
print(sol)
Now, in the documentation examples, I never saw the objective function written the way I have it. In the documentation, the objective function is always returns the residual, not the square of the residual. Additionally, in the documentation they never use the sum! So I'm wondering if the sum and the square are automatically handled under the hood of least_squares()?
(2) Perhaps my second question is a result of my failure to understand how to write the objective function. But anyhow, I'm having trouble getting the algorithm to converge on the minimum. I know this is because the levenberg alogrithm is "greedy" and stops near the closest minima, but I figured that I would be able to at least converge on about the same result given different initial guesses. With slight alterations in the initial guess, I'm getting parameter results with different signs. Additionally, I've yet to find a combination of initial guesses that allows the algo to converge! It always times out before it finds the solution. I've even increased the amount of function evaluations to 10,000 to see if it would. To no avail!
Perhaps somebody could shed some light on my mistakes here! I'm still relatively new to python and the scipy library!
Here is some sample data for tk that I've measured myself from the video here: https://www.youtube.com/watch?v=0Zj_9ypBnzg
tk = [0.52,1.28,2.04,3.17,4.53,6.22]
tk1 = [0.51,1.4,2.09,3,4.42,6.17]
tk2 = [0.63,1.35,2.19,3.02,4.57,6.29]
tk3 = [0.63,1.39,2.23,3.28,4.70,6.32]
tk4 = [0.57,1.4,2.1,3.06,4.53,6.17]
Thanks
1) Yes, as you suspected the sum and the square of the residuals are automatically handled.
2) Hard to say, since I'm not deeply familiar with the problem (e.g., how many local minima exist, what constitutes a 'reasonable' result, etc.). I may investigate more later.
But for kicks I fiddled with some of the values to see what would happen. For example, you can just replace the 1/b constant with a standalone variable b_inv, and this seemed to stabilize the results quite a bit. Here's the code I used to check results. (Note that I rewrote the objective function for brevity. It simply leverages the element-wise operations of numpy arrays, without changing the overall result.)
import numpy as np
from scipy.optimize import least_squares
def fall_time(k,a,b_inv,c0):
return (b_inv / a) * (c0 - np.arcsinh(c0) * np.exp(a * k * 2 * np.pi))
def parameter_estimation_function(x,tk):
return np.asarray(tk) - fall_time(k=np.arange(1,len(tk)+1), a=x[0],b_inv=x[1],c0=x[2])
tk_samples = [
[0.52,1.28,2.04,3.17,4.53,6.22],
[0.51,1.4,2.09,3,4.42,6.17],
[0.63,1.35,2.19,3.02,4.57,6.29],
[0.63,1.39,2.23,3.28,4.70,6.32],
[0.57,1.4,2.1,3.06,4.53,6.17]
]
for i in range(len(tk_samples)):
sol = least_squares(parameter_estimation_function,[0.1,1.25,-0.1],
args=(tk_samples[i],),method='lm',jac='2-point',max_nfev=2000)
print(sol.x)
with console output:
[ 0.03621789 0.64201913 -0.12072879]
[ 3.59319972e-02 1.17129458e+01 -6.53358716e-03]
[ 3.55516005e-02 1.48491493e+01 -5.31098257e-03]
[ 3.18068316e-02 1.11828091e+01 -7.75329834e-03]
[ 3.43920725e-02 1.25160378e+01 -6.36307506e-03]
Actually I have to calculate values of 3 variables from probably 8 or 9 non-linear equations(may be more for accuracy).
I was using lsqnonlin and fsolve.
Using lsqnonlin, it says solver stopped prematurely (mainly due to value of iteration, funEvals and tolerance) and the output is far away from exact solution. I tried but I don't know on what basis I should set those parameters.
Using fsolve, it says no solution found.
I also used LMFnlsq and LMFsolve but it gives the output nowhere near the exact solution? I tried to change other parameters too but I could not bring those solutions to my desired values.
Is there any other way to solve these overdetermined non-linear equations?
My code till now:
x0 = [20 40 275];
eqn = #(x)[((((x(1)-Sat(1,1))^2+(x(2)-Sat(1,2))^2+(x(3)-Sat(1,3))^2))-dis(1)^2);
((((x(1)-Sat(2,1))^2+(x(2)-Sat(2,2))^2+(x(3)-Sat(2,3))^2))-dis(2)^2);
((((x(1)-Sat(3,1))^2+(x(2)-Sat(3,2))^2+(x(3)-Sat(3,3))^2))- dis(3)^2);
((((x(1)-Sat(4,1))^2+(x(2)-Sat(4,2))^2+(x(3)-Sat(4,3))^2))- dis(4))^2;
((((x(1)-Sat(5,1))^2+(x(2)-Sat(5,2))^2+(x(3)-Sat(5,3))^2))- dis(5))^2;
((((x(1)-Sat(6,1))^2+(x(2)-Sat(6,2))^2+(x(3)-Sat(6,3))^2))- dis(6))^2;
((((x(1)-Sat(7,1))^2+(x(2)-Sat(7,2))^2+(x(3)-Sat(7,3))^2))- dis(7))^2;
((((x(1)-Sat(8,1))^2+(x(2)-Sat(8,2))^2+(x(3)-Sat(8,3))^2))- dis(8))^2;
((((x(1)-Sat(9,1))^2+(x(2)-Sat(9,2))^2+(x(3)-Sat(9,3))^2))- dis(9))^2;
((((x(1)-Sat(10,1))^2+(x(2)-Sat(10,2))^2+(x(3)-Sat(10,3))^2))- dis(10))^2];
lb = [0 0 0];
ub = [100 100 10000];
options = optimoptions('lsqnonlin','MaxFunEvals',3000,'MaxIter',700,'TolFun',1e-18);%,'TolX',1);
x= lsqnonlin(eqn,x0,lb,ub,options)
**Error:**
**Solver stopped prematurely.**
lsqnonlin stopped because it exceeded the iteration limit,
options.MaxIter = 700 (the selected value).
x = 20.349 46.633 9561.5
Hoping for some suggestions!
Thanks in advance!
I usually model this explicitly:
min w'w
f_i(x) = w_i
w is a free variable
L<=x<=U
It should be easy to calculate a feasible (but non-optimal) solution in advance. If you can find a "good" initial solution that would be even better. Then use a general purpose NLP solver (e.g. fmincon) and pass on your initial feasible solution (both x and w). The best thing is to use a modeling system that allows automatic differentiation. Otherwise you should provide correct and precise gradients (and if needed second derivatives). See also the advice here.
I wanted to solve the following equation in MATLAB R2013a using the Symbolic Math Toolbox.
(y/x)-(((1+r)^n)-1)/r=0 where y,x and n>3 are given and r is the dependent variable
I tried myself & coded as follows:
f=solve('(y/x)-(((1+r)^n)-1)/r','r')
but as the solution for r is not exact i.e. it is converging on successive iterations hence MATLAB is giving a warning output with the message
Warning: Explicit solution could not be found.
f =
[ empty sym ]
How do I code this?
There are an infinite number of solutions to this for an unspecified value of n > 3 and unknown r. I hope that it's pretty clear why – it's effectively asking for a greater and greater number of roots of (1+r)^n. You can find solutions for fixed values of n, however. Note that as n becomes larger there are more and more solutions and of course some of them are complex. I'm going to assume that you're only interested in real values of r. You can use solve and symbolic math for n = 4, n = 5, and n = 6 (for n = 6, the solution may not be in a convenient form):
y = 441361;
x = 66990;
n = 5;
syms r;
rsol = solve(y/x-((1+r)^n-1)/r==0,r,'IgnoreAnalyticConstraints',true)
double(rsol)
However, the question is "do you need all the solutions or just a particular solution for a given value of n"? If you just need a particular solution, you shouldn't be using symbolic math at all as it's slower and has practical issues like the ones you're experiencing. You can instead just use a numerical approach to find a zero of the equation that is near a specified initial guess. fzero is the standard function for solving this sort of problem in a single variable:
y = 441361;
x = 66990;
n = 5;
f = #(r)y/x-((1+r).^n-1)./r;
r0 = 1;
rsol = fzero(f,r0)
You'll see that the value returned is the same as one of the solutions from the symbolic solution above. If you adjust the initial guess r0 (say r0 = -3), it will return the other solution. When using numeric approaches in cases when there are multiple solutions, if you want specific solutions you'll need to know about the behavior of your function and you'll need to add some clever extra code to choose initial guesses.
I think you forgot to define n as well.
f=solve('(y/x)-(((1+r)^n)-1)/r=0','n-3>0','r','n')
Should solve your problem :)
The following command
syms x real;
f = #(x) log(x^2)*exp(-1/(x^2));
fp(x) = diff(f(x),x);
fpp(x) = diff(fp(x),x);
and
solve(fpp(x)>0,x,'Real',true)
return the result
solve([0.0 < (8.0*exp(-1.0/x^2))/x^4 - (2.0*exp(-1.0/x^2))/x^2 -
(6.0*log(x^2)*exp(-1.0/x^2))/x^4 + (4.0*log(x^2)*exp(-1.0/x^2))/x^6],
[x == RD_NINF..RD_INF])
which is not what I expect.
The first question: Is it possible to force Matlab's solve to return the set of all solutions?
(This is related to this question.) Moreover, when I try to solve the equation
solve(fpp(x)==0,x,'Real',true)
which returns
ans =
-1.5056100417680902125994180096313
I am not satisfied since all solutions are not returned (they are approximately -1.5056, 1.5056, -0.5663 and 0.5663 obtained from WolframAlpha).
I know that vpasolve with some initial guess can handle this. But, I have no idea how I can generally find initial guessed values to obtain all solutions, which is my second question.
Other solutions or suggestions for solving these problems are welcomed.
As I indicated in my comment above, sym/solve is primarily meant to solve for analytic solutions of equations. When this fails, it tries to find a numeric solution. Some equations can have an infinite number of numeric solutions (e.g., periodic equations), and thus, as per the documentation: "The numeric solver does not try to find all numeric solutions for [the] equation. Instead, it returns only the first solution that it finds."
However, one can access the features of MuPAD from within Matlab. MuPAD's numeric::solve function has several additional capabilities. In particular is the 'AllRealRoots' option. In your case:
syms x real;
f = #(x)log(x^2)*exp(-1/(x^2));
fp(x) = diff(f(x),x);
fpp(x) = diff(fp(x),x);
s = feval(symengine,'numeric::solve',fpp(x)==0,x,'AllRealRoots')
which returns
s =
[ -1.5056102995536617698689500437312, -0.56633904710786569620564475006904, 0.56633904710786569620564475006904, 1.5056102995536617698689500437312]
as well as a warning message.
My answer to this question provides other way that various MuPAD solvers can be used, particularly if you can isolate and bracket your roots.
The above is not going to directly help with your inequalities other than telling you where the function changes sign. For those you could try:
s = feval(symengine,'solve',fpp(x)>0,x,'Real')
which returns
s =
(Dom::Interval(0, Inf) union Dom::Interval(-Inf, 0)) intersect solve(0 < 2*log(x^2) - 3*x^2*log(x^2) + 4*x^2 - x^4, x, Real)
Try plotting this function along with fpp.
While this is not a bug per se, The MathWorks still might be interested in this difference in behavior and poor performance of sym/solve (and the underlying symobj::solvefull) relative to MuPAD's solve. File a bug report if you like. For the life of me I don't understand why they can't better unify these parts of Matlab. The separation makes not sense from the perspective of a user.
I am trying to use solve() to solve a system of equations of the following form
eq1=a1x+a2y;
eq2=b1x+b2y;
where a1 = .05 for values of x<5, .1 for values of 5
Is there a way to solve for this using solve? As in sol = solve(eq1,eq2);
I'm not sure what you're trying to do here. Can you please post a real example (with numbers) and what you would like the output to be?
I think you're trying to solve linear simultaeneous equations. Assuming that is what you are trying to do:
I would suggest multiplying all of your equations by 20, so that your minimum quanta size of 0.05 becomes 1.00. Your problem then becomes the solution of linear equations for integer values.
Note that if the system is fully constrained (that is, if there are n independent constraints on the n equations you want to solve) then there will only be one solution and it may not necessarily be an integer solution. For example the system:
1 = 2a + 4b
3 = a + b
has the solution a = 5.5, b = -2.5. No other solution is possible.
For under-constrained systems, i.e.
0 = 3x + y
x > 0
Then there will be an infinite number of solutions, some of which may have both x and y being integer values. (Or there may be no integer solutions at all.)
Okay let me give you a quick rundown.
if you want to solve an equation or a system of equations and conditions then you need to define them as such, so let me explain.
so by example
clear all; %just to be safe
syms x y b
a=0.5;
somevalue=1;
someothervalue=3;
eq1= a*x+a*y == somevalue; %this is your first equation
eq2= b*x+b*y == someothervalue; %this is your 2nd equation
cond1= x<5; %this is a condition which matlab sees as an "equation"
eqs=[eq1,eq2,cond1]; %these are the equations and conditions you want to solve for, use this for solve
eqs=[eq1,eq2]; %use this for vpasolve and set your condition in range
vars=[x,y,b]; %these are the variable you want to solve for
range = [-Inf 5; NaN NaN; NaN NaN]; %NaN means you set no range
%you can use solve or vpasolve, second one being numeric, which is the one you'll probably want
n=5;
sol=zeros(n,numel(vars));
for i = 1:n
temp1 = vpasolve(eqs, vars, range, 'random', true);
temp = vpasolve(eqs, vars, 'random', true);
sol(i,1) = temp.x;
sol(i,2) = temp.y;
sol(i,3) = temp.b;
end
sol
Now when I run this myself I can't get the range to properly work for some reason, still trying to figure that out. When you don't set a range it works just fine, if you can use the solve function then there also isn't a problem.
In theory the range function should work fine like this so it might be a bug on my end.
If you use solve you have some nice options where you can use assume to set extra conditions that are a bit more advanced, like only checking for real solutions or only integers, etc.