Solving non-linear system of equations in MATLAB - matlab

I encountered a non-linear system of equations that has to be solved.
The system of equations can be written as:
Ax + exp(x) = b with b a known Nx1 matrix, A a known NxN matrix, and x the unknown Nx1 vector for which has to be solved. The exp is defined element-wise on the x vector. I tried to search the MATLAB-manual but I'm having a hard time finding how to solve this kind of equations with MATLAB, so I hope someone can help me out.

You can use Newton-Raphson. Re-arrange your system into a zero residual:
R = A * x + exp(x) - b
Then take the derivative of R with respect to x:
dRdx = A + diag(exp(x))
Then iterate. An example is shown below:
n = 3;
a = rand(n, n);
b = rand(n, 1);
% solve a * x + exp(x) = b for x
x = zeros(n, 1);
for itr = 1: 10
x = x - (a + diag(exp(x))) \ (a * x + exp(x) - b);
end
Of course, you could make this more intelligent by stopping iteration after the residual is small enough.

I would solve it iteratively starting with the solution of the linearized system [A+1]x(0)=b-1 as an initial guess, where 1 is an identity matrix. At the each step of the iterative procedure I would add the exponential of the previous solution at the right-hand side: Ax(j)=b-exp(x(j-1))

I just saw this is a cross post.
This is my solution for the other posting:
The function can be written in the form:
$$ f \left( x \right) = A x + \exp \left( x \right) - b $$
Which is equivalent to the above once a root of $ f \left( x \right) $ is found.
One could use Newton's Method for root finding.
The Jacobian (Like the Transpose of Gradient) of $ f \left( x \right) $ is given by:
$$ J \left( f \left( x \right) \right) = A + diag \left( \exp \left( x \right) \right) $$
Hence the newton iteration is given by:
$$ {x}^{k + 1} = {x}^{k} - { J \left( f \left( {x}^{k} \right) \right) }^{-1} f \left( {x}^{k} \right) $$
You can see the code in my Mathematics Q1462386 GitHub Repository which includes both analytic and numerical derivation of the Jacobian.
This is the result of one run:
Pay attention that while it finds a root for this problem there are more than 1 root hence the solution is one of many and depends on the initial point.

Related

Function and its gradient in Matlab

I am working on a Matlab project and I want to make the gradient of the following function in Matlab:
f(x) = c^T * x - sum (log(bi - (ai ^ T) * x)).
Where ai^T are the rows of a random A matrix nxm , where n=2, and m=20
c is random matrix nx1, and x is also random nx1.
b is random matrix mx1.
I've done the following but the results i get don't seem to be right..
function gc0 = gc(x, c, b, A)
for k = 1 : length(A(:,1))
f1(k) = sum(log(b - A(k,:)'*x(k)));
end
gradient(-f1)
gc0 = c - gradient(f1)';
Any ideas? I'd appreciate your help, I'm newbie in Matlab..
It seems that your loop contains a mistake. Looking at the formula above,
I think that the function evaluation should be
f1 = c'*x;
for k = 1 : length(A(1,:))
f1 = f1 - log(b(k) - A(:,k)'*x)
end
A shorter and faster notation for this in Matlab is
f = c'*x - sum(log(b - A' * x)) ;
The function 'gradient' does not calculate the gradient that I think you
want: it returns the differences of matrix entries, and your function f
is a scalar.
Instead, I suggest calculating the derivatives symbolically:
Gradf = c' + sum( A'./(b - A' * x) );

Solve optimization using fmincon MATLAB when objective function is in constraints

I want to solve:
I use following MATLAB code, but it does not work.
Can someone please guide me?
function f=objfun
f=-f;
function [c1,c2,c3]=constraint(x)
a1=1.1; a2=1.1; a3=1.1;
c1=f-log(a1)-log(x(1)/(x(1)+1));
c2=f-log(a2)-log(x(2)/(x(2)+1))-log(1-x(1));
c3=f-log(a3)-log(1-x(1))-log(1-x(2));
x0=[0.01;0.01];
[x,fval]=fmincon('objfun',x0,[],[],[],[],[0;0],[1;1],'constraint')
You need to flip the problem around a bit. You are trying to find the point x (which is (l_1,l_2)) that makes the minimum of the 3 LHS functions the largest. So, you can rewrite your problem as, in pseudocode,
maximise, by varying x in [0,1] X [0,1]
min([log(a1)+log(x(1)/(x(1)+1)) ...
log(a2)+log(x(2)/(x(2)+1))+log(1-x(1)) ...
log(a3)+log(1-x(1))+log(1-x(2))])
Since Matlab has fmincon, rewrite this as a minimisation problem,
minimise, by varying x in [0,1] X [0,1]
max(-[log(a1)+log(x(1)/(x(1)+1)) ...
log(a2)+log(x(2)/(x(2)+1))+log(1-x(1)) ...
log(a3)+log(1-x(1))+log(1-x(2))])
So the actual code is
F=#(x) max(-[log(a1)+log(x(1)/(x(1)+1)) ...
log(a2)+log(x(2)/(x(2)+1))+log(1-x(1)) ...
log(a3)+log(1-x(1))+log(1-x(2))])
[L,fval]=fmincon(F,[0.5 0.5])
which returns
L =
0.3383 0.6180
fval =
1.2800
Can also solve this in the convex optimization package CVX with the following MATLAB code:
cvx_begin
variables T(1);
variables x1(1);
variables x2(1);
maximize(T)
subject to:
log(a1) + x1 - log_sum_exp([0, x1]) >= T;
log(a2) + x2 - log_sum_exp([0, x2]) + log(1 - exp(x1)) >= T;
log(a3) + log(1 - exp(x1)) + log(1 - exp(x2)) >= T;
x1 <= 0;
x2 <= 0;
cvx_end
l1 = exp(x1); l2 = exp(x2);
To use CVX, each constraint and the objective function has to be written in a way that is proveably convex using CVX's ruleset. Making the substitution x1 = log(l1) and x2 = log(l2) allows one to do that. Note that: log_sum_exp([0,x1]) = log(exp(0) + exp(x1)) = log(1 + l1)
This also returns the answers: l1 = .3383, l2 = .6180, T = -1.2800

Series expansion of a function about infinity - how to return coefficients of series as a Matlab array?

This question is connected to this one. Suppose again the following code:
syms x
f = 1/(x^2+4*x+9)
Now taylor allows the function f to be expanded about infinity:
ts = taylor(f,x,inf,'Order',100)
But the following code
c = coeffs(ts)
produces errors, because the series does not contain positive powers of x (it contains negative powers of x).
In such a case, what code should be used?
Since the Taylor Expansion around infinity was likely performed with the substitution y = 1/x and expanded around 0, I would explicitly make that substitution to make the power positive for use on coeffs:
syms x y
f = 1/(x^2+4x+9);
ts = taylor(f,x,inf,'Order',100);
[c,ty] = coeffs(subs(ts,x,1/y),y);
tx = subs(ty,y,1/x);
The output from taylor is not a multivariate polynomial, so coeffs won't work in this case. One thing you can try is using collect (you may get the same or similar result from using simplify):
syms x
f = 1/(x^2 + 4*x + 9);
ts = series(f,x,Inf,'Order',5) % 4-th order Puiseux series of f about 0
c = collect(ts)
which returns
ts =
1/x^2 - 4/x^3 + 7/x^4 + 8/x^5 - 95/x^6
c =
(x^4 - 4*x^3 + 7*x^2 + 8*x - 95)/x^6
Then you can use numden to extract the numerator and denominator from either c or ts:
[n,d] = numden(ts)
which returns the following polynomials:
n =
x^4 - 4*x^3 + 7*x^2 + 8*x - 95
d =
x^6
coeffs can then be used on the numerator. You may find other functions listed here helpful as well.

Matlab calculate the product of an expression

I'm basicaly trying to find the product of an expression that goes like this:
(x-(N-1)/2).....(x+(N-1)/2) for even value of N
x is a value that I will set at the beginning that changes too but that is a different problem...
let's say for the sake of argument that for now x is a constant (ex x=1)
example for N=6
(x-5/2)(x-3/2)(x-1/2)(x+1/2)(x+3/2)*(x+5/2)
the idea was to create a row vector every element of which is each individual term (P(1)=x-5/2) (P(2)=x-3/2)...etc and then calculate its product
N=6;
x=1;
P=ones(1,N);
for k=(-N-1)/2:(N-1)/2
for n=1:N
P(n)=(x-k);
end
end
y=prod(P);
instead this creates a vector that takes only the first value of the epxression and then
repeats the same value at each cell.
there is obviously a fundamental problem with my loop but I just can't see it.
So if anyone can help with that OR suggest a better way to calculate the product I would be grateful.
Use vectorized commands
Why use a loop when you can use vectorized commands like prod?
y = prod(2 * x + [-N + 1 : 2 : N - 1]) / 2;
For convenience, you may want to define an anonymous function for it:
f = #(N,x) reshape(prod(bsxfun(#plus, 2 * x(:), -N + 1 : 2 : N - 1) / 2, 2), size(x));
Note that the function is compatible with a (row or column) vector input x.
Tests in MATLAB's Command Window
>> f(6, [2,2]')
ans =
-14.7656
4.9219
-3.5156
4.9219
-14.7656
>> f(6, [2,2])
ans =
-14.7656 4.9219 -3.5156 4.9219 -14.7656
Benchmark
Here is a comparison of rayreng's approach versus mine. The former emerges as the clear winner... :'( ...at least as N increases.
Varying N, fixed x
Fixed N (= 10), vector x of varying length
Fixed N (= 100), vector x of varying length
Benchmark code
function benchmark
% varying N, fixed x
clear all
n = logspace(2,4,20)';
x = rand(1000,1);
tr = zeros(size(n));
tj = tr;
for k = 1 : numel(n)
% rayreng's approach (poly/polyval)
fr = #() rayreng(n(k), x);
tr(k) = timeit(fr);
% Jubobs's approach (prod/reshape/bsxfun)
fj = #() jubobs(n(k), x);
tj(k) = timeit(fj);
end
figure
hold on
plot(n, tr, 'bo')
plot(n, tj, 'ro')
hold off
xlabel('N')
ylabel('time (s)')
legend('rayreng', 'jubobs')
end
function y = jubobs(N,x)
y = reshape(prod(bsxfun(#plus,...
2 * x(:),...
-N + 1 : 2 : N - 1) / 2,...
2),...
size(x));
end
function y = rayreng(N, x)
p = poly(linspace(-(N-1)/2, (N-1)/2, N));
y = polyval(p, x);
end
function benchmark2
% fixed N, varying x
clear all
n = 100;
nx = round(logspace(2,4,20));
tr = zeros(size(n));
tj = tr;
for k = 1 : numel(nx)
disp(k)
x = rand(nx(k), 1);
% rayreng's approach (poly/polyval)
fr = #() rayreng(n, x);
tr(k) = timeit(fr);
% Jubobs's approach (prod/reshape/bsxfun)
fj = #() jubobs(n, x);
tj(k) = timeit(fj);
end
figure
hold on
plot(nx, tr, 'bo')
plot(nx, tj, 'ro')
hold off
xlabel('number of elements in vector x')
ylabel('time (s)')
legend('rayreng', 'jubobs')
title(['n = ' num2str(n)])
end
function y = jubobs(N,x)
y = reshape(prod(bsxfun(#plus,...
2 * x(:),...
-N + 1 : 2 : N - 1) / 2,...
2),...
size(x));
end
function y = rayreng(N, x)
p = poly(linspace(-(N-1)/2, (N-1)/2, N));
y = polyval(p, x);
end
An alternative
Alternatively, because the terms in your product form an arithmetic progression (each term is greater than the previous one by 1/2), you can use the formula for the product of an arithmetic progression.
I agree with #Jubobs in that you should avoid using for loops for this kind of computation. There are cases where for loops perform fast, but for something as simple as this, avoid using loops if possible.
An alternative approach to what Jubobs has suggested is that you can consider that polynomial equation to be in factored form where each factor denotes a root located at that particular location. You can use poly to convert these factors into a polynomial equation, then use polyval to evaluate the expression at the point you want. First, generate your roots by linspace where the points vary from -(N-1)/2 to (N-1)/2 and there are N of them, then plug this into poly. Finally, for any values of x, put this into polyval with the output of poly. The advantage of this approach is that you can evaluate multiple points of x in a single sweep.
Going with what you have, you would simply do this:
p = poly(linspace(-(N-1)/2, (N-1)/2, N));
out = polyval(p, x);
With your example, supposing that N = 6, this would be the output of the first line:
p =
1.0000 0 -8.7500 0 16.1875 0 -3.5156
As such, this is saying that when we expand out (x-5/2)(x-3/2)(x-1/2)(x+1/2)(x+3/2)(x+5/2), we get:
x^6 - 8.75x^4 + 16.1875x^2 - 3.5156
If we take a look at the roots of this equation, this is what we get:
r = roots(p)
r =
-2.5000
2.5000
-1.5000
1.5000
-0.5000
0.5000
As you can see, each term corresponds to one factor in your polynomial equation, so we do have the right mindset here. Now, all you have to do is use p with your values of x into polyval to obtain your results. For example, if I wanted to evaluate that polynomial from -2 <= x <= 2 where x is an integer, this is the result I get:
polyval(p, -2:2)
ans =
-14.7656 4.9219 -3.5156 4.9219 -14.7656
Therefore, when x = -2, the result is -14.7656 and so on.
Though I would recommend the solution by #Jubobs, it is also good to check what the issue is with your loop.
The first indication that something is wrong, is that you have a nested loop over 2 variables, and only index with one of them to store the result. Probably you just need a single loop.
Here is a loop that you may be interested in that should do roughly what you need:
N=6;
x=1;
k=(-N-1)/2:(N-1)/2
P = ones(size(k));
for n=1:numel(k)
P(n)=(x-k(n));
end
y=prod(P);
I tried to keep the code close to the original, so hopefully it is easy to understand.

how to solve a system of Ordinary Differential Equations (ODE's) in Matlab

I have to solve a system of ordinary differential equations of the form:
dx/ds = 1/x * [y* (g + s/y) - a*x*f(x^2,y^2)]
dy/ds = 1/x * [-y * (b + y) * f()] - y/s - c
where x, and y are the variables I need to find out, and s is the independent variable; the rest are constants. I've tried to solve this with ode45 with no success so far:
y = ode45(#yprime, s, [1 1]);
function dyds = yprime(s,y)
global g a v0 d
dyds_1 = 1./y(1) .*(y(2) .* (g + s ./ y(2)) - a .* y(1) .* sqrt(y(1).^2 + (v0 + y(2)).^2));
dyds_2 = - (y(2) .* (v0 + y(2)) .* sqrt(y(1).^2 + (v0 + y(2)).^2))./y(1) - y(2)./s - d;
dyds = [dyds_1; dyds_2];
return
where #yprime has the system of equations. I get the following error message:
YPRIME returns a vector of length 0, but the length of initial
conditions vector is 2. The vector returned by YPRIME and the initial
conditions vector must have the same number of elements.
Any ideas?
thanks
Certainly, you should have a look at your function yprime. Using some simple model that shares the number of differential state variables with your problem, have a look at this example.
function dyds = yprime(s, y)
dyds = zeros(2, 1);
dyds(1) = y(1) + y(2);
dyds(2) = 0.5 * y(1);
end
yprime must return a column vector that holds the values of the two right hand sides. The input argument s can be ignored because your model is time-independent. The example you show is somewhat difficult in that it is not of the form dy/dt = f(t, y). You will have to rearrange your equations as a first step. It will help to rename x into y(1) and y into y(2).
Also, are you sure that your global g a v0 d are not empty? If any one of those variables remains uninitialized, you will be multiplying state variables with an empty matrix, eventually resulting in an empty vector dyds being returned. This can be tested with
assert(~isempty(v0), 'v0 not initialized');
in yprime, or you could employ a debugging breakpoint.
the syntax for ODE solvers is [s y]=ode45(#yprime, [1 10], [2 2])
and you dont need to do elementwise operation in your case i.e. instead of .* just use *