I have 3 data matrices A,B,C (all 3x3), with which I use the following code to calculate roots of D(p)X = 0
syms p
D = A + B*p + C*(p^2)
solP = double(solve(det(D)))
From this, I get 6 values for solP. But when I try substituting it back into the symbolic matrix D, as follows, I get non-zero values of det(D) sometimes
for i = 1:6
p = solP(i)
det(double(subs(D)) % Should be zero always as we are substituting roots
end
Please help me understand this behaviour.
EDIT ::
Example :
A =
1.0e+11 *
4.8976 7.0936 6.7970
4.4559 7.5469 6.5510
6.4631 2.7603 1.6261
B =
1.0e+11 *
3.9223 7.0605 0.4617
6.5548 0.3183 0.9713
1.7119 2.7692 8.2346
C =
1.0e+11 *
6.9483 0.3445 7.6552
3.1710 4.3874 7.9520
9.5022 3.8156 1.8687
solP =
0.1061 + 0.0000i
1.5311 + 0.0000i
-0.3432 + 0.9356i
-0.3432 - 0.9356i
0.4228 - 0.5465i
0.4228 + 0.5465i
det(D) =
2.2143e+19
-5.4911e+20
-8.6415e+19 + 4.5024e+19i
-8.6415e+19 - 4.5024e+19i
-1.4547e+19 + 9.1135e+19i
-1.4547e+19 - 9.1135e+19i
The problem is related to the relative accuracy of floating point values, typically 1e-16.
The input matrices are of the order 1e+11 - 1e+12, the solution is of the order 1e+0, so the elements of D are also of the order 1e+11 - 1e+12. To calculate a determinant of a 3x3 matrix, one should take products of three matrix elements and add/subtract them. So, each term is of the order of 1e+33 - 1e+36. If you subtract such a values to obtain the determinant, the expected accuracy is in the order of 1e+17 - 1e+20. Indeed, this corresponds with the values you get. Given the relative accuracy, you are not able to reach further to zero.
Note that if you scale your input matrices, i.e. divide it by 1e+11, the solutions are indeed the same, but the determinants are probably more what you would expect.
Related
I want to check the calculation of the laplace filter from scipy.ndimage and compare it to my own method if differentiation. Below I have a piece of code that I ran
import scipy.ndimage.filters
n = 100
x_range = y_range = np.linspace(-1, 1, n)
X, Y = np.meshgrid(x_range, y_range)
f_fun = X ** 2 + Y ** 2
f_fun_laplace = 4 * np.ones(f_fun.shape)
res_laplace = scipy.ndimage.filters.laplace(f_fun, mode='constant')
I expect that the variable res_laplace will have the constant value of 4 over the whole domain (excluding the boundaries for simplicity), since this is what I would get by applying the laplace operator to my function f(x,y) = x^2 + y^2.
However, the value that res_laplace produces is 0.00163 in this case. So my question was.. why is this not equal to 4?
The answer, in this specific case, is that you need to multiply the output of scipy.ndimage.filters.laplace with a factor of (1/delta_x) ** 2. Where delta_x = np.diff(x_range)[0]
I simply assumed that the filter would take care of that, but in hindsight it is of course not able know the value delta_x.
And since we are differentiating twice, we need to square the inverse of this delta_x.
I'm given the a(k) matrix and e(n) and I need to compute y(n) from the following eq:
y(n) = sum(k = 1 to 10 )( a(k)*y(n-k) ) + e(n).
I have the a matrix(filter coefficients) and the e matrix (the residual), therefore there is only 1 unknown: y, which is built by the previous 10 samples of y.
An example to this equation:
say e(0) (my first residual sample) = 3
and y(-10) to y(-1) = 0
then y(0), my first sample in the signal y, would just be e(0) = 3:
y(0) = a(1)*y(-1) + a(2)*y(-2) + .... + a(10)*y(-10) + e(0) = e(0) = 3
and if e(1) = 4, and a(1) = 5, then
y(1) = a(1)*y(0) + a(2)*y(-1) + a(3)&y(-2) + ... + a(10)*y(-9) + e(1) = 19
The problem is
I don't know how to do this without loops because, say, y(n) needs y(n-1), so I need to immediately append y(n-1) into my matrix in order to get y(n).
If n (the number of samples) = say, 10,000,000, then using a loop is not ideal.
What I've done so far
I have not implemented anything. The only thing I've done for this particular problem is research on what kind of Matlab functions I could use.
What I need
A Matlab function that, given an equation and or input matrix, computes the next y(n) and automatically appends that to the input matrix, and then computes the next y(n+1), and automatically append that to the input matrix and so on.
If there is anything regarding my approach
That seems like it's on the wrong track, or my question isn't clear enough, of if there is no such matlab function that exists, then I apologize in advance. Thank you for your time.
I would like to solve the following equation: tan(x) = 1/x
What I did:
syms x
eq = tan(x) == 1/x;
sol = solve(eq,x)
But this gives me only one numerical approximation of the solution. After that I read about the following:
[sol, params, conds] = solve(eq, x, 'ReturnConditions', true)
But this tells me that it can't find an explicit solution.
How can I find numerical solutions to this equation within some given range?
I've never liked using solvers "blindly", that is, without some sort of decent initial value selection scheme. In my experience, the values you will find when doing things blindly, will be without context as well. Meaning, you'll often miss solutions, think something is a solution while in reality the solver exploded, etc.
For this particular case, it is important to realize that fzero uses numerical derivatives to find increasingly better approximations. But, derivatives for f(x) = x · tan(x) - 1 get increasingly difficult to accurately compute for increasing x:
As you can see, the larger x becomes, the better f(x) approximates a vertical line; fzero will simply explode! Therefore it is imperative to get an estimate as closely to the solution as possible before even entering fzero.
So, here's a way to get good initial values.
Consider the function
f(x) = x · tan(x) - 1
Knowing that tan(x) has Taylor expansion:
tan(x) ≈ x + (1/3)·x³ + (2/15)·x⁵ + (7/315)·x⁷ + ...
we can use that to approximate the function f(x). Truncating after the second term, we can write:
f(x) ≈ x · (x + (1/3)·x³) - 1
Now, key to realize is that tan(x) repeats with period π. Therefore, it is most useful to consider the family of functions:
fₙ(x) ≈ x · ( (x - n·π) + (1/3)·(x - n·π)³) - 1
Evaluating this for a couple of multiples and collecting terms gives the following generalization:
f₀(x) = x⁴/3 - 0π·x³ + ( 0π² + 1)x² - (0π + (0π³)/3)·x - 1
f₁(x) = x⁴/3 - 1π·x³ + ( 1π² + 1)x² - (1π + (1π³)/3)·x - 1
f₂(x) = x⁴/3 - 2π·x³ + ( 4π² + 1)x² - (2π + (8π³)/3)·x - 1
f₃(x) = x⁴/3 - 3π·x³ + ( 9π² + 1)x² - (3π + (27π³)/3)·x - 1
f₄(x) = x⁴/3 - 4π·x³ + (16π² + 1)x² - (4π + (64π³)/3)·x - 1
⋮
fₙ(x) = x⁴/3 - nπ·x³ + (n²π² + 1)x² - (nπ + (n³π³)/3)·x - 1
Implementing all this in a simple MATLAB test:
% Replace this with the whole number of pi's you want to
% use as offset
n = 5;
% The coefficients of the approximating polynomial for this offset
C = #(npi) [1/3
-npi
npi^2 + 1
-npi - npi^3/3
-1];
% Find the real, positive polynomial roots
R = roots(C(n*pi));
R = R(imag(R)==0);
R = R(R > 0);
% And use these as initial values for fzero()
x_npi = fzero(#(x) x.*tan(x) - 1, R)
In a loop, this can produce the following table:
% Estimate (polynomial) Solution (fzero)
0.889543617524132 0.860333589019380 0·π
3.425836967935954 3.425618459481728 1·π
6.437309348195653 6.437298179171947 2·π
9.529336042900365 9.529334405361963 3·π
12.645287627956868 12.645287223856643
15.771285009691695 15.771284874815882
18.902410011613000 18.902409956860023
22.036496753426441 22.036496727938566 ⋮
25.172446339768143 25.172446326646664
28.309642861751708 28.309642854452012
31.447714641852869 31.447714637546234
34.586424217960058 34.586424215288922 11·π
As you can see, the approximant is basically equal to the solution. Corresponding plot:
To find a numerical solution to a function within some range, you can use fzero like this:
fun = #(x)x*tan(x)-1; % Multiplied by x so fzero has no issue evaluating it at x=0.
range = [0 pi/2];
sol = fzero(fun,range);
The above would return just one solution (0.8603). If you want additional solutions, you will have to call fzero more times. This can be done, for example, in a loop:
fun = #(x)tan(x)-1/x;
RANGE_START = 0;
RANGE_END = 3*pi;
RANGE_STEP = pi/2;
intervals = repelem(RANGE_START:RANGE_STEP:RANGE_END,2);
intervals = reshape(intervals(2:end-1),2,[]).';
sol = NaN(size(intervals,1),1);
for ind1 = 1:numel(sol)
sol(ind1) = fzero(fun, mean(intervals(ind1,:)));
end
sol = sol(~isnan(sol)); % In case you specified more intervals than solutions.
Which gives:
[0.86033358901938;
1.57079632679490; % Wrong
3.42561845948173;
4.71238898038469; % Wrong
6.43729817917195;
7.85398163397449] % Wrong
Note that:
The function is symmetric, and so are its roots. This means you can solve on just the positive interval (for example) and get the negative roots "for free".
Every other entry in sol is wrong because this is where we have asymptotic discontinuities (tan transitions from +Inf to -Inf), which is mistakenly recognized by MATLAB as a solution. So you can just ignore them (i.e. sol = sol(1:2:end);.
Multiply the equation by x and cos(x) to avoid any denominators that can have the value 0,
f(x)=x*sin(x)-cos(x)==0
Consider the normalized function
h(x)=(x*sin(x)-cos(x)) / (abs(x)+1)
For large x this will be increasingly close to sin(x) (or -sin(x) for large negative x). Indeed, plotting this this is already visually true, up to an amplitude factor, for x>pi.
For the first root in [0,pi/2] use the Taylor approximation at x=0 of second degree x^2-(1-0.5x^2)==0 to get x[0]=sqrt(2.0/3) as root approximation, for the higher ones take the sine roots x[n]=n*pi, n=1,2,3,... as initial approximations in the Newton iteration xnext = x - f(x)/f'(x) to get
n initial 1. Newton limit of Newton
0 0.816496580927726 0.863034004302817 0.860333589019380
1 3.141592653589793 3.336084918413964 3.425618459480901
2 6.283185307179586 6.403911810682199 6.437298179171945
3 9.424777960769379 9.512307014150883 9.529334405361963
4 12.566370614359172 12.635021895208379 12.645287223856643
5 15.707963267948966 15.764435036320542 15.771284874815882
6 18.849555921538759 18.897518573777646 18.902409956860023
7 21.991148575128552 22.032830614521892 22.036496727938566
8 25.132741228718345 25.169597069842926 25.172446326646664
9 28.274333882308138 28.307365162331923 28.309642854452012
10 31.415926535897931 31.445852385744583 31.447714637546234
11 34.557519189487721 34.584873343220551 34.586424215288922
I have a 30x30 matrix as a base matrix (OD_b1), I also have two base vectors (bg and Ag). My aim is to optimize a matrix (X) who's dimensions are 30X30 such that:
1) the squared difference between vector (bg) and vector of sum of all the columns is minimized.
2)the squared difference between vector (Ag) and vector of sum of all rows is minimized.
3)the squared difference between the elements of matrix (X) and matrix (OD_b1) is minimized.
The mathematical form of the equation is as follows:
I have tried this:
fun=#(X)transpose(bg-sum(X,2))*(bg-sum(X,2))+ (Ag-sum(X,1))*transpose(Ag-sum(X,1))+sumsqr(X_b-X);
[val,X]=fmincon(fun,OD_b1,AA,BB,Aeq,beq,LB,UB)
I don't get errors but it seems like it's stuck.
Is it because I have too many variables or is there another reason?
Thanks in advance
This is a simple, unconstrained least squares problem and hence has a simple solution that can be expressed as the solution to a linear system.
I will show you (1) the precise and efficient way to solve this and (2) how to solve with fmincon.
The precise, efficient solution:
Problem setup
Just so we're on the same page, I initialize the variables as follows:
n = 30;
Ag = randn(n, 1); % observe the dimensions
X_b = randn(n, n);
bg = randn(n, 1);
The code:
A1 = kron(ones(1,n), eye(n));
A2 = kron(eye(n), ones(1,n));
A = (A1'*A1 + A2'*A2 + eye(n^2));
b = A1'*bg + A2'*Ag + X_b(:);
x = A \ b; % solves A*x = b
Xstar = reshape(x, n, n);
Why it works:
I first reformulated your problem so the objective is a vector x, not a matrix X. Observe that z = bg - sum(X,2) is equivalent to:
x = X(:) % vectorize X
A1 = kron(ones(1,n), eye(n)); % creates a special matrix that sums up
% stuff appropriately
z = A1*x;
Similarly, A2 is setup so that A2*x is equivalent to Ag'-sum(X,1). Your problem is then equivalent to:
minimize (over x) (bg - A1*x)'*(bg - A1*x) + (Ag - A2*x)'*(Ag - A2*x) + (y - x)'*(y-x) where y = Xb(:). That is, y is a vectorized version of Xb.
This problem is convex and the first order condition is a necessary and sufficient condition for the optimum. Take the derivative with respect to x and that equation will define your solution! Sample example math for almost equivalent (but slightly simpler problem is below):
minimize(over x) (b - A*x)'*(b - A*x) + (y - x)' * (y - x)
rewriting the objective:
b'b- b'Ax - x'A'b + x'A'Ax +y'y - 2y'x+x'x
Is equivalent to:
minimize(over x) (-2 b'A - 2y'*I) x + x' ( A'A + I) * x
the first order condition is:
(A'A+I+(A'A+I)')x -2A'b-2I'y = 0
(A'A+I) x = A'b+I'y
Your problem is essentially the same. It has the first order condition:
(A1'*A1 + A2'*A2 + I)*x = A1'*bg + A2'*Ag + y
How to solve with fmincon
You can do the following:
f = #(X) transpose(bg-sum(X,2))*(bg-sum(X,2)) + (Ag'-sum(X,1))*transpose(Ag'-sum(X,1))+sum(sum((X_b-X).^2));
o = optimoptions('fmincon');%MaxFunEvals',30000);
o.MaxFunEvals = 30000;
Xstar2 = fmincon(f,zeros(n,n),[],[],[],[],[],[],[],o);
You can then check the answers are about the same with:
normdif = norm(Xstar - Xstar2)
And you can see that gap is small, but that the linear algebra based solution is somewhat more precise:
gap = f(Xstar2) - f(Xstar)
If the fmincon approach hangs, try it with a smaller n just to gain confidence that my linear algebra based solution is more precise, way way faster etc... n = 30 is solving a 30^2 = 900 variable optimization problem: not easy. With the linear algebra approach, you can go up to n = 100 (i.e. 10000 variable problem) or even larger.
I would probably solve this as a QP using quadprog using the following reformulation (keeping the objective as simple as possible to make the problem "less nonlinear"):
min sum(i,v(i)^2)+sum(i,w(i)^2)+sum((i,j),z(i,j)^2)
v = bg - sum(c,x)
w = ag - sum(r,x)
Z = xbase-x
The QP solver is more precise (no gradients using finite differences). This approach also allows you to add additional bounds and linear equality and inequality constraints.
The other suggestion to form the first order conditions explicitly is also a good one: it also has no issue with imprecise gradients (the first order conditions are linear). I usually prefer a quadratic model because of its flexibility.
I'm beginner in optimization and welcome any guide in this field.
I have 15 matrices (i.e., Di of size (n*m)) and want to find best weights (i.e., wi) for weighted averaging them and make a better matrix that is more similar to one given matrix (i.e., Dt).
In fact my objective function is like it:
min [norm2(sum(wi * Di) - Dt) + norm2(W)]
for i=1 ... 15 s.t. sum(wi) = 1 , wi >= 0
How can I optimize this function in Matlab?
You are describing a simple Quadratic programming, that can be easily optimized using Matlab's quadprog.
Here how it goes:
You objective function is [norm2(sum(wi * Di) - Dt) + norm2(W)] subject to some linear constraints on w. Let's re-write it using some simplified notations. Let w be a 15-by-1 vector of unknowns. Let D be an n*m-by-15 matrix (each column is one of the Di matrices you have - written as a single column), and Dt is a n*m-by-1 vector (same as your Dt but written as a column vector). Now some linear algebra (using the fact that ||x||^2 = x'*x and that argmin x is equivalent to argmin x^2)
[norm2(sum(wi * Di) - Dt)^2 + norm2(W)^2] =
(D*w-Dt)'*(D*w-Dt) + w'*w =
w'D'Dw - 2w'D'Dt + Dt'Dt + w'w =
w'(D'D+I)w - 2w'D'Dt + Dt'Dt
The last term Dt'Dt is constant w.r.t w and therefore can be discarded during minimization, leaving you with
H = 2*(D'*D+eye(15));
f = -2*Dt'*D;
As for the constraint sum(w)=1, this can easily be defined by
Aeq = ones(1,15);
beq = 1;
And a lower bound lb = zeros(15,1) will ensure that all w_i>=0.
And the quadratic optimization:
w = quadprog( H, f, [], [], Aeq, beq, lb );
Should do the trick for you!