Related
I am currently using fmincon for minimizing a log-Likelihood function in respect to a 18*18 matrix. While on smaller problems the algorithm is very fast, it takes about 2h to converge in the current setup - as I am iterating over this minimisation problem, running through the code may take up to 2 weeks.
Is there a matlab-based, free alternative to fmincon that improves speed on such specific problems? (Costly solutions are discussed here, non-matlab solutions here.) Or would I need to call e.g. a python script from matlab?
The function I want to minimize:
function [L] = logL(A, U, Sigma_e, T, lags)
% A - parameters to optimize w.r.t
logL = 0;
for t = 1 : T - lags
logL(t, 1) = 0.5*(log(det(A * diag(Sigma_e(t,:)) * A' ) ) + ...
U(t,:) * (A * diag(Sigma_e(t,:)) * A' )^(-1) * U(t,:)' );
end
L = sum(logL);
and calling it by:
Options = optimset('Algorithm', 'active-set', 'Display', 'off', 'Hessian','bfgs', ...
'DerivativeCheck','on','Diagnostics','off','GradObj','off','LargeScale','off');
A = fmincon( #(A0)logL(A0, U, Sigma_e, T, lags), A0 , [], [] , [] , [] , [] , [] , [], Options);
(I have tried the different fmincon algorithms without much improvement). Note, T is quite large ~3000.
A and A0 are a 18*18 matrices,
Sigma_e is T*18,
U is T*18
I'm not aware of any speedy alternative to fminconst but you can vectorize the logL function to speed up the algorithm. Here is a vectorized version:
function [L] = logL(A, U, Sigma_e, T, lags)
ia = inv(A);
iat = ia.';
N = T - lags;
UU = zeros(N,1);
for t = 1: N
UU (t) = U(t,:) * (iat .* 1./Sigma_e(t,:) * ia) * U(t,:)';
end
L = 0.5 *sum( log(det(A) ^ 2 .* prod(Sigma_e(1:N,:),2)) + UU);
end
In some tests in Octave it nearly is 10X faster than the your solution.
Note that if some elements of Sigma_e are equal to zero you need to compute UU as:
UU (t)=U(t,:) * (A * diag(Sigma_e(t,:)) * A' )^(-1) * U(t,:)';
These relations are used to convert the loop solution to the vectorized one:
det(a * b * c) == det(a) * det(b) * det(c)
det(a) == det(a.')
det(diag(a)) == prod(a)
(a * b * c)^-1 == c^-1 * b^-1 * a^-1
a * diag(b) == a .* b
inv(diag(a)) == diag(1./a)
2nd ODE to solve in MATLAB:
( (a + f(t))·d²x/dt² + (b/2 + k(t))·dx/dt ) · dx/dt - g(t) = 0
Boundary condition:
dx/dt(0) = v0
where
t is the time,
x is the position
dx/dt is the velocity
d2x/dt2 is the acceleration
a, b, v0 are constants
f(t), k(t) and h(t) are KNOWN functions dependent on t
(I do not write them because they are quite big)
As an example, using symbolic variables:
syms t y
%% --- Initial conditions ---
phi = 12.5e-3;
v0 = 300;
e = 3e-3;
ro = 1580;
E = 43e9;
e_r = 0.01466;
B = 0.28e-3;
%% --- Intermediate calculations ---
v_T = sqrt(((1 + e_r) * 620e6) /E) - sqrt(E/ro) * e_r;
R_T = v_T * t;
m_acc = pi * e * ro *(R_T^2);
v_L = sqrt (E/ro);
R_L = v_L * t;
z = 2 * R_L;
E_4 = B * ((e_r^2)* B * (0.9^(z/B)-1)) /(log(0.9));
E_1 = E * e * pi * e_r^2 * (-phi* (phi - 2*v_T*t)) /16;
E_2 = pi * R_T^2 * 10e9;
E_3 = pi * R_T^2 * 1e6 * e;
%% Resolution of the problem
g_t = -diff(E_1 + E_2 + E_3, t);
f(t,y)=(g_t - (pi*v_T*e*ro/2 + E_4) * y^2 /(y* (8.33e-3 + m_acc))];
fun=matlabFunction(f);
[T,Y]=ode45(fun,[0 1], v0]);
How can I rewrite this to get x as y=dx/dt? I'm new to Matlab and any help is very welcome !
First, you chould use subs to evaluate a symbolic function. Another approach is to use matlabFunction to convert all symbolic expressions to anonymous functions, as suggested by Horchler.
Second, you're integrating the ODE as if it is 1st order in dx/dt. If you're interested in x(t) as well as dx/dt(t), then you'll have to modify the function like so:
fun = #(t,y) [y(2);
( subs(g) - (b/2 + subs(k))*y(2)*y(2) ) / ( y(2) * (a + subs(f))) ];
and of course, provide an initial value for x0 = x(0) as well as v0 = dx/dt(0).
Third, the absolute value of the parameters is hardly ever a real concern. IEEE754 double-precision floating point format can effortlessly represent numbers between 2.225073858507201e-308 and 1.797693134862316e+308 (realmin and realmax, respectively). So for the coefficients you gave (O(1014)), this is absolutely not a problem. You might lose a few digits of precision if you don't take precautions (rescale to [-1 +1], reformulate the problem in different units, ...), but the relative error due to this is more than likely to be tiny and insignificant compared to the algorithmic error made by ode45.
<RANDOM_OPINIONATED_RANT>
Fourth, WHY do you use symbolic math for this purpose?! You are doing a numerical integration, meaning, there is no analytic solution anyway. Why bother with symbolics then? Doing the integration with symbolics (through vpa even) is going to be dozens, hundreds, yes, often even thousands of times slower than keeping (or re-implementing) everything numerical (which some would argue is already slow in MATLAB compared to a bare-metal approach).
Yes, of course, for this specific, individual, isolated use case it may not matter much, but for the future I'd strongly advise you to learn to:
use symbolics for derivations, proving theorems, simplifying expressions, ...
use numerics to implement any algorithm or function from which actual numbers are expected.
In other words, symbolics for drafting, numerics for crunching. And exactly zero symbolics should appear in any good implementation of any algorithm.
Although it's possible to mix them to some extent, that does not mean it is a good idea to do so. In fact, that's almost never. And the few isolated cases where it is the only viable option are not a vindication of the approach.
They are rare, isolated cases after all, far from the abundant norm.
For me it bears resemblance with the evil eval, with similar reasons for why it Should. Be. Avoided.
</RANDOM_OPINIONATED_RANT>
With the full code, it's easy to come up with a complete solution:
% Initial conditions
phi = 12.5e-3;
v0 = 300;
x0 = 0; % (my assumption)
e = 3e-3;
ro = 1580;
E = 43e9;
e_r = 0.01466;
B = 0.28e-3;
% Intermediate calculations
v_T = sqrt(((1 + e_r) * 620e6) /E) - sqrt(E/ro) * e_r;
R_T = #(t) v_T * t;
m_acc = #(t) pi * e * ro *(R_T(t)^2);
v_L = sqrt (E/ro);
R_L = #(t) v_L * t;
z = #(t) 2 * R_L(t);
E_4 = #(t) B * ((e_r^2)* B * (0.9^(z(t)/B)-1)) /(log(0.9));
% UNUSED
%{
E_1 = #(t) -phi * E * e * pi * e_r^2 * (phi - 2*v_T*t) /16;
E_2 = #(t) pi * R_T(t)^2 * 10e9;
E_3 = #(t) pi * R_T(t)^2 * 1e6 * e;
%}
% Resolution of the problem
g_t = #(t) -( phi * E * e * pi * e_r^2 * v_T / 8 + ... % dE_1/dt
pi * 10e9 * 2 * R_T(t) * v_T + ... % dE_2/dt
pi * 1e6 * e * 2 * R_T(t) * v_T ); % dE_3/dt
% The derivative of Z = [x(t); x'(t)] equals Z' = [x'(t); x''(t)]
f = #(t,y)[y(2);
(g_t(t) - (0.5*pi*v_T*e*ro + E_4(t)) * y(2)^2) /(y(2) * (8.33e-3 + m_acc(t)))];
% Which is readily integrated
[T,Y] = ode45(f, [0 1], [x0 v0]);
% Plot solutions
figure(1)
plot(T, Y(:,1))
xlabel('t [s]'), ylabel('position [m]')
figure(2)
plot(T, Y(:,2))
xlabel('t [s]'), ylabel('velocity [m/s]')
Results:
Note that I've not used symbolics anywhere, except to double-check my hand-derived derivatives.
As you see in the screen shot above, I have the following expression in my Matlab m-file code:
K = P * D * D' * P;
Where, P is an nxn matrix, and D is a nx1 column vector (n=4, if it matters).
Why am I getting this warning message?
What changes if I use or don't use parenthesis there?
Floating-point arithmetic is not associative. So in general, a * (b * c) won't necessarily give the same result as (a * b) * c.
Your statement as written is equivalent to ((P * D) * D') * P, so the compiler is warning you that if you're relying on the Hermitian symmetry of D * D', you should force it to calculate exactly that.
As a side note: you can always do
K = (K + K') / 2;
To enforce the Hermetian-ity of K, but it's better to compute it as Hermitian in the first place as is suggested by the P * (D * D') * P hint.
Edit: Actually, one thing to note is that K is only going to be necessarily Hermitian if P is diagonal in general. Even with P as a permutation matrix (as the letter implies), there's not guarantee of K being Hermitian. The only guaranteed Hermitian part D * D'.
I have this matrix division issue. I have something like this
(AxB)/(C*C).
I think I can write it as
(A/C) * (B/C). Correct me if I am wrong.
Now is there any way to eliminate this from taking this form. B and C are both very huge matrices and calculating B/C takes almost 1 minute in matlab. So is there any other way I can manipulate this?
If all of your matrices are square, then for your first expression you have the equivalence
A * B / (C * C) <==> A * B * inv(C * C) <==> A * B * inv(C) * inv(C)
On the other hand, your second expression is equivalent to
(A / C) * (B / C) <==> A * inv(C) * B * inv(C)
Since matrices don't commute in general, these don't have to be the same. If we equate the right-hand sides, we find that (as long as A and C are invertible) we can make some cancellations, and end up with the equation
B * inv(C) == inv(C) * B
i.e. the two expressions are the same if B commutes with inv(C). In fact we can multiply on the left and right by C, and get
C * B = B * C
so this is the same as requiring that B commutes with C.
I´m trying to solve this set of equations in MATLAB and I get an empty sym object:
equations = {'I2B+I2EQAB=I22B+I2EQBC',...
'I2A=I2EQAB+I2EQAC+I22A',...
'I2C+I2EQBC+I2EQAC=I22C',...
'I22B=IZB+IC1B',...
'IZB=IC2B+IZBB',...
'I22C=-I2C*Z2C*YC/2+IZC',...
'IZC=IC2C+IZCC',...
'I22A=IC1A+IZA1',...
'IC4A+IZA2=IZBB+IZCC',...
'IZB*Z2LB+IC2B*2/YB=IC1B*2/YB',...
'I2C*Z2C=-IC2C*2/YC+IZC*Z2LC',...
'IZA1*m*Z2LA+IC2A*2/(m*YA)=IC1A*2/(m*YA)',...
'IC4A*2/((1-m)*YA)=IC2A*2/(m*YA)+IZA2*(1-m)*Z2LA',...
'I2EQBC*Z2EQBC+IZC*Z2LC=IZB*Z2LB',...
'I2B*Z2B+IC1B*2/YB',...
'I2C*Z2C+IC1C*2/YC',...
'I2A*Z2A+IC1A*2/(m*YA)',...
'IZB*Z2LB+(1-m)*Z2LA*IZA2=IZA1*m*ZL2A-I2EQAB*Z2EQAB',...
'IZA1*m*Z2LA=IZA2*(1-m)*Z2LA+IZC*Z2LC+I2EQAC*Z2EQAC',...
'IC4A/((1-m)*YA)=IC2C/YC'};
variables = {'m','I2A','I2B','I2C','I2EQAB','I2EQAC','I2EQBC',...
'IZA1','IC1A','IC2A','IZA2','IC4A','IC1B','IZB',...
'IC2B','IZBB','IZC','IC2C','IZCC'};
LL = solve(equations{:},variables{:})
Can you help me figure out what's going wrong?
Warning: 20 equations in 19 variables.
> In solve at 139
Warning: Explicit solution could not be found.
> In solve at 170
LL =
[ empty sym ]
I think that's self explanatory, if not check out the documentation related to DSOLVE where:
Diagnostics If dsolve cannot find an
analytic solution for an equation, it
prints the warning: Warning: Explicit
solution could not be found. and
returns an empty sym object.
solve() has issues with variable names in capital letters, as yours.
Please refer to http://www.mathworks.com/matlabcentral/newsreader/view_thread/303201
I tried reformatting the equations and inputting directly into the symbolic toolbox, and the solve function just spits out all the equations, so it cannot solve for those variables as the current equations stand.
Do you have any knowledge about the domains or constraints for all those variables? If you do I'd look at specifying all those, perhaps it would allow the solver to find a solution for you.
To get you quickly up and running in the symbolic toolbox, here's your equations reformatted to fit:
equations := {
I2B + I2EQAB = I22B + I2EQBC,
I2A = I2EQAB + I2EQAC + I22A,
I2C + I2EQBC + I2EQAC = I22C,
I22B = IZB + IC1B,
IZB = IC2B + IZBB,
I22C = -I2C * Z2C * YC / 2 + IZC,
IZC = IC2C + IZCC,
I22A = IC1A + IZA1,
IC4A + IZA2 = IZBB + IZCC,
IZB * Z2LB + IC2B * 2 / YB = IC1B * 2 / YB,
I2C * Z2C = -IC2C * 2 / YC + IZC * Z2LC,
IZA1 * m * Z2LA + IC2A * 2 / (m * YA) = IC1A * 2 / (m * YA),
IC4A * 2 / ((1 - m) * YA) = IC2A * 2 / (m * YA) + IZA2 * (1 - m) * Z2LA,
I2EQBC * Z2EQBC + IZC * Z2LC = IZB * Z2LB,
I2B * Z2B + IC1B * 2 / YB,
I2C * Z2C + IC1C * 2 / YC,
I2A * Z2A + IC1A * 2 / (m * YA),
IZB * Z2LB + (1 - m) * Z2LA * IZA2 = IZA1 * m * ZL2A - I2EQAB * Z2EQAB,
IZA1 * m * Z2LA = IZA2 * (1 - m) * Z2LA + IZC * Z2LC + I2EQAC * Z2EQAC,
IC4A / ((1 - m) * YA) = IC2C / YC
}:
variables := {
m, I2A, I2B, I2C, I2EQAB, I2EQAC ,I2EQBC,
IZA1, IC1A, IC2A, IZA2, IC4A, IC1B, IZB,
IC2B, IZBB, IZC, IC2C, IZCC
}:
solve(equations, variables)
To specify that all your known variables are real numbers, use this command:
assume(variables, Type::Real)
Also note that I count 36 unique variables (unless I made a mistake) in the equations, you'd be getting a huge list of "what-if's" for those equations if the solver was able to produce a result. I'd look at your equations and see if you could group them out and solve them in smaller sets.
Matlab, symbolic solve: solve()
I think has issues with symbolic variables who's names are more than one character.
a-z works, but anytime i try to solve something with two letters or more it just spits
back out the empty set.
For instance, something as simple as solve('xy*10 = 1', 'xy') doesn't work :(