I have recently studied the concepts of CCA and wanted to implement it in MATLAB. However there is an existing matlab command canoncorr present. I wanted to write my own code. I have studied it extensively and found three approaches :
1: Hardoon : The approach uses lagrange multipliers to decompose the problem into an generalised eigenvalue problem. The code can be found here : cca_hardoon
For sanity sake I am also giving the code here : The data has to be centered previously.
function [Wx, Wy, r] = cca(X,Y)
% CCA calculate canonical correlations
%
% [Wx Wy r] = cca(X,Y) where Wx and Wy contains the canonical correlation
% vectors as columns and r is a vector with corresponding canonical
% correlations.
%
% Update 31/01/05 added bug handling.
if (nargin ~= 2)
disp('Inocorrect number of inputs');
help cca;
Wx = 0; Wy = 0; r = 0;
return;
end
% calculating the covariance matrices
z = [X; Y];
C = cov(z.');
sx = size(X,1);
sy = size(Y,1);
Cxx = C(1:sx, 1:sx) + 10^(-8)*eye(sx);
Cxy = C(1:sx, sx+1:sx+sy);
Cyx = Cxy';
Cyy = C(sx+1:sx+sy,sx+1:sx+sy) + 10^(-8)*eye(sy);
%calculating the Wx cca matrix
Rx = chol(Cxx);
invRx = inv(Rx);
Z = invRx'*Cxy*(Cyy\Cyx)*invRx;
Z = 0.5*(Z' + Z); % making sure that Z is a symmetric matrix
[Wx,r] = eig(Z); % basis in h (X)
r = sqrt(real(r)); % as the original r we get is lamda^2
Wx = invRx * Wx; % actual Wx values
% calculating Wy
Wy = (Cyy\Cyx) * Wx;
% by dividing it by lamda
Wy = Wy./repmat(diag(r)',sy,1);
2. MATLAB approach Please note the centering of data is done within the code itself.
3. CCA by Normal SVD only : This approach does not require the qr decomposition and utilizes the svd decomposition only. I have referred top this article here : cca by svd. Please refer to the text articles below which are taken from the referred article.
I have tried to code this program myself but unsuccessfully.
function [A,B,r,U,V] = cca_by_svd(x,y)
% computing the means
N = size(x,1); mu_x = mean(x,1); mu_y = mean(y,1);
% substracting the means
x = x - repmat(mu_x,N,1); y = y - repmat(mu_y,N,1);
x = x.'; y = y.';
% computing the covariance matrices
Cxx = (1/N)*x*(x.'); Cyy = (1/N)*y*(y.'); Cxy = (1/N)*x*(y.');
%dimension
m = min(rank(x),rank(y));
%m = min(size(x,1),size(y,1));
% computing the quare root inverse of the matrix
[V,D]=eig(Cxx); d = diag(D);
% Making all the eigen values positive
d = (d+abs(d))/2; d2 = 1./sqrt(d); d2(d==0)=0; Cxx_iv=V*diag(d2)*inv(V);
% computing the quare root inverse of the matrix
[V,D]=eig(Cyy); d = diag(D);
% Making all the eigen values positive
d = (d+abs(d))/2; d2 = 1./sqrt(d); d2(d==0)=0; Cyy_iv=V*diag(d2)*inv(V);
Omega = Cxx_iv*Cxy*Cyy_iv;
[C,Sigma,D] = svd(Omega);
A = Cxx_iv*C; A = A(:,1:m);
B = Cyy_iv*D.'; B = B(:,1:m);
A = real(A); B = real(B);
U = A.'*x; V = B.'*y;
r = Sigma(1:m,1:m);
I am running this code snippet:
clc;clear all;close all;
load carbig;
X = [Displacement Horsepower Weight Acceleration MPG];
nans = sum(isnan(X),2) > 0;
x = X(~nans,1:3);
y = X(~nans,4:5);
[A1, B1, r1, U1, V1] = canoncorr(x, y);
[A2, B2, r2, U2, V2] = cca_by_svd(x, y);
[A3, B3, r3] = cca(x.',y.',1);
The projection vector is coming out to be this :
>> A1
A1 =
0.0025 0.0048
0.0202 0.0409
-0.0000 -0.0027
>> A2
A2 =
0.0025 0.0048
0.0202 0.0410
-0.0000 -0.0027
>> A3
A3 =
-0.0302 -0.0050 -0.0022
0.0385 -0.0420 -0.0176
0.0020 0.0027 -0.0001
>> B1
B1 =
-0.1666 -0.3637
-0.0916 0.1078
>> B2
B2 =
-0.1668 -0.3642
-0.0917 0.1079
>> B3
B3 =
0.0000 + 0.0000i 0.3460 + 0.0000i 0.1336 + 0.0000i
0.0000 + 0.0000i -0.0967 + 0.0000i 0.0989 + 0.0000i
Question: Can someone please tell me where I am going wrong. The three approaches that I have referred all solve the same problem and ideally their solutions should converge. I admit my code 'cca_by_svd' may be wrong but hardoon's code and matlab's output should be same. Please point out to me where I am going wrong. edit I have rechecked and corrected my code. Now for this dataset the method 2 and 3 converge.
There's a few things that cca(X,Y) doesn't do that canoncorr does:
One is normalizing the data. If you add X = normc(X')' (also for Y) to your cca(X,Y) function, the output r will match that of canoncorr. If you look into canoncorr's code, you'll see that it starts by QR decomposition of X and Y.
Another difference is that eig sorts the eigenvalues in ascending order, so cca(X,Y) should flip the output of eig(Z).
NOTE: Despite correcting these differences, I wasn't able to fully recover Wx and Wy to match the outputs of canoncorr. Ideally, Wx'*Wx should look exactly alike between cca and canoncorr.
Related
I've had problems with my code as I've tried to make an integral compute, but it will not for the power, P2.
I've tried using anonymous function handles to use the integral() function on MATLAB as well as just using int(), but it will still not compute. Are the values too small for MATLAB to integrate or am I just missing something small?
Any help or advice would be appreciated to push me in the right direction. Thanks!
The problem in the code is in the bottom of the section labelled "Power Calculations". My integral also gets quite messy if that makes a difference.
%%%%%%%%%%% Parameters %%%%%%%%%%%%
n0 = 1; %air
n1 = 1.4; %layer 1
n2 = 2.62; %layer 2
n3 = 3.5; %silicon
L0 = 650*10^(-9); %centre wavelength
L1 = 200*10^(-9): 10*10^(-9): 2200*10^(-9); %lambda from 200nm to 2200nm
x = ((pi./2).*(L0./L1)); %layer phase thickness
r01 = ((n0 - n1)./(n0 + n1)); %reflection coefficient 01
r12 = ((n1 - n2)./(n1 + n2)); %reflection coefficient 12
r23 = ((n2 - n3)./(n2 + n3)); %reflection coefficient 23
t01 = ((2.*n0)./(n0 + n1)); %transmission coefficient 01
t12 = ((2.*n1)./(n1 + n2)); %transmission coefficient 12
t23 = ((2.*n2)./(n2 + n3)); %transmission coefficient 23
Q1 = [1 r01; r01 1]; %Matrix Q1
Q2 = [1 r12; r12 1]; %Matrix Q2
Q3 = [1 r23; r23 1]; %Matrix Q3
%%%%%%%%%%%% Graph of L vs R %%%%%%%%%%%
R = zeros(size(x));
for i = 1:length(x)
P = [exp(j.*x(i)) 0; 0 exp(-j.*x(i))]; %General Matrix P
T = ((1./(t01.*t12.*t23)).*(Q1*P*Q2*P*Q3)); %Transmission
T11 = T(1,1); %T11 value
T21 = T(2,1); %T21 value
R(i) = ((abs(T21./T11))^2).*100; %Percent reflectivity
end
plot(L1,R)
title('Percent Reflectance vs. wavelength for 2 Layers')
xlabel('Wavelength (m)')
ylabel('Reflectance (%)')
%%%%%%%%%%% Power Calculation %%%%%%%%%%
syms L; %General lamda
y = ((pi./2).*(L0./L)); %Layer phase thickness with variable Lamda
P1 = [exp(j.*y) 0; 0 exp(-j.*y)]; %Matrix P with variable Lambda
T1 = ((1./(t01.*t12.*t23)).*(Q1*P1*Q2*P1*Q3)); %Transmittivity matrix T1
I = ((6.16^(15))./((L.^(5)).*exp(2484./L) - 1)); %Blackbody Irradiance
Tf11 = T1(1,1); %New T11 section of matrix with variable Lambda
Tf2 = (((abs(1./Tf11))^2).*(n3./n0)); %final transmittivity
P1 = Tf2.*I; %Power before integration
L_initial = 200*10^(-9); %Initial wavelength
L_final = 2200*10^(-9); %Final wavelength
P2 = int(P1, L, L_initial, L_final) %Power production
I've refactored your code
to make it easier to read
to improve code reuse
to improve performance
to make it easier to understand
Why do you use so many unnecessary parentheses?!
Anyway, there's a few problems I saw in your code.
You used i as a loop variable, and j as the imaginary unit. It was OK for this one instance, but just barely so. In the future it's better to use 1i or 1j for the imaginary unit, and/or m or ii or something other than i or j as the loop index variable. You're helping yourself and your colleagues; it's just less confusing that way.
Towards the end, you used the variable name P1 twice in a row, and in two different ways. Although it works here, it's confusing! Took me a while to unravel why a matrix-producing function was producing scalars instead...
But by far the biggest problem in your code is the numerical problems with the blackbody irradiance computation. The term
L⁵ · exp(2484/L) - 1
for λ₀ = 200 · 10⁻⁹ m will require computing the quantity
exp(1.242 · 10¹⁰)
which, needless to say, is rather difficult for a computer :) Actually, the problem with your computation is two-fold. First, the exponentiation is definitely out of range of 64 bit IEEE-754 double precision, and will therefore result in ∞. Second, the parentheses are wrong; Planck's law should read
C/L⁵ · 1/(exp(D) - 1)
with C and D the constants (involving Planck's constant, speed of light, and Boltzmann constant), which you've presumably precomputed (I didn't check the values. I do know choice of units can mess these up, so better check).
So, aside from the silly parentheses error, I suspect the main problem is that you simply forgot to rescale λ to nm. Changing everything in the blackbody equation to nm and correcting those parentheses gives the code
I = 6.16^(15) / ( (L*1e+9)^5 * (exp(2484/(L*1e+9)) - 1) );
With this, I got a finite value for the integral of
P2 = 1.052916498836486e-010
But, again, you'd better double-check everything.
Note that I used quadgk(), because it's one of the better ones available on R2010a (which I'm stuck with), but you can just as easily replace this with integral() available on anything newer than R2012a.
Here's the code I ended up with:
function pwr = my_fcn()
% Parameters
n0 = 1; % air
n1 = 1.4; % layer 1
n2 = 2.62; % layer 2
n3 = 3.5; % silicon
L0 = 650e-9; % centre wavelength
% Reflection coefficients
r01 = (n0 - n1)/(n0 + n1);
r12 = (n1 - n2)/(n1 + n2);
r23 = (n2 - n3)/(n2 + n3);
% Transmission coefficients
t01 = (2*n0) / (n0 + n1);
t12 = (2*n1) / (n1 + n2);
t23 = (2*n2) / (n2 + n3);
% Quality factors
Q1 = [1 r01; r01 1];
Q2 = [1 r12; r12 1];
Q3 = [1 r23; r23 1];
% Initial & Final wavelengths
L_initial = 200e-9;
L_final = 2200e-9;
% plot reflectivity for selected lambda range
plot_reflectivity(L_initial, L_final, 1000);
% Compute power production
pwr = quadgk(#power_production, L_initial, L_final);
% Helper functions
% ========================================
% Graph of lambda vs reflectivity
function plot_reflectivity(L_initial, L_final, N)
L = linspace(L_initial, L_final, N);
R = zeros(size(L));
for ii = 1:numel(L)
% Transmission
T = transmittivity(L(ii));
% Percent reflectivity
R(ii) = 100 * abs(T(2,1)/T(1,1))^2 ;
end
plot(L, R)
title('Percent Reflectance vs. wavelength for 2 Layers')
xlabel('Wavelength (m)')
ylabel('Reflectance (%)')
end
% Compute transmittivity matrix for a single wavelength
function T = transmittivity(L)
% Layer phase thickness with variable Lamda
y = pi/2 * L0/L;
% Matrix P with variable Lambda
P1 = [exp(+1j*y) 0
0 exp(-1j*y)];
% Transmittivity matrix T1
T = 1/(t01*t12*t23) * Q1*P1*Q2*P1*Q3;
end
% Power for a specific wavelength. Note that this function
% accepts vector-valued wavelengths; needed for quadgk()
function pwr = power_production(L)
pwr = zeros(size(L));
for ii = 1:numel(L)
% Transmittivity matrix
T1 = transmittivity(L(ii));
% Blackbody Irradiance
I = 6.16^(15) / ( (L(ii)*1e+9)^5 * (exp(2484/(L(ii)*1e+9)) - 1) );
% final transmittivity
Tf2 = abs(1/T1(1))^2 * n3/n0;
% Power before integration
pwr(ii) = Tf2 * I;
end
end
end
Assume we have three equations:
eq1 = x1 + (x1 - x2) * t - X == 0;
eq2 = z1 + (z1 - z2) * t - Z == 0;
eq3 = ((X-x1)/a)^2 + ((Z-z1)/b)^2 - 1 == 0;
while six of known variables are:
a = 42 ;
b = 12 ;
x1 = 316190;
z1 = 234070;
x2 = 316190;
z2 = 234070;
So we are looking for three unknown variables that are:
X , Z and t
I wrote two method to solve it. But, since I need to run these code for 5.7 million data, it become really slow.
Method one (using "solve"):
tic
S = solve( eq1 , eq2 , eq3 , X , Z , t ,...
'ReturnConditions', true, 'Real', true);
toc
X = double(S.X(1))
Z = double(S.Z(1))
t = double(S.t(1))
results of method one:
X = 316190;
Z = 234060;
t = -2.9280;
Elapsed time is 0.770429 seconds.
Method two (using "fsolve"):
coeffs = [a,b,x1,x2,z1,z2]; % Known parameters
x0 = [ x2 ; z2 ; 1 ].'; % Initial values for iterations
f_d = #(x0) myfunc(x0,coeffs); % f_d considers x0 as variables
options = optimoptions('fsolve','Display','none');
tic
M = fsolve(f_d,x0,options);
toc
results of method two:
X = 316190; % X = M(1)
Z = 234060; % Z = M(2)
t = -2.9280; % t = M(3)
Elapsed time is 0.014 seconds.
Although, the second method is faster, but it still needs to be improved. Please let me know if you have a better solution for that. Thanks
* extra information:
if you are interested to know what those 3 equations are, the first two are equations of a line in 2D and the third equation is an ellipse equation. I need to find the intersection of the line with the ellipse. Obviously, we have two points as result. But, let's forget about the second answer for simplicity.
My suggestion it's to use the second approce,which it's the recommended by matlab for nonlinear equation system.
Declare a M-function
function Y=mysistem(X)
%X(1) = X
%X(2) = t
%X(3) = Z
a = 42 ;
b = 12 ;
x1 = 316190;
z1 = 234070;
x2 = 316190;
z2 = 234070;
Y(1,1) = x1 + (x1 - x2) * X(2) - X(1);
Y(2,1) = z1 + (z1 - z2) * X(2) - X(3);
Y(3,1) = ((X-x1)/a)^2 + ((Z-z1)/b)^2 - 1;
end
Then for solving use
x0 = [ x2 , z2 , 1 ];
M = fsolve(#mysistem,x0,options);
If you may want to reduce the default precision by changing StepTolerance (default 1e-6).
Also for more increare you may want to use the jacobian matrix for greater efficencies.
For more reference take a look in official documentation:
fsolve Nonlinear Equations with Analytic Jacobian
Basically giving the solver the Jacobian matrix of the system(and special options) you can increase method efficency.
When performing the innovation update for Kalman filter, I am getting Warnings
Warning: Matrix is close to singular or badly scaled. Results may be
inaccurate. RCOND = 2.169130e-017.
Maybe due to this, the result is not accurate. How can I solve this problem? I tried introducing a loop
[R,p] = chol(Ppred);
if p> 0;
count = 300;
return;
end
where count is just a variable to stall the code until a good matrix is found. but this does not help.
UPDATE:
Linear system representation of Moving Average, MA(2) model
x_n+1 = Bw_n
y_n = Cx_n + v_n
% w = N(0,Q); v = N(0,R)
%true coefficients, h = [1 0.5 -0.9];
CODE SNIPPETS
These are the functions for the 3 modules of Kalman Filter
C = [ 1 0 0 ];
B = [1 0.5 -0.9 ;
0 1 0.5;
0 0 1];
noise_var = rand(1,1); % measurement noise
order = 2;
xpred = rand(order,1);
P = 10* eye(d,d);
A = P;
P = P + B*sqrt(noise_var)*B';
P = dlyap(A,B*B');
for i = 1:N
[xpred, Ppred] = predict(xpred,B,Ppred, Q);
[nu, S] = innovation(xpred, Ppred, y(i), C, noise_var);
[xnew, Ppred, yhat, KalmanGain] = innovation_update(xpred,Ppred,nu,S,C);
if(isnan(Ppred))
count = 300;
return;
end
end
function [xpred, Ppred] = predict(input_t,B,P, Q)
xpred = B*input_t;
Ppred = P + Q;
end
function [nu, S] = innovation(xpred, Ppred, y, C, R)
nu = y - C*xpred; %% innovation
S = R + C*Ppred*C'; %% innovation covariance
end
function [xnew, Pnew, yhat, K] = innovation_update(xpred, Ppred, nu, S, C)
K1 = Ppred*C';
K = K1'*inv(S);
xnew = xpred + K'*nu; %% new state
Pnew = Ppred - Ppred*K'*C; %% new covariance
yhat = C*xnew;
end
Numerical issues are a common problem with Kalman filters. You didn't share enough of your code to be sure (especially Q), but it is common for roundoff errors to cause P to become non-positive-definite (especially with the P update form you used).
If you google "Kalman filter numerical stability" you can find a lot of references on the subject. Simple things to try in this case would be increasing Q (aka "fictitious process noise") to avoid an ill-conditioned P, using the Joseph form of the covariance update, or forcing P to be symmetric by setting P = 0.5 * ( P + P' ).
More complex options include switching to a square root form (e.g. UDU'), or using a floating point representation with more precision (e.g. double instead of float, which is mainly hard because you're probably already at double).
I have data points (x, y) that I need to fit an exponential function to,
y = A + B * exp(C * x),
but I can neither use the Curve Fitting Toolbox nor the Optimization Toolbox.
User rayryeng was good enough to help me with working code:
x = [0 0.0036 0.0071 0.0107 0.0143 0.0178 0.0214 0.0250 0.0285 0.0321 0.0357 0.0392 0.0428 0.0464 0.0464];
y = [1.3985 1.3310 1.2741 1.2175 1.1694 1.1213 1.0804 1.0395 1.0043 0.9691 0.9385 0.9080 0.8809 0.7856 0.7856];
M = [ones(numel(x),1), x(:)]; %// Ensure x is a column vector
lny = log(y(:)); %// Ensure y is a column vector and take ln
X = M\lny; %// Solve for parameters
A = exp(X(1)); %// Solve for A
b = X(2); %// Get b
xval = linspace(min(x), max(x));
yval = A*exp(b*xval);
plot(x,y,'r.',xval,yval,'b');
However, this code only fits the equation without offset
y = A * exp(B * x).
How can I extend this code to fit the three-parameter equation?
In another attempt, I managed to fit the function using fminsearch:
function [xval, yval] = curve_fitting_exponential_1_optimized(x,y,xval)
start_point = rand(1, 3);
model = #expfun;
est = fminsearch(model, start_point);
function [sse, FittedCurve] = expfun(params)
A = params(1);
B = params(2);
C = params(3);
FittedCurve = A + B .* exp(-C * x);
ErrorVector = FittedCurve - y;
sse = sum(ErrorVector .^ 2);
end
yval = est(1)+est(2) * exp(-est(3) * xval);
end
The problem here is that the result depends on the starting point which is randomly chosen, so I don't get a stable solution. But since I need the function for automatization, I need something stable. How can I get a stable solution?
How to adapt rayryeng's code for three parameters?
rayryeng used the strategy to linearize a nonlinear equation so that standard regression methods can be applied. See also Jubobs' answer to a similar question.
This strategy does no longer work if there is a non-zero offset A. We can fix the situation by getting a rough estimate of the offset. As rubenvb mentioned in the comments, we could estimate A by min(y), but then the logarithm gets applied to a zero. Instead, we could leave a bit of space between our guess of A and the minimum of the data, say half its range. Then we subtract A from the data and use rayreng's method:
x = x(:); % bring the data into the standard, more
y = y(:); % convenient format of column vectors
Aguess = min(y) - (max(y) - min(y) / 2;
guess = [ones(size(x)), -x] \ log(y - Aguess);
Bguess = exp(guess(1));
Cguess = guess(2);
For the given data, this results in
Aguess = 0.4792
Bguess = 0.9440
Cguess = 21.7609
Other than for the two-parameter situation, we cannot expect this to be a good fit. Its SSE is 0.007331.
How to get a stable solution?
This guess is however useful as a starting point for the nonlinear optimization:
start_point = [Aguess, Bguess, Cguess];
est = fminsearch(#expfun, start_point);
Aest = est(1);
Best = est(2);
Cest = est(3);
Now the optimization arrives at a stable estimate, because the computation is deterministic:
Aest = -0.1266
Best = 1.5106
Cest = 10.2314
The SSE of this estimate is 0.004041.
This is what the data (blue dots) and fitted curves (green: guess, red: optimized) look like:
Here is the whole function in all its glory - special thanks to A. Donda!
function [xval, yval] = curve_fitting_exponential_1_optimized(x,y,xval)
x = x(:); % bring the data into the standard, more convenient format of column vectors
y = y(:);
Aguess = min(y) - (max(y)-min(y)) / 2;
guess = [ones(size(x)), -x] \ log(y - Aguess);
Bguess = exp(guess(1));
Cguess = guess(2);
start_point = [Aguess, Bguess, Cguess];
est = fminsearch(#expfun, start_point);
function [sse, FittedCurve] = expfun(params)
A = params(1);
B = params(2);
C = params(3);
FittedCurve = A + B .* exp(-C * x);
ErrorVector = FittedCurve - y;
sse = sum(ErrorVector .^ 2);
end
yval = est(1)+est(2) * exp(-est(3) * xval);
end
I need to plot parameterized solutions for the following systems of equations using t values from 0 to 0.3 incremented by 0.001 each time:
x′ = 12.3 x − 2.7 y
y′ = 5.7 x − 3.7 y
This is what I have so far, but I'm pretty sure my parametric curves are wrong. I'd be expecting some exponential looking thing, not a lot of straight lines. What am I doing wrong?
A = [ 12.3, -2.7; 5.7, -3.7 ]; %initial matrix
[P D] = eig(A); %finding eigenvalues and eigenvectors
i = [1;4.3]; %initial conditions
H = inv(P)*i; %solving for c1 and c2
t = 0:0.001:0.3;
c1 = 0.2580; %constant
c2 = 4.2761; %constant
B1 = [0.9346;0.3558]; %eigenvector
B2 = [0.1775;0.9841]; %eigenvector
a = 11.2721; %eigenvalue
b = -2.6721; %eigenvalue
x1 = c1*B1*exp(a*t) + c2*B1*exp(b.*t);
x2 = c1*B2*exp(a*t) + c2*B2*exp(b.*t);
plot(x1,x2);
Your problem was calculating x1 and x2. Since B1 and B2 are vectors, doing this:
x1 = c1*B1*exp(a*t) + c2*B1*exp(b.*t);
x2 = c1*B2*exp(a*t) + c2*B2*exp(b.*t);
made x1 and x2 2 by 301 matrices.
The correct result is simpler:
x = c1*B1*exp(a*t) + c2*B2*exp(b*t);
and plotting it gives:
plot(x(1,:),x(2,:));