Working in MATLAB. I have the following equation:
S = aW + bX + cY + dZ
where S,W,X,Y, and Z are all known n x 1 vectors. I am trying to fit the data of S with a linear combination of the basis vectors W,X,Y, and Z with the constraint of the constants (a,b,c,d) being greater than 0. I have managed to do this in Excel's solver, and have attempted to figure it out on MATLAB, being directed towards functions like fmincon, but I am not overly familiar with MATLAB and feel I am misunderstanding the use of fmincon.
I am looking for help with understanding fmincon's use towards my problem, or redirection towards a more efficient method for solving.
Currently I have:
initials = [0.2 0.2 0.2 0.2];
fun = #(x)x(1)*W + x(2)*X + x(3)*Y + x(4)*Z;
lb = [0,0,0,0];
soln = fmincon(fun,initials,data,b,Aeq,beq,lb,ub);
I am receiving an error stating "A must have 4 column(s)". Where A is referring to my variable data which corresponds to my S in the above equation. I do not understand why it is expecting 4 columns. Also to note the variables that are in my above snippet that are not explicitly defined are defined as [], serving as space holders.
Using fmincon is a huge overkill in this case. It's like using big heavy microscope to crack nuts... or martian rover to tow a pushcart or... anyway :) May be it's OK if you don't have to process large sets of vectors. If you need to fit hundreds of thousands of such vectors it can take hours. But this classic solution will be faster by several orders of magnitude.
%first make a n x 4 matrix of your vectors;
P=[W,X,Y,Z];
% now your equation looks like this S = P*m where m is 4 x 1 vectro
% containing your coefficients ( m = [a,b,c,d] )
% so solution will be simply
m_1 = inv(P'*P)*P'*S;
% or you can use this form
m_2 = (P'*P)\P'*S;
% or even simpler
m_3 = (S'/P')';
% all three solutions should give exactly same resul
% third solution is the neatest but may not work in every version of matlab
% Your modeled vector will be
Sm = P*m_3; %you can use any of m_1, m_2 or m_3;
% And your residual
R = S-Sm;
If you need to procees many vectors don't use for cycle. For cycles are very slow in Matlab and you should use matrices instead, if possible. S can also be nk matrix, where k is number vectors you want to process. In this case m will be 4k matrix.
What you are trying to do is similar to the answer I gave at is there a python (or matlab) function that achieves the minimum MSE between given set of output vector and calculated set of vector?.
The procedure is similar to what you are doing in EXCEL with solver. You create an objective function that takes (a, b, c, d) as the input parameters and output a measure of fit (mse) and then use fmincon or similar solver to get the best (a, b, c, d) that minimize this mse. See the code below (no MATLAB to test it but it should work).
function [a, b, c, d] = optimize_abcd(W, X, Y, Z)
%=========================================================
%Second argument is the starting point, second to the last argument is the lower bound
%to ensure the solutions are all positive
res = fmincon(#MSE, [0,0,0,0], [], [], [], [], [0,0,0,0], []);
a=res(1);
b=res(2);
c=res(3);
d=res(4);
function mse = MSE(x_)
a_=x_(1);
b_=x_(2);
c_=x_(3);
d_=x_(4);
S_ = a_*W + b_*X + c_*Y + d_*Z
mse = norm(S_-S);
end
end
Related
I need to solve the linear system
A x = b
which can be done efficiently by
x = A \ b
But now A is very large and I actually only need one component, say x(1). Is there a way to solve this more efficiently than to compute all components of x?
A is not sparse. Here, efficiency is actually an issue because this is done for many b.
Also, storing the inverse of K and multiplying only its first row to b is not possible because K is badly conditioned. Using the \ operator employs the LDL solver in this case, and accuracy is lost when the inverse is explicitly used.
I don't think you'd technically get a speed-up over the very optimized Matlab routine however if you understand how it is solved then you can just solve for one part of x. E.g the following. in traditional solver you use backsub for QR solve for instance. In LU solve you use both back sub and front sub. I could get LU. Unfortunately, it actually starts at the end due to how it solves it. The same is true for LDL which would employ both. That doesn't preclude that fact there may be more efficient ways of solving whatever you have.
function [Q,R] = qrcgs(A)
%Classical Gram Schmidt for an m x n matrix
[m,n] = size(A);
% Generates the Q, R matrices
Q = zeros(m,n);
R = zeros(n,n);
for k = 1:n
% Assign the vector for normalization
w = A(:,k);
for j=1:k-1
% Gets R entries
R(j,k) = Q(:,j)'*w;
end
for j = 1:k-1
% Subtracts off orthogonal projections
w = w-R(j,k)*Q(:,j);
end
% Normalize
R(k,k) = norm(w);
Q(:,k) = w./R(k,k);
end
end
function x = backsub(R,b)
% Backsub for upper triangular matrix.
[m,n] = size(R);
p = min(m,n);
x = zeros(n,1);
for i=p:-1:1
% Look from bottom, assign to vector
r = b(i);
for j=(i+1):p
% Subtract off the difference
r = r-R(i,j)*x(j);
end
x(i) = r/R(i,i);
end
end
The method mldivide, generally represented as \ accepts solving many systems with the same A at once.
x = A\[b1 b2 b3 b4] # where bi are vectors with n rows
Solves the system for each b, and will return an nx4 matrix, where each column is the solution of each b. Calling mldivide like this should improve efficiency becaus the descomposition is only done once.
As in many decompositions like LU od LDL' (and in the one you are interested in particular) the matrix multiplying x is upper diagonal, the first value to be solved is x(n). However, having to do the LDL' decomposition, a simple backwards substitution algorithm won't be the bottleneck of the code. Therefore, the decomposition can be saved in order to avoid repeating the calculation for every bi. Thus, the code would look similar to this:
[LA,DA] = ldl(A);
DA = sparse(DA);
% LA = sparse(LA); %LA can also be converted to sparse matrix
% loop over bi
xi = LA'\(DA\(LA\bi));
% end loop
As you can see in the documentation of mldivide (Algorithms section), it performs some checks on the input matrixes, and having defined LA as full and DA as sparse, it should directly go for a triangular solver and a tridiagonal solver. If LA was converted to sparse, it would use a triangular solver too, and I don't know if the conversion to sparse would represent any improvement.
I'm trying to find the line which best fits to the data. I use the following code below but now I want to have the data placed into an array sorted so it has the data which is closest to the line first how can I do this? Also is polyfit the correct function to use for this?
x=[1,2,2.5,4,5];
y=[1,-1,-.9,-2,1.5];
n=1;
p = polyfit(x,y,n)
f = polyval(p,x);
plot(x,y,'o',x,f,'-')
PS: I'm using Octave 4.0 which is similar to Matlab
You can first compute the error between the real value y and the predicted value f
err = abs(y-f);
Then sort the error vector
[val, idx] = sort(err);
And use the sorted indexes to have your y values sorted
y2 = y(idx);
Now y2 has the same values as y but the ones closer to the fitting value first.
Do the same for x to compute x2 so you have a correspondence between x2 and y2
x2 = x(idx);
Sembei Norimaki did a good job of explaining your primary question, so I will look at your secondary question = is polyfit the right function?
The best fit line is defined as the line that has a mean error of zero.
If it must be a "line" we could use polyfit, which will fit a polynomial. Of course, a "line" can be defined as first degree polynomial, but first degree polynomials have some properties that make it easy to deal with. The first order polynomial (or linear) equation you are looking for should come in this form:
y = mx + b
where y is your dependent variable and X is your independent variable. So the challenge is this: find the m and b such that the modeled y is as close to the actual y as possible. As it turns out, the error associated with a linear fit is convex, meaning it has one minimum value. In order to calculate this minimum value, it is simplest to combine the bias and the x vectors as follows:
Xcombined = [x.' ones(length(x),1)];
then utilized the normal equation, derived from the minimization of error
beta = inv(Xcombined.'*Xcombined)*(Xcombined.')*(y.')
great, now our line is defined as Y = Xcombined*beta. to draw a line, simply sample from some range of x and add the b term
Xplot = [[0:.1:5].' ones(length([0:.1:5].'),1)];
Yplot = Xplot*beta;
plot(Xplot, Yplot);
So why does polyfit work so poorly? well, I cant say for sure, but my hypothesis is that you need to transpose your x and y matrixies. I would guess that that would give you a much more reasonable line.
x = x.';
y = y.';
then try
p = polyfit(x,y,n)
I hope this helps. A wise man once told me (and as I learn every day), don't trust an algorithm you do not understand!
Here's some test code that may help someone else dealing with linear regression and least squares
%https://youtu.be/m8FDX1nALSE matlab code
%https://youtu.be/1C3olrs1CUw good video to work out by hand if you want to test
function [a0 a1] = rtlinreg(x,y)
x=x(:);
y=y(:);
n=length(x);
a1 = (n*sum(x.*y) - sum(x)*sum(y))/(n*sum(x.^2) - (sum(x))^2); %a1 this is the slope of linear model
a0 = mean(y) - a1*mean(x); %a0 is the y-intercept
end
x=[65,65,62,67,69,65,61,67]'
y=[105,125,110,120,140,135,95,130]'
[a0 a1] = rtlinreg(x,y); %a1 is the slope of linear model, a0 is the y-intercept
x_model =min(x):.001:max(x);
y_model = a0 + a1.*x_model; %y=-186.47 +4.70x
plot(x,y,'x',x_model,y_model)
I have some data points to which I need to fit an exponential curve of the form
y = B * exp(A/x)
(without the help of Curve Fitting Toolbox).
What I have tried so far to linearize the model by applying log, which results in
log(y/B) = A/x
log(y) = A/x + log(B)
I can then write it in the form
Y = AX + B
Now, if I neglect B, then I am able to solve it with
A = pseudoinverse (X) * Y
but I am stuck with values of B...
Fitting a curve of the form
y = b * exp(a / x)
to some data points (xi, yi) in the least-squares sense is difficult. You cannot use linear least-squares for that, because the model parameters (a and b) do not appear in an affine manner in the equation. Unless you're ready to use some nonlinear-least-squares method, an alternative approach is to modify the optimization problem so that the modified problem can be solved using linear least squares (this process is sometimes called "data linearization"). Let's do that.
Under the assumption that b and the yi's be positive, you can apply the natural logarithm to both sides of the equations:
log(y) = log(b) + a / x
or
a / x + log(b) = log(y)
By introducing a new parameter b2, defined as log(b), it becomes evident that parameters a and b2 appear in a linear (affine, really) manner in the new equation:
a / x + b2 = log(y)
Therefore, you can compute the optimal values of those parameters using least squares; all you have left to do is construct the right linear system and then solve it using MATLAB's backslash operator:
A = [1 ./ x, ones(size(x))];
B = log(y);
params_ls = A \ B;
(I'm assuming x and y are column vectors, here.)
Then, the optimal values (in the least-squares sense) for the modified problem are given by:
a_ls = params_ls(1);
b_ls = exp(params_ls(2));
Although those values are not, in general, optimal for the original problem, they are often "good enough" in practice. If needed, you can always use them as initial guesses for some iterative nonlinear-least-squares method.
Doing the log transform then using linear regression should do it. Wikipedia has a nice section on how to do this:
http://en.wikipedia.org/wiki/Linear_least_squares_%28mathematics%29#The_general_problem
%MATLAB code for finding the best fit line using least squares method
x=input('enter a') %input in the form of matrix, rows contain points
a=[1,x(1,1);1,x(2,1);1,x(3,1)] %forming A of Ax=b
b=[x(1,2);x(2,2);x(3,2)] %forming b of Ax=b
yy=inv(transpose(a)*a)*transpose(a)*b %computing projection of matrix A on b, giving x
%plotting the best fit line
xx=linspace(1,10,50);
y=yy(1)+yy(2)*xx;
plot(xx,y)
%plotting the points(data) for which we found the best fit line
hold on
plot(x(2,1),x(2,2),'x')
hold on
plot(x(1,1),x(1,2),'x')
hold on
plot(x(3,1),x(3,2),'x')
hold off
I'm sure the code can be cleaned up, but that's the gist of it.
I need help fitting data in a least square sense to a nonlinear function. Given data, how do I proceed when I have following equation?
f(x) = 20 + ax + b*e^(c*2x)
So I want to find a,b and c. If it was products, I would linearize the function by takin the natural logaritm all over the function but I can not seem to do that in this case.
Thanks
You can use the nlinfit tool, which doesn't require the Curve Fitting Toolbox (I don't think...)
Something like
f = #(b,x)(20 + b(1)*x + b(2)*exp(b(3)*2*x));
beta0 = [1, 1, 1];
beta = nlinfit(x, Y, f, beta0);
When MATLAB solves this least-squares problem, it passes the coefficients into the anonymous function f in the vector b. nlinfit returns the final values of these coefficients in the beta vector. beta0 is an initial guess of the values of b(1), b(2), and b(3). x and Y are the vectors with the data that you want to fit.
Alternatively, you can define the function in its own file, if it is a little more complicated. For this case, you would have something like (in the file my_function.m)
function y = my_function(b,x)
y = 20 + b(1)*x + b(2)*exp(b(3)*2*x);
end
and the rest of the code would look like
beta0 = [1, 1, 1];
beta = nlinfit(x, Y, #my_function, beta0);
See also: Using nlinfit in Matlab?
You can try the cftool which is an interactive tool for fitting data. The second part I don't quite understand. It may help if you describe it in more detail.
I have an unknown non-linear system and I want to model it using another system with some adaptable parameters (for instance, a neural network). So, I want to fix an online learning structure of the unknown system without knowing its dynamics, I can only interact with it through inputs-outputs. My problem is that I can not make it work in MATLAB using ode solvers. Lets say that we have this real system (my actual system is more complicated, but I will give a simple example in order to be understood):
function dx = realsystem(t, x)
u = 2;
dx = -3*x+6*u;
end
and we solve the equations like this:
[t,x_real] = ode15s(#(t,x)realsystem(t,x), [0 1], 0)
We suppose that is an unknown system and we do not know the coefficients 3 and 6 so we take an adaptive system with the 2 adaptive laws:
dx(t) = -p1(t)*x(t) + p2(t)*u(t)
dp1(t) = -e(t)*x(t)
dp2(t) = e(t)*u(t)
with e(t) the error e(t) = x(t) - x_real(t).
The thing is that I cannot find a way to feed the real values for each t to the ode solver in order to have online learning.
I tried with something like this but it didn't work:
function dx = adaptivesystem(t, x, x_real)
dx = zeros(3,1);
e = x_real - x;
u = 2;
dx(1) = -x(2)*x(1)+x(3)*u;
dx(2) = -e*x(1); %dx(2) = dp1(t)
dx(3) = e*u; %dx(3) = dp2(t)
end
You should be aware that your problem is ill-posed as it is. Given any trajectory x(t) obtained via sampling and smoothing/interpolating, you can choose p1(t) at will and set
p2(t) = ( x'(t) - p1(t)*x(t) ) / u.
So you have to formulate restrictions. One obvious is that the functions p1 and p2 should be valid for all trajectories of the black-box system. Do you have different trajectories available?
Another variant is to demand that p1 and p2 are constants. Actually, in this case and if you have equally spaced samples available, it would be easier to first find a good difference equation for the data. With the samples x[n] for time t[n]=t0+n*dt form a matrix X with rows
[ -u, x[n], x[n+1], ... ,x[n+k] ] for n=0, ... , N-k
and apply QR decomposition or SVD to X to determine the right hand kernel vectors. QR may fail to show a usable rank deficiency, so use the SVD on the top square part of R = USV^T, S diagonal, ordered as usual, U,V square and orthogonal, and use the last row of V, with coefficients
[b, a[0], ..., a[k] ],
corresponding to the smallest eigenvalue, to form the difference equation
a[0]*x[n]+a[1]*x[n-1]+...+a[k]*x[n-k]=b*u.
If the effective rank of R resp. S is not (k-1), then reduce k to be the effective rank plus one and start again.
If in the end k=1 is found, then you can make a differential equation out of it. Reformulate the difference equation as
a[0]*(x[n]-x[n-1])/dt = -(a[0]+a[1])/dt * x[n-1] + b/dt * u
and read off the differential equation
x'(t) = -(a[0]+a[1])/(a[0]*dt) * x(t) + b/(a[0]*dt) * u
One may reject this equation if the coefficients become uncomfortably large.