Given a set of datapoints I'm trying to approximate the coefficients a,b in the function U(x)=8-ax^b using Newtons method in MATLAB.
x = [150 200 300 500 1000 2000]';
y = [2 3 4 5 6 7]';
a=170; b=-0.7; iter = 0;
for iter=1:5
f=8-a*x.^(b) -y;
J = [-x.^b -a*b*x.^(b-1)]; %Jacobis matrix
h=J\f;
a=a-h(1); b=b-h(2);
disp(norm(f))
iter = iter+1;
end
The results are incorrect and I've not been sucessful of finding the misstep. All help is appreciated.
The jacobi matrix is wrong. Using Newton's method, you're trying to find the values of a and bthat would solve the equations 8-ax^b - y = 0. So, your Jacobi should be the derivatives of f with respect to a and b. That is J = [df/da df/db], resulting in:
J = [-x.^b -a.*x.^b.*log(x)]
and you will get the following curve for the 5 iterations:
Note that you can easily linearize your model with
Log(8 - U(x)) = Log(a) + b Log(x)
Related
I am going to generate 4000 samples(x has 10 dimentions) from a pre-defined function in interval ([-10,10]^10) (f5 =#(x) 5*x(1)- 6*x(3)+x(4)^2+x(8)*x(10))
How can I do this in matlab? Actually, I thought that I should condition over all dimensions.
I would appreciate any help you can provide
you can generate some rand x in the range using the following method:
nums = rand(4000,10); % between zero to one
% If "x" in "[0,1]", "(b-a)*x + a" would be in "[a,b]"
nums = 20*nums - 10; % between -10 to 10
and then apply the function on the nums:
I = 1:size(nums,1);
F = #(i) f5(nums(i,:));
vals = arrayfun(F, I.');
I decided to take a look at two functions linspace and logspace. Below I give two examples, one using MATLAB's built-in linspace and one for logspace along with their hand made implementation. In the first case both the built-in function linspace and the handmade code give the same results. However, this is not true when examining the logspace function. Could you please help me to found the error in the handmade code?
a = 1; b = 5; n = 7;
y = linspace(1,5,7);
yy = zeros(1,n); yy(1) = a;
for i=2:n
yy(i) = yy(i-1) + (b-a)/(n-1);
end
x = logspace(1,5,7);
xx = zeros(1,n); xx(1) = 10^a;
for i=2:n
xx(i) = xx(i-1) + (10^b-10^a)/(n-1);
end
Thank you!
The only difference between linspace and logspace is that they go one step further and take the power of 10 for every element in the linspace array.
As such, you'd simply take your equation for linspace you generated, take the result and raise it to the power of 10. However, with your code, you are relying on the previous result and that is already raised to the power of 10. Therefore, you'll need to take the anti-log to convert the previous result back to a linear form, then use the same logic was used to generate the linspace, then raise it back to the power of 10. Therefore, the relationship is:
xx[n] = 10^(log10(xx[n-1]) + ((b-a)/(n-1)))
You can certainly simplify this, taking advantage of the fact that 10^(log10(z)) = z, as long as z > 0. We can also split up the terms in the power using the property that 10^(m + n) = (10^m) * (10^n). Therefore:
xx[n] = xx[n-1] * (10^((b-a)/(n-1)))
As such, simply take your previous result multiply with 10^((b-a)/(n-1))
a = 1; b = 5; n = 7;
x = logspace(1,5,7);
xx = zeros(1,n); xx(1) = 10^a;
for i=2:n
xx(i) = xx(i-1)*(10^((b-a)/(n-1))); %// Change
end
We get for both x and xx:
>> format long g;
>> x
x =
Columns 1 through 4
10 46.4158883361278 215.443469003188 1000
Columns 5 through 7
4641.58883361278 21544.3469003189 100000
>> xx
xx =
Columns 1 through 4
10 46.4158883361278 215.443469003188 1000
Columns 5 through 7
4641.58883361278 21544.3469003188 100000
In Matlab I want to create the partial derivative of a cost function called J(theta_0, theta_1) (in order to do the calculations necessary to do gradient descent).
The function J(theta_0, theta_1) is defined as:
Lets say h_theta(x) = theta_1 + theta_2*x. Also: alpha is fixed, the starting values of theta_1 and theta_2 are given. Let's say in this example: alpha = 0.1 theta_1 = 0, theta_2 = 1. Also I have all the values for x and y in two different vectors.
VectorOfX =
5
5
6
VectorOfX =
6
6
10
Steps I took to try to solve this in Matlab: I have no clue how to solve this problem in matlab. So I started off with trying to define a function in Matlab and tried this:
theta_1 = 0
theta_2 = 1
syms x;
h_theta(x) = theta_1 + t2*x;
This worked, but is not what I really wanted. I wanted to get x^(i), which is in a vector. The next thing I tried was:
theta_1 = 0
theta_2 = 1
syms x;
h_theta(x) = theta_1 + t2*vectorOfX(1);
This gives the following error:
Error using sym/subsindex (line 672)
Invalid indexing or function definition. When defining a
function, ensure that the body of the function is a SYM
object. When indexing, the input must be numeric, logical or
':'.
Error in prog1>gradientDescent (line 46)
h_theta(x) = theta_1 + theta_2*vectorOfX(x);
I looked up this error and don't know how to solve it for this particular example. I have the feeling that I make matlab work against me instead of using it in my favor.
When I have to perform symbolic computations I prefer to use Mathematica. In that environment this is the code to get the partial derivatives you are looking for.
J[th1_, th2_, m_] := Sum[(th1 + th2*Subscript[x, i] - Subscript[y, i])^2, {i, 1, m}]/(2*m)
D[J[th1, th2, m], th1]
D[J[th1, th2, m], th2]
and gives
Coming back to MATLAB we can solve this problem with the following code
%// Constants.
alpha = 0.1;
theta_1 = 0;
theta_2 = 1;
X = [5 ; 5 ; 6];
Y = [6 ; 6 ; 10];
%// Number of points.
m = length(X);
%// Partial derivatives.
Dtheta1 = #(theta_1, theta_2) sum(2*(theta_1+theta_2*X-Y))/2/m;
Dtheta2 = #(theta_1, theta_2) sum(2*X.*(theta_1+theta_2*X-Y))/2/m;
%// Loop initialization.
toll = 1e-5;
maxIter = 100;
it = 0;
err = 1;
theta_1_Last = theta_1;
theta_2_Last = theta_2;
%// Iterations.
while err>toll && it<maxIter
theta_1 = theta_1 - alpha*Dtheta1(theta_1, theta_2);
theta_2 = theta_2 - alpha*Dtheta2(theta_1, theta_2);
it = it + 1;
err = norm([theta_1-theta_1_Last ; theta_2-theta_2_Last]);
theta_1_Last = theta_1;
theta_2_Last = theta_2;
end
Unfortunately for this case the iterations does not converge.
MATLAB is not very flexible for symbolic computations, however a way to get those partial derivatives is the following
m = 10;
syms th1 th2
x = sym('x', [m 1]);
y = sym('y', [m 1]);
J = #(th1, th2) sum((th1+th2.*x-y).^2)/2/m;
diff(J, th1)
diff(J, th2)
In an attempt to vectorize a particular piece of Matlab code, I could not find a straightforward function to generate a list of the binomial coefficients. The best I could find was nchoosek, but for some inexplicable reason this function only accepts integers (not vectors of integers). My current solution looks like this:
mybinom = #(n) arrayfun(#nchoosek, n*ones(1,n), 1:n)
This generates the set of binomial coefficients for a given value of n. However, since the binomial coefficients are always symmetric, I know that I am doing twice as much work as necessary. I'm sure that I could create a solution that exploits the symmetry, but I'm sure that it would be at the expense of readability.
Is there a more elegant solution than this, perhaps using a Matlab function that I am not aware of? Note that I am not interested in using the symbolic toolbox.
If you want to minimize operations you can go along these lines:
n = 6;
k = 1:n;
result = [1 cumprod((n-k+1)./k)]
>> result
result =
1 6 15 20 15 6 1
This requires very few operations per coefficient, because each cofficient is obtained exploiting the previously computed one.
You can reduce the number of operations by approximately half if you take into account the symmetry:
m1 = floor(n/2);
m2 = ceil(n/2);
k = 1:m2;
result = [1 cumprod((n-k+1)./k)];
result(n+1:-1:m1+2) = result(1:m2);
What about a modified version of Luis Mendo's solution - but in logarithms:
n = 1e4;
m1 = floor(n/2);
m2 = ceil(n/2);
k = 1:m2;
% Attempt to compute real value
out0 = [1 cumprod((n-k+1)./k)];
out0(n+1:-1:m1+2) = out0(1:m2);
% In logarithms
out1 = [0 cumsum((log(n-k+1)) - log(k))];
out1(n+1:-1:m1+2) = out1(1:m2);
plot(log(out0) - out1, 'o-')
The advantage of working with logarithms is that you can set n = 1e4; and still obtain a good approximation of the real value (nchoosek(1e4, 5e3) returns Inf and this is not a good approximation at all!).
EDIT following horchler's comment
You can use the gammaln function to obtain the same result but it's not faster. The two approximations seem to be quite different:
n = 1e7;
m1 = floor(n/2);
m2 = ceil(n/2);
k = 1:m2;
% In logarithms
tic
out1 = [0 cumsum((log(n-k+1)) - log(k))];
out1(n+1:-1:m1+2) = out1(1:m2);
toc
% Elapsed time is 0.912649 seconds.
tic
k = 0:m2;
out2 = gammaln(n + 1) - gammaln(k + 1) - gammaln(n - k + 1);
out2(n+1:-1:m1+2) = out2(1:m2);
toc
% Elapsed time is 1.020188 seconds.
tmp = out2 - out1;
plot(tmp, '.')
prctile(tmp, [0 2.5 25 50 75 97.5 100])
% 1.0e-006 *
% -0.2217 -0.1462 -0.0373 0.0363 0.1225 0.2943 0.3846
Is adding three gammaln worse than adding n logarithms? Or viceversa?
This works for Octave only
You can use bincoeff function.
Example: bincoeff(5, 0:5)
EDIT :
Only improvement I can think of goes like this. Maybe you already thought this trivial solution and didn't like it.
# Calculate only the first half
mybinomhalf = #(n) arrayfun(#nchoosek, n*ones(1,n/2+1), 0:n/2)
# pad your array symmetrically
mybinom = #(n) padarray(mybinomhalf(n), [0 n/2], 'symmetric', 'post')
# I couldn't test it and this line may not work
I use convolution and for loops (too much for loops) for calculating the interpolation using
Lagrange's method , here's the main code :
function[p] = lagrange_interpolation(X,Y)
L = zeros(n);
p = zeros(1,n);
% computing L matrice, so that each row i holds the polynom L_i
% Now we compute li(x) for i=0....n ,and we build the polynomial
for k=1:n
multiplier = 1;
outputConv = ones(1,1);
for index = 1:n
if(index ~= k && X(index) ~= X(k))
outputConv = conv(outputConv,[1,-X(index)]);
multiplier = multiplier * ((X(k) - X(index))^-1);
end
end
polynimialSize = length(outputConv);
for index = 1:polynimialSize
L(k,n - index + 1) = outputConv(polynimialSize - index + 1);
end
L(k,:) = multiplier .* L(k,:);
end
% continues
end
Those are too much for loops for computing the l_i(x) (this is done before the last calculation of P_n(x) = Sigma of y_i * l_i(x)) .
Any suggestions into making it more matlab formal ?
Thanks
Yeah, several suggestions (implemented in version 1 below): if loop can be combined with for above it (just make index skip k via something like jr(jr~=j) below); polynomialSize is always equal length(outputConv) which is always equal n (because you have n datapoints, (n-1)th polynomial with n coefficients), so the last for loop and next line can be also replaced with simple L(k,:) = multiplier * outputConv;
So I replicated the example on http://en.wikipedia.org/wiki/Lagrange_polynomial (and adopted their j-m notation, but for me j goes 1:n and m is 1:n and m~=j), hence my initialization looks like
clear; clc;
X=[-9 -4 -1 7]; %example taken from http://en.wikipedia.org/wiki/Lagrange_polynomial
Y=[ 5 2 -2 9];
n=length(X); %Lagrange basis polinomials are (n-1)th order, have n coefficients
lj = zeros(1,n); %storage for numerator of Lagrange basis polyns - each w/ n coeff
Lj = zeros(n); %matrix of Lagrange basis polyns coeffs (lj(x))
L = zeros(1,n); %the Lagrange polynomial coefficients (L(x))
then v 1.0 looks like
jr=1:n; %j-range: 1<=j<=n
for j=jr %my j is your k
multiplier = 1;
outputConv = 1; %numerator of lj(x)
mr=jr(jr~=j); %m-range: 1<=m<=n, m~=j
for m = mr %my m is your index
outputConv = conv(outputConv,[1 -X(m)]);
multiplier = multiplier * ((X(j) - X(m))^-1);
end
Lj(j,:) = multiplier * outputConv; %jth Lagrange basis polinomial lj(x)
end
L = Y*Lj; %coefficients of Lagrange polinomial L(x)
which can be further simplified if you realize that numerator of l_j(x) is just a polynomial with specific roots - for that there is a nice command in matlab - poly. Similarly the denominator is just that polyn evaluated at X(j) - for that there is polyval. Hence, v 1.9:
jr=1:n; %j-range: 1<=j<=n
for j=jr
mr=jr(jr~=j); %m-range: 1<=m<=n, m~=j
lj=poly(X(mr)); %numerator of lj(x)
mult=1/polyval(lj,X(j)); %denominator of lj(x)
Lj(j,:) = mult * lj; %jth Lagrange basis polinomial lj(x)
end
L = Y*Lj; %coefficients of Lagrange polinomial L(x)
Why version 1.9 and not 2.0? well, there is probably a way to get rid of this last for loop, and write it all in 1 line, but I can't think of it right now - it's a todo for v 2.0 :)
And, for dessert, if you want to get the same picture as wikipedia:
figure(1);clf
x=-10:.1:10;
hold on
plot(x,polyval(Y(1)*Lj(1,:),x),'r','linewidth',2)
plot(x,polyval(Y(2)*Lj(2,:),x),'b','linewidth',2)
plot(x,polyval(Y(3)*Lj(3,:),x),'g','linewidth',2)
plot(x,polyval(Y(4)*Lj(4,:),x),'y','linewidth',2)
plot(x,polyval(L,x),'k','linewidth',2)
plot(X,Y,'ro','linewidth',2,'markersize',10)
hold off
xlim([-10 10])
ylim([-10 10])
set(gca,'XTick',-10:10)
set(gca,'YTick',-10:10)
grid on
produces
enjoy and feel free to reuse/improve
Try:
X=0:1/20:1; Y=cos(X) and create L and apply polyval(L,1).
polyval(L,1)=0.917483227909543
cos(1)=0.540302305868140
Why there is huge difference?