I want to integrate
f(x) = exp(-x^2/2)
from x=-infinity to x=+infinity
by using the Monte Carlo method. I use the function randn() to generate all x_i for the function f(x_i) = exp(-x_i^2/2) I want to integrate to calculate afterwards the mean value of f([x_1,..x_n]). My problem is, that the result depends on what values I choose for my borders x1 and x2 (see below). My result is going far away from the real value by increasing the value of x1 and x2. Actually the result should be better and better by increasing x1 and x2.
Does someone see my mistake?
Here is my Matlab code
clear all;
b=10; % border
x1 = -b; % left border
x2 = b; % right border
n = 10^6; % number of random numbers
x = randn(n,1);
f = ones(n,1);
g = exp(-(x.^2)/2);
F = ((x2-x1)/n)*f'*g;
The right value should be ~2.5066.
Thanks
Try this:
clear all;
b=10; % border
x1 = -b; % left border
x2 = b; % right border
n = 10^6; % number of random numbers
x = sort(abs(x1 - x2) * rand(n,1) + x1);
f = exp(-x.^2/2);
F = trapz(x,f)
F =
2.5066
Ok, lets start with writing of general case of MC integration:
I = S f(x) * p(x) dx, x in [a...b]
S here is integral sign.
Usually, p(x) is normalized probability density function, f(x) you want to integrate, and algorithm is very simple one:
set accumulator s to zero
start loop of N events
sample x randomly from p(x)
given x, compute f(x) and add to accumulator
back to start loop if not done
if done, divide accumulator by N and return it
In simplest textbook case you have
I = S f(x) dx, x in [a...b]
where it means PDF is equal to uniformly distributed one
p(x) = 1/(b-a)
but what you have to sum is actually (b-a)*f(x), because your integral now looks like
I = S (b-a)*f(x) 1/(b-a) dx, x in [a...b]
In general, if both f(x) and p(x) could serve as PDF, then it is matter of choice whether you integrate f(x) over p(x), or p(x) over f(x). No difference! (Well, except maybe computation time)
So, back to particular integral (which is equal to \sqrt{2\pi}, i believe)
I = S exp(-x^2/2) dx, x in [-infinity...infinity]
You could use more traditional approach like #Agriculturist and write it
I = S exp(-x^2/2)*(2a) 1/(2a) dx, x in [-a...a]
and sample x from U(0,1) in [-a...a] interval, and for each x compute exp() and average it and get the result
From what I understand, you want to use exp() as PDF, so your integral looks like
I = S D * exp(-x^2/2)/D dx, x in [-infinity...infinity]
PDF to be normalized so it shall include normalization factor D, which is exactly equal to \sqrt{2 \pi} from gaussian integral.
Now f(x) is just a constant equal to D. It doesn't depend on x. It means that you for each sampled x should add to accumulator a CONSTANT value of D. After running N samples,
in accumulator you'll have exactly N*D. To find mean you'll divide by N and as a result you'll get perfect D, which is \sqrt{2 \pi}, which, in turn, is
2.5066.
Too rusty to write any matlab, and Happy New Year anyway
Related
I am trying to calculate the integral of a function in Matlab and Mathematica that the software cannot do symbolically.
Here is my MatLab code so far, but I understand it may not be very helpful as is.
f = #(t) asin(0.5*sin(t));
a = #(t) sin(t);
F = int(f,t) % Matlab can't do this
F =
int(asin(sin(t)/2), t)
A = int(a,t) % This works
A =
-cos(t)
dt = 1/(N-1); % some small number
for i=1:N
F(i) = integral(f,(i-1)*dt,i*dt);
A(i) = integral(a,(i-1)*dt,i*dt);
end
Both of the calculations in the for loop give a rough approximation of f or a not their integrals after multiplying by dt.
On the math stack-exchange I found a question that derives a finite difference like method for the integral at a point. However, when I did the calculation in Matlab it output a scaled down version of f which was evident after plotting (see above for what I mean by scaled down). I think that's because for smaller intervals the integral basically approximates the function to varying degrees of accuracy (again see above).
I am trying to get either a symbolic equation for the integral, or an approximation of the integral of the function at each location.
So my question is then if I have a function f that MatLab and Mathematica cannot easily take the integral of
can I approximate the integral directly with an integral calculator besides the default ones? (int,integral,trapz)
or
can I approximate the function with finite differences first and then evaluate the integral symbolically?
Your code is nearly fine it's just that
for i=1:N
F(i) = integral(f,0,i*dt);
end
You could also do
F(1)=integral(f,0,dt)
for i=2:N
F(i) = F(i-1)+integral(f,(i-1)*dt,i*dt);
end
Second option is surely more efficient
Because the primitive is really F(x)=int(f(x), 0, x) (0 defines a certain constant ) and for sufficiently small dx you have shown that f(x)=int(f(x), x,x+dx)/dx i. You have proven that MATLAB intégral function does its job.
For example let's take = the function above will compute if you wish to compute just replace 0 above by the constant a you like.
now and so you should get F containing a discretization of
The accepted answer in general is by far the best method I would say but if certain restrictions on your functions are allowable then there is a second method.
For two functions f and g see below
T = 1; % Period
NT = 1; % Number of periods
dt = 0.01; % time interval
time = 0:dt:NT*T; % time
syms t
x = K*sin(2*pi*t+B); % edit as appropriate
% f = A/tanh(K)*tanh(K*sin(2*pi*t+p))
% g = A/asin(K)*asin(K*sin(2*pi*t+p))
formulas found here
f = A1/tanh(K1)*(2^(2*1)-1)*2^(2*1)*bernoulli(2*1)/factorial(2*1)*x^(2*1-1);
% |K1|<pi/2
g = A2/asin(K2)*factorial(2*0)/(2^(2*0)*factorial(0)^2*(2*0+1))*x^(2*0+1);
% |K2|<1
there are no such limitations in the accepted answer
N = 60;
for k=2:N
a1 = (2^(2*k)-1)*2^(2*k)*bernoulli(2*k)/factorial(2*k);
f = f + A1/tanh(K1)*a1*x^(2*k-1);
a2 = factorial(2*k)/(2^(2*k)*factorial(k)^2*(2*k+1));
g = g + A2/asin(K2)*a*x^(2*k+1);
end
MATLAB can calculate sin^n(t) for n being an integer.
F = int(f,t);
phi = double(subs(F,t,time));
G = int(g,t);
psi = double(subs(G,t,time));
Let's say you have some vector z and you compute [f, x] = ecdf(z);, hence your empirical CDF can be plotted with stairs(x, f).
Is there a simple way to compute what all the percentile scores are for z?
I could do something like:
Loop through z, that is for each entry z(i) of z
Binary search through sorted vector x to find where z(i) is. (find index j such that x(j) = z(i))
Find the corresponding value f(j)
It feels like there should be a simpler, already implemented way to do this...
Let f be a monotone function defined at values x, for which you want to compute the inverse function at values p. In your case f is monotone because it is a CDF; and the values p define the desired quantiles. Then you can simply use interp1 to interpolate x, considered as a function of f, at values p:
z = randn(1,1e5); % example data: normalized Gaussian distribution
[f, x] = ecdf(z); % compute empirical CDF
p = [0.5 0.9 0.95]; % desired values for quantiles
result = interp1(f, x, p);
In an example run of the above code, this produces
result =
0.001706069265714 1.285514249607186 1.647546848952448
For the specific case of computing quantiles p from data z, you can directly use quantile and thus avoid computing the empirical CDF:
result = quantile(z, p)
The results may be slightly different depending on how the empirical CDF has been computed in the first method:
>> quantile(z, p)
ans =
0.001706803588857 1.285515826972878 1.647582486507752
For comparison, the theoretical values for the above example (Gaussian distribution) are
>> norminv(p)
ans =
0 1.281551565544601 1.644853626951472
I actually want to use a linear model to fit a set of 'sin' data, but it turns out the loss function goes larger during each iteration. Is there any problem with my code below ? (gradient descent method)
Here is my code in Matlab
m=20;
rate = 0.1;
x = linspace(0,2*pi,20);
x = [ones(1,length(x));x]
y = sin(x);
w = rand(1,2);
for i=1:500
h = w*x;
loss = sum((h-y).^2)/m/2
total_loss = [total_loss loss];
**gradient = (h-y)*x'./m ;**
w = w - rate.*gradient;
end
Here is the data I want to fit
There isn't a problem with your code. With your current framework, if you can define data in the form of y = m*x + b, then this code is more than adequate. I actually ran it through a few tests where I define an equation of the line and add some Gaussian random noise to it (amplitude = 0.1, mean = 0, std. dev = 1).
However, one problem I will mention to you is that if you take a look at your sinusoidal data, you define a domain between [0,2*pi]. As you can see, you have multiple x values that get mapped to the same y value but of different magnitude. For example, at x = pi/2 we get 1 but at x = -3*pi/2 we get -1. This high variability will not bode well with linear regression, and so one suggestion I have is to restrict your domain... so something like [0, pi]. Another reason why it probably doesn't converge is the learning rate you chose is too high. I'd set it to something low like 0.01. As you mentioned in your comments, you already figured that out!
However, if you want to fit non-linear data using linear regression, you're going to have to include higher order terms to account for the variability. As such, try including second order and/or third order terms. This can simply be done by modifying your x matrix like so:
x = [ones(1,length(x)); x; x.^2; x.^3];
If you recall, the hypothesis function can be represented as a summation of linear terms:
h(x) = theta0 + theta1*x1 + theta2*x2 + ... + thetan*xn
In our case, each theta term would build a higher order term of our polynomial. x2 would be x^2 and x3 would be x^3. Therefore, we can still use the definition of gradient descent for linear regression here.
I'm also going to control the random generation seed (via rng) so that you can produce the same results I have gotten:
clear all;
close all;
rng(123123);
total_loss = [];
m = 20;
x = linspace(0,pi,m); %// Change
y = sin(x);
w = rand(1,4); %// Change
rate = 0.01; %// Change
x = [ones(1,length(x)); x; x.^2; x.^3]; %// Change - Second and third order terms
for i=1:500
h = w*x;
loss = sum((h-y).^2)/m/2;
total_loss = [total_loss loss];
% gradient is now in a different expression
gradient = (h-y)*x'./m ; % sum all in each iteration, it's a batch gradient
w = w - rate.*gradient;
end
If we try this, we get for w (your parameters):
>> format long g;
>> w
w =
Columns 1 through 3
0.128369521905694 0.819533906064327 -0.0944622478526915
Column 4
-0.0596638117151464
My final loss after this point is:
loss =
0.00154350916582836
This means that our equation of the line is:
y = 0.12 + 0.819x - 0.094x^2 - 0.059x^3
If we plot this equation of the line with your sinusoidal data, this is what we get:
xval = x(2,:);
plot(xval, y, xval, polyval(fliplr(w), xval))
legend('Original', 'Fitted');
I'm pretty confused on how I would go about summing an infinite amount of matrices in MATLAB. Lets say I have this function (a gaussian):
%Set up grid/coordinate system
Ngrid=400;
w=Ngrid;
h=Ngrid;
%Create Gaussian Distribution
G = zeros ([w, h]);
Sig = 7.3; %I want the end/resultant G to be a summation of Sign from 7.3 to 10 with dx
for x = 1 : w
for y = 1 : h
G (x, y) = exp (-((Sig^-2)*((x-w/2+1)^2 + (y-h/2+1)^2)) / (2));
end
end
I essentially want the end/resultant function G to be a summation of Sign from 7.3 to 10 with dx (which is infinitesimally) small ie integration. How would I go about doing this? I am pretty confused. Can it even be done?
You don't appear to actually be summing G over a range of Sig values. You never change the value of Sig. In any case, assuming that dx isn't too small and that you have the memory this can be done without any loops, let alone two.
Ngrid = 400;
w = Ngrid;
h = Ngrid;
% Create range for Sig
dx = 0.1;
Sig = 7.3:dx:10;
% Build mesh of x and y points
x = 1:w;
y = 1:h;
[X,Y] = meshgrid(x,y);
% Evaluate columnized mesh points at each value of Sig, sum up, reshape to matrix
G = reshape(sum(exp(bsxfun(#rdivide,-((X(:)-w/2+1).^2+(Y(:)-h/2+1).^2),2*Sig.^2)),2),[h w]);
figure
imagesc(G)
axis equal
This results in a figure like this
The long complicated line above can be replaced by this (uses less memory, but may be slower):
G = exp(-((X-w/2+1).^2+(Y-h/2+1).^2)/(2*Sig(1)^2));
for i = 2:length(Sig)
G = G+exp(-((X-w/2+1).^2+(Y-h/2+1).^2)/(2*Sig(i)^2));
end
I'm trying to get Matlab to take this as a function of x_1 through x_n and y_1 through y_n, where k_i and r_i are all constants.
So far my idea was to take n from the user and make two 1×n vectors called x and y, and for the x_i just pull out x(i). But I don't know how to make an arbitrary sum in MATLAB.
I also need to get the gradient of this function, which I don't know how to do either. I was thinking maybe I could make a loop and add that to the function each time, but MATLAB doesn't like that.
I don't believe a loop is necessary for this calculation. MATLAB excels at vectorized operations, so would something like this work for you?
l = 10; % how large these vectors are
k = rand(l,1); % random junk values to work with
r = rand(l,1);
x = rand(l,1);
y = rand(l,1);
vals = k(1:end-1) .* (sqrt(diff(x).^2 + diff(y).^2) - r(1:end-1)).^2;
sum(vals)
EDIT: Thanks to #Amro for correcting the formula and simplifying it with diff.
You can solve for the gradient symbolically with:
n = 10;
k = sym('k',[1 n]); % Create n variables k1, k2, ..., kn
x = sym('x',[1 n]); % Create n variables x1, x2, ..., xn
y = sym('y',[1 n]); % Create n variables y1, y2, ..., yn
r = sym('r',[1 n]); % Create n variables r1, r2, ..., rn
% Symbolically sum equation
s = sum((k(1:end-1).*sqrt((x(2:end)-x(1:end-1)).^2+(y(2:end)-y(1:end-1)).^2)-r(1:end-1)).^2)
grad_x = gradient(s,x) % Gradient with respect to x vector
grad_y = gradient(s,y) % Gradient with respect to y vector
The symbolic sum and gradients can be evaluated and converted to floating point with:
% n random data values for k, x, y, and r
K = rand(1,n);
X = rand(1,n);
Y = rand(1,n);
R = rand(1,n);
% Substitute in data for symbolic variables
S = double(subs(s,{[k,x,y,r]},{[K,X,Y,R]}))
GRAD_X = double(subs(grad_x,{[k,x,y,r]},{[K,X,Y,R]}))
GRAD_Y = double(subs(grad_y,{[k,x,y,r]},{[K,X,Y,R]}))
The gradient function is the one overloaded for symbolic variables (type help sym/gradient) or see the more detailed documentation online).
Yes, you could indeed do this with a loop, considering that x, y, k, and r are already defined.
n = length(x);
s = 0;
for j = 2 : n
s = s + k(j-1) * (sqrt((x(j) - x(j-1)).^2 + (y(j) - y(j-1)).^2) - r(j-1)).^2
end
You should derive the gradient analytically and then plug in numbers. It should not be too hard to expand these terms and then find derivatives of the resulting polynomial.
Vectorized solution is something like (I wonder why do you use sqrt().^2):
is = 2:n;
result = sum( k(is - 1) .* abs((x(is) - x(is-1)).^2 + (y(is) - y(is-1)).^2 - r(is-1)));
You can either compute gradient symbolically or rewrite this code as a function and make a standard +-eps calculation. If you need a gradient to run optimization (you code looks like a fitness function) you could use algorithms that calculate them themselves, for example, fminsearch can do this