numerical integration for Gaussian function - indefinite integral - matlab

My approach
fun = #(y) (1/sqrt(pi))*exp(-(y-1).^2).*log(1 + exp(-4*y))
integral(fun,-Inf,Inf)
This gives NaN.
So I tried plotting it.
y= -10:0.1:10;
plot(y,exp(-(y-1).^2).*log(1 + exp(-4*y)))
Then understood that domain (siginificant part) is from -4 to +4.
So changed the limits to
integral(fun,-10,10)
However I do not want to always plot the graph and then know its limits. So is there any way to know the integral directly from -Inf to Inf.

Discussion
If your integrals are always of the form
I would use a high-order Gauss–Hermite quadrature rule.
It's similar to the Gauss-Legendre-Kronrod rule that forms the basis for quadgk but is specifically tailored for integrals over the real line with a standard Gaussian multiplier.
Rewriting your equation with the substitution x = y-1, we get
.
The integral can then be computed using the Gauss-Hermite rule of arbitrary order (within reason):
>> order = 10;
>> [nodes,weights] = GaussHermiteRule(order);
>> f = #(x) log(1 + exp(-4*(x+1)))/sqrt(pi);
>> sum(f(nodes).*weights)
ans =
0.1933
I'd note that the function below builds a full order x order matrix to compute nodes, so it shouldn't be made too large.
There is a way to avoid this by explicitly computing the weights, but I decided to be lazy.
Besides, event at order 100, the Gaussian multiplier is about 2E-98, so the integrand's contribution is extremely minimal.
And while this isn't inherently adaptive, a high-order rule should be sufficient in most cases ... I hope.
Code
function [nodes,weights] = GaussHermiteRule(n)
% ------------------------------------------------------------------------------
% Find the nodes and weights for a Gauss-Hermite Quadrature integration.
%
if (n < 1)
error('There is no Gauss-Hermite rule of order 0.');
elseif (n < 0) || (abs(n - round(n)) > eps())
error('Given order ''n'' must be a strictly positive integer.');
else
n = round(n);
end
% Get the nodes and weights from the Golub-Welsch function
n = (0:n)' ;
b = n*0 ;
a = b + 0.5 ;
c = n ;
[nodes,weights] = GolubWelsch(a,b,c,sqrt(pi));
end
function [xk,wk] = GolubWelsch(ak,bk,ck,mu0)
%GolubWelsch
% Calculate the approximate* nodes and weights (normalized to 1) of an orthogonal
% polynomial family defined by a three-term reccurence relation of the form
% x pk(x) = ak pkp1(x) + bk pk(x) + ck pkm1(x)
%
% The weight scale factor mu0 is the integral of the weight function over the
% orthogonal domain.
%
% Calculate the terms for the orthonormal version of the polynomials
alpha = sqrt(ak(1:end-1) .* ck(2:end));
% Build the symmetric tridiagonal matrix
T = full(spdiags([[alpha;0],bk,[0;alpha]],[-1,0,+1],length(alpha),length(alpha)));
% Calculate the eigenvectors and values of the matrix
[V,xk] = eig(T,'vector');
% Calculate the weights from the eigenvectors - technically, Golub-Welsch requires
% a normalization, but since MATLAB returns unit eigenvectors, it is omitted.
wk = mu0*(V(1,:).^2)';
end

I've had success with transforming such infinite-bounded integrals using a numerical variable transformation, as explained in Numerical Recipes 3e, section 4.5.3. Basically, you substitute in y=c*tan(t)+b and then numerically integrate over t in (-pi/2,pi/2), which sweeps y from -infinity to infinity. You can tune the values of c and b to optimize the process. This approach largely dodges the question of trying to determine cutoffs in the domain, but for this to work reliably using quadrature you have to know that the integrand does not have features far from y=b.

A quick and dirty solution would be to look for a position, where your function is sufficiently small enough and then taking it as limits. This assumes that for x>0 the function fun decreases montonically and fun(x) is roughly the same size as fun(-x) for all x.
%// A small number
epsilon = eps;
%// Stepsize for searching bound
stepTest = 1;
%// Starting position for searching bound
position = 0;
%// Not yet small enough
smallEnough = false;
%// Search bound
while ~smallEnough
smallEnough = (fun(position) < eps);
position = position + stepTest;
end
%// Calculate integral
integral(fun, -position, position)
If your were happy with plotting the function, deciding by eye where you can cut, then this code will suffice, I guess.

Related

Finite difference scheme in Matlab

I am trying to implement a finite difference scheme for KdV equation in MATLAB, and I have most of the code ready, except for approximation at the first level using initial condition. It was suggested I use Euler's method to obtain 'u' at m=1, and then use the scheme for m>=2.
How does one apply Euler's method in this context? Even just a general answer for approximation at the first level would be appreciated.
I am including my code for reference
close all
clear
clc
% Generating grid with n points, with the space between two points being
%(x2-x1)/(n-1)
x = linspace(-5,5,1001);
N=1001;
h=x(2)-x(1); % grid size
dt=0.05;
%Soliton initial condition
Am=8; %Amplitude
mu=sqrt(Am/2);
x0=-15;
c=1;
syms H(x)
H(x)=piecewise(x < 0,0,x > 0,1);
u= Am*(sech(mu*(x'-x0))).^2+c^2*H(x);
% Creating a matrix A - First order
A = diag(ones(N-1,1),1)-diag(ones(N-1,1),-1);
Cvector = zeros(N, 1);
Cvector(end) = 1;
u_ic = Cvector;
% First order finite difference scheme
diff_first=A*u/(2*h)+1/(2*h)*u_ic;
% Weighted average matrix for the term 'u'
A_w = diag(ones(N-1,1),1)+diag(ones(N-1,1),-1)+diag(ones(N,1));
diff_w=2*A_w*u +2*u_ic;
% Matrix multiplication of first derivative and weighted average for 6uu_x
diff_middle=diff_first.*diff_w;
% Creating a Third Order Matrix
r = zeros(1,N);
r(2:3) = [-2,1];
c = -r;
A_third = toeplitz(c,r);
% Difference scheme for third order term
diff_third=A_third*u/(h*h*h)-1/(h*h*h)*u_ic;
%Computing finite difference method
u = u - 2*dt*diff_middle-dt*diff_third;
plot(u)

Gaussian iterative curve fitting [duplicate]

I have a set of frequency data with peaks to which I need to fit a Gaussian curve and then get the full width half maximum from. The FWHM part I can do, I already have a code for that but I'm having trouble writing code to fit the Gaussian.
Does anyone know of any functions that'll do this for me or would be able to point me in the right direction? (I can do least squares fitting for lines and polynomials but I can't get it to work for gaussians)
Also it would be helpful if it was compatible with both Octave and Matlab as I have Octave at the moment but don't get access to Matlab until next week.
Any help would be greatly appreciated!
Fitting a single 1D Gaussian directly is a non-linear fitting problem. You'll find ready-made implementations here, or here, or here for 2D, or here (if you have the statistics toolbox) (have you heard of Google? :)
Anyway, there might be a simpler solution. If you know for sure your data y will be well-described by a Gaussian, and is reasonably well-distributed over your entire x-range, you can linearize the problem (these are equations, not statements):
y = 1/(σ·√(2π)) · exp( -½ ( (x-μ)/σ )² )
ln y = ln( 1/(σ·√(2π)) ) - ½ ( (x-μ)/σ )²
= Px² + Qx + R
where the substitutions
P = -1/(2σ²)
Q = +2μ/(2σ²)
R = ln( 1/(σ·√(2π)) ) - ½(μ/σ)²
have been made. Now, solve for the linear system Ax=b with (these are Matlab statements):
% design matrix for least squares fit
xdata = xdata(:);
A = [xdata.^2, xdata, ones(size(xdata))];
% log of your data
b = log(y(:));
% least-squares solution for x
x = A\b;
The vector x you found this way will equal
x == [P Q R]
which you then have to reverse-engineer to find the mean μ and the standard-deviation σ:
mu = -x(2)/x(1)/2;
sigma = sqrt( -1/2/x(1) );
Which you can cross-check with x(3) == R (there should only be small differences).
Perhaps this has the thing you are looking for? Not sure about compatability:
http://www.mathworks.com/matlabcentral/fileexchange/11733-gaussian-curve-fit
From its documentation:
[sigma,mu,A]=mygaussfit(x,y)
[sigma,mu,A]=mygaussfit(x,y,h)
this function is doing fit to the function
y=A * exp( -(x-mu)^2 / (2*sigma^2) )
the fitting is been done by a polyfit
the lan of the data.
h is the threshold which is the fraction
from the maximum y height that the data
is been taken from.
h should be a number between 0-1.
if h have not been taken it is set to be 0.2
as default.
i had similar problem.
this was the first result on google, and some of the scripts linked here made my matlab crash.
finally i found here that matlab has built in fit function, that can fit Gaussians too.
it look like that:
>> v=-30:30;
>> fit(v', exp(-v.^2)', 'gauss1')
ans =
General model Gauss1:
ans(x) = a1*exp(-((x-b1)/c1)^2)
Coefficients (with 95% confidence bounds):
a1 = 1 (1, 1)
b1 = -8.489e-17 (-3.638e-12, 3.638e-12)
c1 = 1 (1, 1)
I found that the MATLAB "fit" function was slow, and used "lsqcurvefit" with an inline Gaussian function. This is for fitting a Gaussian FUNCTION, if you just want to fit data to a Normal distribution, use "normfit."
Check it
% % Generate synthetic data (for example) % % %
nPoints = 200; binSize = 1/nPoints ;
fauxMean = 47 ;fauxStd = 8;
faux = fauxStd.*randn(1,nPoints) + fauxMean; % REPLACE WITH YOUR ACTUAL DATA
xaxis = 1:length(faux) ;fauxData = histc(faux,xaxis);
yourData = fauxData; % replace with your actual distribution
xAxis = 1:length(yourData) ;
gausFun = #(hms,x) hms(1) .* exp (-(x-hms(2)).^2 ./ (2*hms(3)^2)) ; % Gaussian FUNCTION
% % Provide estimates for initial conditions (for lsqcurvefit) % %
height_est = max(fauxData)*rand ; mean_est = fauxMean*rand; std_est=fauxStd*rand;
x0 = [height_est;mean_est; std_est]; % parameters need to be in a single variable
options=optimset('Display','off'); % avoid pesky messages from lsqcurvefit (optional)
[params]=lsqcurvefit(gausFun,x0,xAxis,yourData,[],[],options); % meat and potatoes
lsq_mean = params(2); lsq_std = params(3) ; % what you want
% % % Plot data with fit % % %
myFit = gausFun(params,xAxis);
figure;hold on;plot(xAxis,yourData./sum(yourData),'k');
plot(xAxis,myFit./sum(myFit),'r','linewidth',3) % normalization optional
xlabel('Value');ylabel('Probability');legend('Data','Fit')

Optimization of motion law using Opti-toolbox in Matlab

I have a question concerning the numerical derivation of an ideal motion law using the Opti-toolbox in Matlab.
There are many constraints to follow, but the main principle of the target function looks like this:
This function is periodic, but only 1 period is used.
All variables are dimensionless and the number of points is n (which is an odd number).
At 0 and 1, the function yields 0.
In the middle, at (n+1)/2, the function yields e.g. 0.3.
The function needs to monotonically increase in the first halve, and decrease in the second half.
The velocity is needs to be between -0.8 and 1.
The peak acceleration will be limited when an other cost function is used.
I want to have an as low as possible peak value for the acceleration. Later this cost function will be changed.
My attempt using the Opti-toolbox looks like this:
clear all
close all
%n is number of grid points, only odd numbers allowed!
n=11;
% parameters
tijd=[0:(1/(n-1)):1];
velmin=-0.8;
velmax=1;
accmax=100;
accmin=-100;
%First order derivative:
%Last value doesn't contribute, because function is periodic
%Implementation using FE: deltax=delta_i+1-delta_i-1/2*deltax
%With deltax=(1/(n-1))
A_1=zeros(n-1,n);
A_1(2:n-1,1:n-2)=A_1(2:n-1,1:n-2)-eye(n-2);
A_1(1:n-2,2:n-1)=A_1(1:n-2,2:n-1)+eye(n-2);
A_1(1,n-1)=-1;
A_1(n-1,1)=1;
A_1=A_1*(n-1)/2;
%Lower bound and upper bound + monotonically in and decreasing
lb_111=0.1*ones((n-1)/2,1);
ub_111=velmax*ones((n-1)/2,1);
lb_112=velmin*ones((n-1)/2,1);
ub_112=-0.1*ones((n-1)/2,1);
ub_1=[ub_111; ub_112];
lb_1=[lb_111; lb_112];
%2nd order derivative, same as before.
A_12=zeros(n-1,n);
A_12(2:n-1,1:n-2)=A_12(2:n-1,1:n-2)+eye(n-2);
A_12(1:n-2,2:n-1)=A_12(1:n-2,2:n-1)+eye(n-2);
A_12(1:n-1,1:n-1)=A_12(1:n-1,1:n-1)-2*eye(n-1);
A_12(1,n-1)=1;
A_12(n-1,1)=1;
A_12=A_12*((n-1)^2);
lb_12=accmin*ones(n-1,1);
ub_12=accmax*ones(n-1,1);
%First and last values are 0.
A_2=zeros(2,n);
A_2(1,1)=1;
A_2(2,n)=1;
lb_2=zeros(2,1);
ub_2=zeros(2,1);
%Middle value is known
A_4=zeros(1,n);
A_4(1,(n+1)/2)=1;
lb_4=zeros(1,1);
lb_4(1)=0.3;
ub_4=zeros(1,1);
ub_4(1)=0.3;
%Main derivatives
%1st
af_1=#(x) A_1*x;
%2nd
af_2=#(x) A_12*x;
%Will be needed later, not relevant.
% a_3 = zeros(n-1,n);
% a_3(1:(n-1)/2,(n+1)/2:n-1)=a_3(1:(n-1)/2,(n+1)/2:n-1)+eye((n-1)/2);
% a_3((n+1)/2:n-1,1:(n-1)/2)=a_3((n+1)/2:n-1,1:(n-1)/2)+eye((n-1)/2);
% a_fase=#(x) [a_3*x; x((n+1)/2)];
%Put all linear constraints together
A = [A_4;A_1;A_12;A_2];
lb = [lb_4;lb_1;lb_12;lb_2];
ub = [ub_4;ub_1;ub_12;ub_2];
%Cost function
f = #(x) norm(af_2(x),inf);
%Non linear function later
%nlcon = #(x) [norm(g(x),inf)];
%cl = [-15];
%cu = [15];
%Estimate x0
temp = fliplr(tijd);
x_0 = [tijd(1:(n+1)/2) temp(((n+1)/2+1):end)];
%Options in solver
opts = optiset('solver','ipopt','display','iter');
Opt = opti('fun',f,'lin',A,lb,ub,'x0',x_0,'options',opts);
%Extract solution
delta1 = solve(Opt);
%delta2=a_fase(delta1);
vel1=af_1(delta1);
%vel2=af_1(delta2);
acc1=af_2(delta1);
%acc2=af_2(delta2);
%Tas=g(delta1);
%%%%%%%%%%%%%%%%
%PLOTS
%%%%%%%%%%%%%%%%
figure
subplot(2,2,1);
plot(tijd,delta1)
title('positie')
subplot(2,2,2);
plot(tijd,[vel1;vel1(1)])
title('snelheid')
subplot(2,2,3);
plot(tijd,[acc1;acc1(1)])
title('acceleratie')
subplot(2,2,4);
%plot(tijd,Tas)
title('Askoppel')
But this yields in an infeasible problem.
Does someone has an idea what I'm doing wrong?
See comment:
Result:
Motion result

Basic SVM Implemented in MATLAB

Linearly Non-Separable Binary Classification Problem
First of all, this program isn' t working correctly for RBF ( gaussianKernel() ) and I want to fix it.
It is a non-linear SVM Demo to illustrate classifying 2 class with hard margin application.
Problem is about 2 dimensional radial random distrubuted data.
I used Quadratic Programming Solver to compute Lagrange multipliers (alphas)
xn = input .* (output*[1 1]); % xiyi
phi = gaussianKernel(xn, sigma2); % Radial Basis Function
k = phi * phi'; % Symmetric Kernel Matrix For QP Solver
gamma = 1; % Adjusting the upper bound of alphas
f = -ones(2 * len, 1); % Coefficient of sum of alphas
Aeq = output'; % yi
beq = 0; % Sum(ai*yi) = 0
A = zeros(1, 2* len); % A * alpha <= b; There isn't like this term
b = 0; % There isn't like this term
lb = zeros(2 * len, 1); % Lower bound of alphas
ub = gamma * ones(2 * len, 1); % Upper bound of alphas
alphas = quadprog(k, f, A, b, Aeq, beq, lb, ub);
To solve this non linear classification problem, I wrote some kernel functions such as gaussian (RBF), homogenous and non-homogenous polynomial kernel functions.
For RBF, I implemented the function in the image below:
Using Tylor Series Expansion, it yields:
And, I seperated the Gaussian Kernel like this:
K(x, x') = phi(x)' * phi(x')
The implementation of this thought is:
function phi = gaussianKernel(x, Sigma2)
gamma = 1 / (2 * Sigma2);
featDim = 10; % Length of Tylor Series; Gaussian Kernel Converge 0 so It doesn't have to Be Inf Dimension
phi = []; % Kernel Output, The Dimension will be (#Sample) x (featDim*2)
for k = 0 : (featDim - 1)
% Gaussian Kernel Trick Using Tylor Series Expansion
phi = [phi, exp( -gamma .* (x(:, 1)).^2) * sqrt(gamma^2 * 2^k / factorial(k)) .* x(:, 1).^k, ...
exp( -gamma .* (x(:, 2)).^2) * sqrt(gamma^2 * 2^k / factorial(k)) .* x(:, 2).^k];
end
end
*** I think my RBF implementation is wrong, but I don' t know how to fix it. Please help me here.
Here is what I got as output:
where,
1) The first image : Samples of Classes
2) The second image : Marking The Support Vectors of Classes
3) The third image : Adding Random Test Data
4) The fourth image : Classification
Also, I implemented Homogenous Polinomial Kernel " K(x, x') = ( )^2 ", code is:
function phi = quadraticKernel(x)
% 2-Order Homogenous Polynomial Kernel
phi = [x(:, 1).^2, sqrt(2).*(x(:, 1).*x(:, 2)), x(:, 2).^2];
end
And I got surprisingly nice output:
To sum up, the program is working correctly with using homogenous polynomial kernel but when I use RBF, it isn' t working correctly, there is something wrong with RBF implementation.
If you know about RBF (Gaussian Kernel) please let me know how I can make it right..
Edit: If you have same issue, use RBF directly that defined above and dont separe it by phi.
Why do you want to compute phi for Gaussian Kernel? Phi will be infinite dimensional vector and you are bounding the terms in your taylor series to 10 when we don't even know whether 10 is enough to approximate the kernel values or not! Usually, the kernel is computed directly instead of getting phi (and the computing k). For example [1].
Does this mean we should never compute phi for Gaussian? Not really, no, but we have to be slightly smarter about it. There have been recent works [2,3] which show how to compute phi for Gaussian so that you can compute approximate kernel matrices while having just finite dimensional phi's. Here [4] I give the very simple code to generate the approximate kernel using the trick from the paper. However, in my experiments I needed to generate anywhere from 100 to 10000 dimensional phi's to be able to get a good approximation of the kernel (depending upon on the number of features the original input had as well as the rate at which the eigenvalues of the original matrix tapers off).
For the moment, just use code similar to [1] to generate the Gaussian kernel and then observe the result of SVM. Also, play around with the gamma parameter, a bad gamma parameter can result in really bad classification.
[1] https://github.com/ssamot/causality/blob/master/matlab-code/Code/mfunc/indep/HSIC/rbf_dot.m
[2] http://www.eecs.berkeley.edu/~brecht/papers/07.rah.rec.nips.pdf
[3] http://www.eecs.berkeley.edu/~brecht/papers/08.rah.rec.nips.pdf
[4] https://github.com/aruniyer/misc/blob/master/rks.m
Since Gaussian kernel is often referred as mapping to infinity dimensions, I always have faith in its capacity. The problem here maybe due to a bad parameter while keeping in mind grid search is always needed for SVM training. Thus I propose you could take a look at here where you could find some tricks for parameter tuning. Exponentially increasing sequence is usually used as candidates.

Numerical derivative of a vector

I have a problem with numerical derivative of a vector that is x: Nx1 with respect to another vector t (time) that is the same size of x.
I do the following (x is chosen to be sine function as an example):
t=t0:ts:tf;
x=sin(t);
xd=diff(x)/ts;
but the answer xd is (N-1)x1 and I figured out that it does not compute derivative corresponding to the first element of x.
is there any other way to compute this derivative?
You are looking for the numerical gradient I assume.
t0 = 0;
ts = pi/10;
tf = 2*pi;
t = t0:ts:tf;
x = sin(t);
dx = gradient(x)/ts
The purpose of this function is a different one (vector fields), but it offers what diff doesn't: input and output vector of equal length.
gradient calculates the central difference between data points. For an
array, matrix, or vector with N values in each row, the ith value is
defined by
The gradient at the end points, where i=1 and i=N, is calculated with
a single-sided difference between the endpoint value and the next
adjacent value within the row. If two or more outputs are specified,
gradient also calculates central differences along other dimensions.
Unlike the diff function, gradient returns an array with the same
number of elements as the input.
I know I'm a little late to the game here, but you can also get an approximation of the numerical derivative by taking the derivatives of the polynomial (cubic) splines that runs through your data:
function dy = splineDerivative(x,y)
% the spline has continuous first and second derivatives
pp = spline(x,y); % could also use pp = pchip(x,y);
[breaks,coefs,K,r,d] = unmkpp(pp);
% pre-allocate the coefficient vector
dCoeff = zeroes(K,r-1);
% Columns are ordered from highest to lowest power. Both spline and pchip
% return 4xn matrices, ordered from 3rd to zeroth power. (Thanks to the
% anonymous person who suggested this edit).
dCoeff(:, 1) = 3 * coefs(:, 1); % d(ax^3)/dx = 3ax^2;
dCoeff(:, 2) = 2 * coefs(:, 2); % d(ax^2)/dx = 2ax;
dCoeff(:, 3) = 1 * coefs(:, 3); % d(ax^1)/dx = a;
dpp = mkpp(breaks,dCoeff,d);
dy = ppval(dpp,x);
The spline polynomial is always guaranteed to have continuous first and second derivatives at each point. I haven not tested and compared this against using pchip instead of spline, but that might be another option as it too has continuous first derivatives (but not second derivatives) at every point.
The advantage of this is that there is no requirement that the step size be even.
There are some options to work-around your issue.
First: you can make your domain larger. Instead of N, use N+1 gridpoints.
Second: depending on the end-point of interest, you can use
Forward difference: F(x + dx) - F(x)
Backward difference: F(x) - F(x - dx)