Related
I have a question about the use of Matlab to compute solution of stochastic differentials equations. The equations are the 2.2a,b, page 3, in this paper (PDF).
My professor suggested using ode45 with a small time step, but the results do not match with those in the article. In particular the time series and the pdf. I also have a doubt about the definition of the white noise in the function.
Here the code for the integration function:
function dVdt = R_Lang( t,V )
global sigma lambda alpha
W1=sigma*randn(1,1);
W2=sigma*randn(1,1);
dVdt=[alpha*V(1)+lambda*V(1)^3+1/V(1)*0.5*sigma^2+W1;
sigma/V(1)*W2];
end
Main script:
clear variables
close all
global sigma lambda alpha
sigma=sqrt(2*0.0028);
alpha=3.81;
lambda=-5604;
tspan=[0,10];
options = odeset('RelTol',1E-6,'AbsTol',1E-6,'MaxStep',0.05);
A0=random('norm',0,0.5,[2,1]);
[t,L]=ode45(#(t,L) R_Lang(t,L),tspan,A0,options);
If you have any suggestions I'd be grateful.
Here the new code to confront my EM method and 'sde_euler'.
lambda = -5604;
sigma=sqrt(2*0.0028) ;
Rzero = 0.03; % problem parameters
phizero=-1;
dt=1e-5;
T = 0:dt:10;
N=length(T);
Xi1 = sigma*randn(1,N); % Gaussian Noise with variance=sigma^2
Xi2 = sigma*randn(1,N);
alpha=3.81;
Rem = zeros(1,N); % preallocate for efficiency
Rtemp = Rzero;
phiem = zeros(1,N); % preallocate for efficiency
phitemp = phizero;
for j = 1:N
Rtemp = Rtemp + dt*(alpha*Rtemp+lambda*Rtemp^3+sigma^2/(2*Rtemp)) + sigma*Xi1(j);
phitemp=phitemp+sigma/Rtemp*Xi2(j);
phiem(j)=phitemp;
Rem(j) = Rtemp;
end
f = #(t,V)[alpha*V(1)+lambda*V(1)^3+0.5*sigma^2/V(1)/2;
0]; % Drift function
g = #(t,V)[sigma;
sigma/V(1)]; % Diffusion function
A0 = [0.03;0]; % 2-by-1 initial condition
opts = sdeset('RandSeed',1,'SDEType','Ito'); % Set random seed, use Ito formulation
L = sde_euler(f,g,T,A0,opts);
plot(T,Rem,'r')
hold on
plot(T,L(:,1),'b')
Thanks again for the help !
ODEs and SDEs are very different and one should not use tools for ODEs, like ode45, to try to solve SDEs. Looking at the paper you linked to, they used a basic Euler-Maruyama scheme to integrate the system. This a very simple solver to implement yourself.
Before proceeding, you (and your professor!) should take some time to read up on SDEs and how to solve them numerically. I recommend this paper, which includes many Matlab examples:
Desmond J. Higham, 2001, An Algorithmic Introduction to Numerical Simulation of Stochastic Differential Equations, SIAM Rev. (Educ. Sect.), 43 525–46. http://dx.doi.org/10.1137/S0036144500378302
The URL to the Matlab files in the paper won't work; use this one. Note, that as this a 15-year old paper, some of the code related to random number generation is out of date (use rng(1) instead of randn('state',1) to seed the generator).
If you are familiar with ode45 you might look at my SDETools Matlab toolbox on GitHub. It was designed to be fast and has an interface that works very similarly to Matlab's ODE suite. Here is how you might code up your example using the Euler-Maruyma solver:
sigma = 1e-1*sqrt(2*0.0028);
lambda = -5604;
alpha = 3.81;
f = #(t,V)[alpha*V(1)+lambda*V(1)^3+0.5*sigma^2/V(1);
0]; % Drift function
g = #(t,V)[sigma;
sigma/V(1)]; % Diffusion function
dt = 1e-3; % Time step
t = 0:dt:10; % Time vector
A0 = [0.03;-2]; % 2-by-1 initial condition
opts = sdeset('RandSeed',1,'SDEType','Ito'); % Set random seed, use Ito formulation
L = sde_euler(f,g,t,A0,opts); % Integrate
figure;
subplot(211);
plot(t,L(:,2));
ylabel('\phi');
subplot(212);
plot(t,L(:,1));
ylabel('r');
xlabel('t');
I had to reduce the size of sigma or the noise was so large that it could cause the radius variable to go negative. I'm not sure if the paper discusses how they handle this singularity. You can try the 'NonNegative' option within sdeset to try to handle this or you may need to construct your own solver. I also couldn't find what integration time step the paper used. You should also consider contacting the authors of the paper directly.
UPDATE
Here's an Euler-Maruyama implementation that matches the sde_euler code above:
sigma = 1e-1*sqrt(2*0.0028);
lambda = -5604;
alpha = 3.81;
f = #(t,V)[alpha*V(1)+lambda*V(1)^3+0.5*sigma^2/V(1);
0]; % Drift function
g = #(t,V)[sigma;
sigma/V(1)]; % Diffusion function
dt = 1e-3; % Time step
t = 0:dt:10; % Time vector
A0 = [0.03;-2]; % 2-by-1 initial condition
% Create and initialize state vector (L here is transposed relative to sde_euler output)
lt = length(t);
n = length(A0);
L = zeros(n,lt);
L(:,1) = A0;
% Set seed and pre-calculate Wiener increments with order matching sde_euler
rng(1);
r = sqrt(dt)*randn(lt-1,n).';
% General Euler-Maruyama integration loop
for i = 1:lt-1
L(:,i+1) = L(:,i)+f(t(i),L(:,i))*dt+r(:,i).*g(t(i),L(:,i));
end
figure;
subplot(211);
plot(t,L(2,:));
ylabel('\phi');
subplot(212);
plot(t,L(1,:));
ylabel('r');
xlabel('t');
I am playing with image processing algorithms in MATLAB. One of the basic ones is convolving an image with a Gaussian. I ran the following test on a grayscale 800x600 image:
function [Y1, Y2] = testConvolveTime(inputImage)
[m,n] = size(inputImage);
% casting...
inputImage = cast(inputImage, 'single');
Gvec = [1 4 6 4 1]; % sigma=1;
Y1 = zeros(size(inputImage)); % modify it
Y2 = zeros(size(inputImage)); % modify it
%%%%%%%%%%%%%%%%%%% MATLAB CONV %%%%%%%%%%%%%%%%%%%%%
t1 = cputime;
for i=1:m
Y1(i,:) = conv(inputImage(i,:),Gvec,'same');
end
for j=1:n
Y1(:,j) = conv(inputImage(:,j),Gvec','same');
end
Y1 = round(Y1 / 16);
e1 = cputime - t1
%%%%%%%%%%%%%%%%%%% HAND-CODED CONV %%%%%%%%%%%%%%%%%%%%%
t2 = cputime;
for i=1:m
Y2(i,:) = myConv(inputImage(i,:),Gvec)';
end
for j=1:n
Y2(:,j) = myConv(inputImage(:,j),Gvec');
end
Y2 = round(Y2 / 16);
e2 = cputime - t2
end
Here is the code I wrote implementing convolution of 2 vectors:
% mimic MATLAB's conv(u,v,'same') function
% always returns column vec
function y = myConv(u_in, v_in)
len1 = length(u_in);
len2 = length(v_in);
if (len1 >= len2)
u = u_in;
v = v_in;
else
u = v_in;
v = u_in;
end
% from here on: v is the shorter vector (len1 >= len2)
len1 = length(u);
len2 = length(v);
maxLen = len1 + len2 - 1;
ytemp = zeros(maxLen,1);
% first part -- partial overlap
for i=1:len2-1
sum = 0;
for j=1:i
sum = sum + u(i-j+1)*v(j);
end
ytemp(i) = sum;
end
% main part -- complete overlap
for i=len2:len1
sum = 0;
for j=1:len2
sum = sum + u(i-j+1)*v(j);
end
ytemp(i) = sum;
end
% finally -- end portion
for i=len1+1:maxLen
%i-len1+1
sum = 0;
for j=i-len1+1:len2
sum = sum + u(i-j+1)*v(j);
end
ytemp(i) = sum;
end
%y = ytemp;
startIndex = round((maxLen - length(u_in))/2 + 1);
y = ytemp(startIndex:startIndex+length(u_in)-1);
% ^ note: to match MATLAB's conv(u,v,'same'), the output length must be
% that of the first argument.
end
Here are my test results:
>> [Y1, Y2] = testConvolveTime(A1);
e1 =
0.5313
e2 =
195.8906
>> norm(Y1 - Y2)
ans =
0
The norm being 0 verifies mathematical correctness. My questions are as follows:
How can my hand-coded function be >360x slower than the one that uses MATLAB's conv?
Even MATLAB's conv is still "slow" for image processing. If convolving with a Gaussian takes 0.5 of a second, what hope is there for running any image processing algorithms in real-time (e.g. at 24 FPS)? For reference my CPU is Intel N3540 # 2.16 GHz. w/ 8GB of RAM.
^ The real question: when I switch to OpenCV on C++, will operations like this become much faster?
1) conv is so much faster because it is an built-in native function, while your function is interpreted MATLAB code with nested loops.
2) Try imfilter in the Image Processing Toolbox. It may be faster than conv, and it works on uint8 arrays. Or, if you get a more recent version of MATLAB, and if you only need the Gaussian filter, try imgaussfilt.
Because (discrete) convolution is often represented via linear algebra but certainly not via for loops. In fact everytime you walk through rows or columns you should seek for ways to represent it as an algebraic operation.
The typical way is to do it via Toeplitz matrices but can be extended to way faster algorithms. And once you have the toeplitz structure then you can optimize the multiplication even further
https://en.wikipedia.org/wiki/Toeplitz_matrix#Discrete_convolution
http://www.netlib.org/utk/people/JackDongarra/etemplates/node384.html
Note that native Matlab functions can still be slow. It is not an indication of speed but maintenance level. Often you can find the algorithm used linked in the documentation and you can decide whether you should go for the custom implementation or the standard.
Why convolution in matalab is faster?
The implementation itself is very efficient.
They use fast techniques to perform multiplication and convolution.
check the BLAS, ATLAS packages if you want to see tricks to do these things very fast.
in practical (convolution in the original domain \ time or space) and (multiplication in frequency domain) are equivalent. what they do is to transform to frequency domain by using FFT (Fast Fourier Transform) and then perform the multiplication and then go back to the original domain.
Given a M x M desired covariance, R, and a desired number of sample vectors, N calculate a N x M Gaussian random vector, X in vanilla MATLAB (i.e. can't use r = mvnrnd(MU,SIGMA,cases)).
Not really sure how to tackle this, usually you need a covariance AND mean to generate a Gaussian random variable. I think sqrtm and chol could be useful.
If you have access to the MATLAB statistics toolbox you can type edit mvnrnd in MATLAB to see their solution.
[T p] = chol(sigma);
if m1 == c
mu = mu';
end
mu = mu(ones(cases,1),:);
r = randn(cases,c) * T + mu;
It feels almost like cheating to point this out, but editing MATLAB's source is very useful to understand things in general. You can also search for mvnrnd.m on google if you don't have the toolbox.
Example:
% Gaussian mean and covariance
d = 2; % number of dimensions
mu = rand(1,d);
sigma = rand(d,d); sigma = sigma*sigma';
% generate 100 samples from above distribution
num = 100;
X = mvnrnd(mu, sigma, num);
% plot samples (only for 2D case)
scatter(X(:,1), X(:,2), 'filled'), hold on
ezcontour(#(x,y) mvnpdf([x y], mu, sigma), xlim(), ylim())
title('X~N(\mu,\sigma)')
xlabel('X_1'), ylabel('X_2')
The above code uses functions from the Statistics toolbox (mvnrnd and mvnpdf). If you don't have access to it, consider these replacements (using the same concepts mentioned by others):
mvnrnd = #(mu,S,num) bsxfun(#plus, randn(num,numel(mu))*cholcov(S), mu);
mvnpdf = #(x,mu,S) exp(-0.5*(x-mu)*(S\(x-mu)')) / sqrt((2*pi)^d*det(S));
I am using the matlab code from this book: http://books.google.com/books/about/Probability_Markov_chains_queues_and_sim.html?id=HdAQdzAjl60C
Here is the Code:
function [pi] = GE(Q)
A = Q';
n = size(A);
for i=1:n-1
for j=i+1:n
A(j,i) = -A(j,i)/A(i,i);
end
for j =i+1:n
for k=i+1:n
A(j,k) = A(j,k)+ A(j,i) * A(i,k);
end
end
end
x(n) = 1;
for i = n-1:-1:1
for j= i+1:n
x(i) = x(i) + A(i,j)*x(j);
end
x(i) = -x(i)/A(i,i);
end
pi = x/norm(x,1);
Is there a faster code that I am not aware of? I am calling this functions millions of times, and it takes too much time.
MATLAB has a whole set of built-in linear algebra routines - type help slash, help lu or help chol to get started with a few of the common ways to efficiently solve linear equations in MATLAB.
Under the hood these functions are generally calling optimised LAPACK/BLAS library routines, which are generally the fastest way to do linear algebra in any programming language. Compared with a "slow" language like MATLAB it would not be unexpected if they were orders of magnitude faster than an m-file implementation.
Hope this helps.
Unless you are specifically looking to implement your own, you should use Matlab's backslash operator (mldivide) or, if you want the factors, lu. Note that mldivide can do more than Gaussian elimination (e.g., it does linear least squares, when appropriate).
The algorithms used by mldivide and lu are from C and Fortran libraries, and your own implementation in Matlab will never be as fast. If, however, you are determined to use your own implementation and want it to be faster, one option is to look for ways to vectorize your implementation (maybe start here).
One other thing to note: the implementation from the question does not do any pivoting, so its numerical stability will generally be worse than an implementation that does pivoting, and it will even fail for some nonsingular matrices.
Different variants of Gaussian elimination exist, but they are all O(n3) algorithms. If any one approach is better than another depends on your particular situation and is something you would need to investigate more.
function x = naiv_gauss(A,b);
n = length(b); x = zeros(n,1);
for k=1:n-1 % forward elimination
for i=k+1:n
xmult = A(i,k)/A(k,k);
for j=k+1:n
A(i,j) = A(i,j)-xmult*A(k,j);
end
b(i) = b(i)-xmult*b(k);
end
end
% back substitution
x(n) = b(n)/A(n,n);
for i=n-1:-1:1
sum = b(i);
for j=i+1:n
sum = sum-A(i,j)*x(j);
end
x(i) = sum/A(i,i);
end
end
Let's assume Ax=d
Where A and d are known matrices.
We want to represent "A" as "LU" using "LU decomposition" function embedded in matlab thus:
LUx = d
This can be done in matlab following:
[L,U] = lu(A)
which in terms returns an upper triangular matrix in U and a permuted lower triangular matrix in L such that A = LU. Return value L is a product of lower triangular and permutation matrices. (https://www.mathworks.com/help/matlab/ref/lu.html)
Then if we assume Ly = d where y=Ux.
Since x is Unknown, thus y is unknown too, by knowing y we find x as follows:
y=L\d;
x=U\y
and the solution is stored in x.
This is the simplest way to solve system of linear equations providing that the matrices are not singular (i.e. the determinant of matrix A and d is not zero), otherwise, the quality of the solution would not be as good as expected and might yield wrong results.
if the matrices are singular thus cannot be inversed, another method should be used to solve the system of the linear equations.
For the naive approach (aka without row swapping) for an n by n matrix:
function A = naiveGauss(A)
% find's the size
n = size(A);
n = n(1);
B = zeros(n,1);
% We have 3 steps for a 4x4 matrix so we have
% n-1 steps for an nxn matrix
for k = 1 : n-1
for i = k+1 : n
% step 1: Create multiples that would make the top left 1
% printf("multi = %d / %d\n", A(i,k), A(k,k), A(i,k)/A(k,k) )
for j = k : n
A(i,j) = A(i,j) - (A(i,k)/A(k,k)) * A(k,j);
end
B(i) = B(i) - (A(i,k)/A(k,k)) * B(k);
end
end
function Sol = GaussianElimination(A,b)
[i,j] = size(A);
for j = 1:i-1
for i = j+1:i
Sol(i,j) = Sol(i,:) -( Sol(i,j)/(Sol(j,j)*Sol(j,:)));
end
end
disp(Sol);
end
I think you can use the matlab function rref:
[R,jb] = rref(A,tol)
It produces a matrix in reduced row echelon form.
In my case it wasn't the fastest solution.
The solution below was faster in my case by about 30 percent.
function C = gauss_elimination(A,B)
i = 1; % loop variable
X = [ A B ];
[ nX mX ] = size( X); % determining the size of matrix
while i <= nX % start of loop
if X(i,i) == 0 % checking if the diagonal elements are zero or not
disp('Diagonal element zero') % displaying the result if there exists zero
return
end
X = elimination(X,i,i); % proceeding forward if diagonal elements are non-zero
i = i +1;
end
C = X(:,mX);
function X = elimination(X,i,j)
% Pivoting (i,j) element of matrix X and eliminating other column
% elements to zero
[ nX mX ] = size( X);
a = X(i,j);
X(i,:) = X(i,:)/a;
for k = 1:nX % loop to find triangular form
if k == i
continue
end
X(k,:) = X(k,:) - X(i,:)*X(k,j);
end
I need to write MATLAB code that will integrate over a R^5 hypercube using Monte Carlo. I have a basic algorithm that works when I have a generic function. But the function I need to integrate is:
∫dA
A is an element of R^5.
If I had ∫f(x)dA then I think my algorithm would work.
Here is the algorithm:
% Writen by Jerome W Lindsey III
clear;
n = 10000;
% Make a matrix of the same dimension
% as the problem. Each row is a dimension
A = rand(5,n);
% Vector to contain the solution
B = zeros(1,n);
for k = 1:n
% insert the integrand here
% I don't know how to enter a function {f(1,n), f(2,n), … f(5n)} that
% will give me the proper solution
% I threw in a function that will spit out 5!
% because that is the correct solution.
B(k) = 1 / (2 * 3 * 4 * 5);
end
mean(B)
In any case, I think I understand what the intent here is, although it does seem like somewhat of a contrived exercise. Consider the problem of trying to find the area of a circle via MC, as discussed here. Here samples are being drawn from a unit square, and the function takes on the value 1 inside the circle and 0 outside. To find the volume of a cube in R^5, we could sample from something else that contains the cube and use an analogous procedure to compute the desired volume. Hopefully this is enough of a hint to make the rest of the implementation straightforward.
I'm guessing here a bit since the numbers you give as "correct" answer don't match to how you state the exercise (volume of unit hypercube is 1).
Given the result should be 1/120 - could it be that you are supposed to integrate the standard simplex in the hypercube?
The your function would be clear. f(x) = 1 if sum(x) < 1; 0 otherwise
%Question 2, problem set 1
% Writen by Jerome W Lindsey III
clear;
n = 10000;
% Make a matrix of the same dimension
% as the problem. Each row is a dimension
A = rand(5,n);
% Vector to contain the solution
B = zeros(1,n);
for k = 1:n
% insert the integrand here
% this bit of code works as the integrand
if sum(A(:,k)) < 1
B(k) = 1;
end
end
clear k;
clear A;
% Begin error estimation calculations
std_mc = std(B);
clear n;
clear B;
% using the error I calculate a new random
% vector of corect length
N_new = round(std_mc ^ 2 * 3.291 ^ 2 * 1000000);
A_new = rand(5, N_new);
B_new = zeros(1,N_new);
clear std_mc;
for k = 1:N_new
if sum(A_new(:,k)) < 1
B_new(k) = 1;
end
end
clear k;
clear A_new;
% collect descriptive statisitics
M_new = mean(B_new);
std_new = std(B_new);
MC_new_error_999 = std_new * 3.921 / sqrt(N_new);
clear N_new;
clear B_new;
clear std_new;
% Display Results
disp('Integral in question #2 is');
disp(M_new);
disp(' ');
disp('Monte Carlo Error');
disp(MC_new_error_999);