I'm trying to (elegantly) normalize a function numerically, depending on the parameters that are passed into it. The example below gives a Gaussian that tries to normalize within the function but fails.
I know I could do this with a couple extra lines of code, or for this example normalize analytically; I'm specifically asking about creating a numerically normalized anonymous function:
x = linspace(-10,10,10000);
my_gauss = #(w) exp(-1/(2*w^2) * x.^2) / trapz((x(2)-x(1))*my_gauss(w));
There is a big difference between an elegant solution and a solution that is efficient, clear, and correct. MATLAB has no built-in method that normalizes an array.
Based on the example that you have provided, you are trying to do too much with your anonymous function, and as a result it's not actually that general (can't deal with non-uniform sampling, etc).
I think you'd be better off creating two anonymous functions: one to perform the normalization, and the other to compute the gaussian:
x = linspace(-10, 10, 10000);
normalize = #(x, g)g ./ trapz(x, g);
gaussian = #(x, w)exp(-1 / (2 * w^2) * x.^2);
normalized_gaussian = normalize(x, gaussian(x, 10));
disp(trapz(x, normalized_gaussian))
1
% Or if you want to provide a single anonymous function
ngaussian = #(x, w)normalize(x, gaussian(w));
This is much more explicit and breaks out the functionality into more logical units that can be better understood and tested. Additionally, it can handle functions evaluated at random intervals.
Related
I want to minimize a function like below:
Here, n can be 5,10,50 etc. I want to use Matlab and want to use Gradient Descent and Quasi-Newton Method with BFGS update to solve this problem along with backtracking line search. I am a novice in Matlab. Can anyone help, please? I can find a solution for a similar problem in that link: https://www.mathworks.com/help/optim/ug/unconstrained-nonlinear-optimization-algorithms.html .
But, I really don't know how to create a vector-valued function in Matlab (in my case input x can be an n-dimensional vector).
You will have to make quite a leap to get where you want to be -- may I suggest to go through some basic tutorial first in order to digest basic MATLAB syntax and concepts? Another useful read is the very basic example to unconstrained optimization in the documentation. However, the answer to your question touches only basic syntax, so we can go through it quickly nevertheless.
The absolute minimum to invoke the unconstraint nonlinear optimization algorithms of the Optimization Toolbox is the formulation of an objective function. That function is supposed to return the function value f of your function at any given point x, and in your case it reads
function f = objfun(x)
f = sum(100 * (x(2:end) - x(1:end-1).^2).^2 + (1 - x(1:end-1)).^2);
end
Notice that
we select the indiviual components of the x vector by matrix indexing, and that
the .^ notation effects that the operand is to be squared elementwise.
For simplicity, save this function to a file objfun.m in your current working directory, so that you have it available from the command window.
Now all you have to do is to call the appropriate optimization algorithm, say, the quasi Newton method, from the command window:
n = 10; % Use n variables
options = optimoptions(#fminunc,'Algorithm','quasi-newton'); % Use QM method
x0 = rand(n,1); % Random starting guess
[x,fval,exitflag] = fminunc(#objfun, x0, options); % Solve!
fprintf('Final objval=%.2e, exitflag=%d\n', fval, exitflag);
On my machine I see that the algorithm converges:
Local minimum found.
Optimization completed because the size of the gradient is less than
the default value of the optimality tolerance.
Final objval=5.57e-11, exitflag=1
I'm solving a pair of non-linear equations for each voxel in a dataset of a ~billion voxels using fsolve() in MATLAB 2016b.
I have done all the 'easy' optimizations that I'm aware of. Memory localization is OK, I'm using parfor, the equations are in fairly numerically simple form. All discontinuities of the integral are fed to integral(). I'm using the Levenberg-Marquardt algorithm with good starting values and a suitable starting damping constant, it converges on average with 6 iterations.
I'm now at ~6ms per voxel, which is good, but not good enough. I'd need a order of magnitude reduction to make the technique viable. There's only a few things that I can think of improving before starting to hammer away at accuracy:
The splines in the equation are for quick sampling of complex equations. There are two for each equation, one is inside the 'complicated nonlinear equation'. They represent two equations, one which is has a large amount of terms but is smooth and has no discontinuities and one which approximates a histogram drawn from a spectrum. I'm using griddedInterpolant() as the editor suggested.
Is there a faster way to sample points from pre-calculated distributions?
parfor i=1:numel(I1)
sols = fsolve(#(x) equationPair(x, input1, input2, ...
6 static inputs, fsolve options)
output1(i) = sols(1); output2(i) = sols(2)
end
When calling fsolve, I'm using the 'parametrization' suggested by Mathworks to input the variables. I have a nagging feeling that defining a anonymous function for each voxel is taking a large slice of the time at this point. Is this true, is there a relatively large overhead for defining the anonymous function again and again? Do I have any way to vectorize the call to fsolve?
There are two input variables which keep changing, all of the other input variables stay static. I need to solve one equation pair for each input pair so I can't make it a huge system and solve it at once. Do I have any other options than fsolve for solving pairs of nonlinear equations?
If not, some of the static inputs are the fairly large. Is there a way to keep the inputs as persistent variables using MATLAB's persistent, would that improve performance? I only saw examples of how to load persistent variables, how could I make it so that they would be input only once and future function calls would be spared from the assumedly largish overhead of the large inputs?
EDIT:
The original equations in full form look like:
Where:
and:
Everything else is known, solving for x_1 and x_2. f_KN was approximated by a spline. S_low (E) and S_high(E) are splines, the histograms they are from look like:
So, there's a few things I thought of:
Lookup table
Because the integrals in your function do not depend on any of the parameters other than x, you could make a simple 2D-lookup table from them:
% assuming simple (square) range here, adjust as needed
[x1,x2] = meshgrid( linspace(0, xmax, N) );
LUT_high = zeros(size(x1));
LUT_low = zeros(size(x1));
for ii = 1:N
LUT_high(:,ii) = integral(#(E) Fhi(E, x1(1,ii), x2(:,ii)), ...
0, E_high, ...
'ArrayValued', true);
LUT_low(:,ii) = integral(#(E) Flo(E, x1(1,ii), x2(:,ii)), ...
0, E_low, ...
'ArrayValued', true);
end
where Fhi and Flo are helper functions to compute those integrals, vectorized with scalar x1 and vector x2 in this example. Set N as high as memory will allow.
Those lookup tables you then pass as parameters to equationPair() (which allows parfor to distribute the data). Then just use interp2 in equationPair():
F(1) = I_high - interp2(x1,x2,LUT_high, x(1), x(2));
F(2) = I_low - interp2(x1,x2,LUT_low , x(1), x(2));
So, instead of recomputing the whole integral every time, you evaluate it once for the expected range of x, and reuse the outcomes.
You can specify the interpolation method used, which is linear by default. Specify cubic if you're really concerned about accuracy.
Coarse/Fine
Should the lookup table method not be possible for some reason (memory limitations, in case the possible range of x is too big), here's another thing you could do: split up the whole procedure in 2 parts, which I'll call coarse and fine.
The intent of the coarse method is to improve your initial estimates really quickly, but perhaps not so accurately. The quickest way to approximate that integral by far is via the rectangle method:
do not approximate S with a spline, just use the original tabulated data (so S_high/low = [S_high/low#E0, S_high/low#E1, ..., S_high/low#E_high/low]
At the same values for E as used by the S data (E0, E1, ...), evaluate the exponential at x:
Elo = linspace(0, E_low, numel(S_low)).';
integrand_exp_low = exp(x(1)./Elo.^3 + x(2)*fKN(Elo));
Ehi = linspace(0, E_high, numel(S_high)).';
integrand_exp_high = exp(x(1)./Ehi.^3 + x(2)*fKN(Ehi));
then use the rectangle method:
F(1) = I_low - (S_low * Elo) * (Elo(2) - Elo(1));
F(2) = I_high - (S_high * Ehi) * (Ehi(2) - Ehi(1));
Running fsolve like this for all I_low and I_high will then have improved your initial estimates x0 probably to a point close to "actual" convergence.
Alternatively, instead of the rectangle method, you use trapz (trapezoidal method). A tad slower, but possibly a bit more accurate.
Note that if (Elo(2) - Elo(1)) == (Ehi(2) - Ehi(1)) (step sizes are equal), you can further reduce the number of computations. In that case, the first N_low elements of the two integrands are identical, so the values of the exponentials will only differ in the N_low + 1 : N_high elements. So then just compute integrand_exp_high, and set integrand_exp_low equal to the first N_low elements of integrand_exp_high.
The fine method then uses your original implementation (with the actual integral()s), but then starting at the updated initial estimates from the coarse step.
The whole objective here is to try and bring the total number of iterations needed down from about 6 to less than 2. Perhaps you'll even find that the trapz method already provides enough accuracy, rendering the whole fine step unnecessary.
Vectorization
The rectangle method in the coarse step outlined above is easy to vectorize:
% (uses R2016b implicit expansion rules)
Elo = linspace(0, E_low, numel(S_low));
integrand_exp_low = exp(x(:,1)./Elo.^3 + x(:,2).*fKN(Elo));
Ehi = linspace(0, E_high, numel(S_high));
integrand_exp_high = exp(x(:,1)./Ehi.^3 + x(:,2).*fKN(Ehi));
F = [I_high_vector - (S_high * integrand_exp_high) * (Ehi(2) - Ehi(1))
I_low_vector - (S_low * integrand_exp_low ) * (Elo(2) - Elo(1))];
trapz also works on matrices; it will integrate over each column in the matrix.
You'd call equationPair() then using x0 = [x01; x02; ...; x0N], and fsolve will then converge to [x1; x2; ...; xN], where N is the number of voxels, and each x0 is 1×2 ([x(1) x(2)]), so x0 is N×2.
parfor should be able to slice all of this fairly easily over all the workers in your pool.
Similarly, vectorization of the fine method should also be possible; just use the 'ArrayValued' option to integral() as shown above:
F = [I_high_vector - integral(#(E) S_high(E) .* exp(x(:,1)./E.^3 + x(:,2).*fKN(E)),...
0, E_high,...
'ArrayValued', true);
I_low_vector - integral(#(E) S_low(E) .* exp(x(:,1)./E.^3 + x(:,2).*fKN(E)),...
0, E_low,...
'ArrayValued', true);
];
Jacobian
Taking derivatives of your function is quite easy. Here is the derivative w.r.t. x_1, and here w.r.t. x_2. Your Jacobian will then have to be a 2×2 matrix
J = [dF(1)/dx(1) dF(1)/dx(2)
dF(2)/dx(1) dF(2)/dx(2)];
Don't forget the leading minus sign (F = I_hi/lo - g(x) → dF/dx = -dg/dx)
Using one or both of the methods outlined above, you can implement a function to compute the Jacobian matrix and pass this on to fsolve via the 'SpecifyObjectiveGradient' option (via optimoptions). The 'CheckGradients' option will come in handy there.
Because fsolve usually spends the vast majority of its time computing the Jacobian via finite differences, manually computing a value for it manually will normally speed the algorithm up tremendously.
It will be faster, because
fsolve doesn't have to do extra function evaluations to do the finite differences
the convergence rate will increase due to the improved precision of the Jacobian
Especially if you use the rectangle method or trapz like above, you can reuse many of the computations you've already done for the function values themselves, meaning, even more speed-up.
Rody's answer was the correct one. Supplying the Jacobian was the single largest factor. Especially with the vectorized version, there were 3 orders of magnitude of difference in speed with the Jacobian supplied and not.
I had trouble finding information about this subject online so I'll spell it out here for future reference: It is possible to vectorize independant parallel equations with fsolve() with great gains.
I also did some work with inlining fsolve(). After supplying the Jacobian and being smarter about the equations, the serial version of my code was mostly overhead at ~1*10^-3 s per voxel. At that point most of the time inside the function was spent passing around a options -struct and creating error-messages which are never sent + lots of unused stuff assumedly for the other optimization functions inside the optimisation function (levenberg-marquardt for me). I succesfully butchered the function fsolve and some of the functions it calls, dropping the time to ~1*10^-4s per voxel on my machine. So if you are stuck with a serial implementation e.g. because of having to rely on the previous results it's quite possible to inline fsolve() with good results.
The vectorized version provided the best results in my case, with ~5*10^-5 s per voxel.
I have a code that needs to evaluate the arc length equation below:
syms x
a = 10; b = 10; c = 10; d = 10;
fun = 4*a*x^3+3*b*x^2+2*c*x+d
int((1+(fun)^2)^.5)
but all that returns is below:
ans = int(((40*x^3 + 30*x^2 + 20*x + 10)^2 + 1)^(1/2), x)
Why wont matlab evaluate this integral? I added a line under to check if it would evaulate int(x) and it returned the desired result.
Problems involving square roots of functions may be tricky to intgrate. I am not sure whether the integral exists or not, but it if you look up the integral of a second order polynomial you will see that this one is already quite a mess. What you would have, would you expand the function inside the square root, would be a ninth order polynomial. If this integral actually would exist it may be too complex to calculate.
Anyway, if you think about it, would anyone really become any wiser by finding the analytical solution of this? If that is not the case a numerical solution should be sufficient.
EDIT
As thewaywewalk said in the comment, a general rule to calculate these kinds of integrals would be valuable, but to know the primitive function to the particular integral would probably be overkill (if a solution could be found).
Instead define the function as an anonymous function
fun = #(x) sqrt((4*a*x.^3+3*b*x.^2+2*c*x+d).^2+1);
and use integral to evaluate the function between some range, eg
integral(fun,0,100);
for evaluating the function in the closed interval [0,100].
I'm trying to compute a rather ugly integral using MATLAB. What I'm having problem with though is a part where I multiply a very big number (>10^300) with a very small number (<10^-300). MATLAB returns 'inf' for this even though it should be in the range of 0-0.0005. This is what I have
besselFunction = #(u)besseli(qb,2*sqrt(lambda*(theta + mu)).*u);
exponentFuncion = #(u)exp(-u.*(lambda + theta + mu));
where qb = 5, lambda = 12, theta = 10, mu = 3. And what I want to find is
besselFunction(u)*exponentFunction(u)
for all real values of u. The problem is that whenever u>28 it will be evaluated as 'inf'. I've heared, and tried, to use MATLAB function 'vpa' but it doesn't seem to work well when I want to use functions...
Any tips will be appreciated at this point!
I'd use logarithms.
Let x = Bessel function of u and y = x*exp(-u) (simpler than your equation, but similar).
Since log(v*w) = log(v) + log(w), then log(y) = log(x) + log(exp(-u))
This simplifies to
log(y) = log(x) - u
This will be better behaved numerically.
The other key will be to not evaluate that Bessel function that turns into a large number and passing it to a math function to get the log. Better to write your own that returns the logarithm of the Bessel function directly. Look at a reference like Abramowitz and Stegun to try and find one.
If you are doing an integration, consider using Gauss–Laguerre quadrature instead. The basic idea is that for equations of the form exp(-x)*f(x), the integral from 0 to inf can be approximated as sum(w(X).*f(X)) where the values of X are the zeros of a Laguerre polynomial and W(X) are specific weights (see the Wikipedia article). Sort of like a very advanced Simpson's rule. Since your equation already has an exp(-x) part, it is particularly suited.
To find the roots of the polynomial, there is a function on MATLAB Central called LaguerrePoly, and from there it is pretty straightforward to compute the weights.
I have a function in MATLAB which takes another function as an argument. I would like to somehow define a piecewise inline function that can be passed in. Is this somehow possible in MATLAB?
Edit: The function I would like to represent is:
f(x) = { 1.0, 0.0 <= x <= 0.5,
-1.0, 0.5 < x <= 1.0
where 0.0 <= x <= 1.0
You really have defined a piecewise function with three break points, i.e., at [0, 0.5, 1]. However, you have not defined the value of the function outside of the breaks. (By the way, I've used the term "break" here, because we are really defining a simple form of spline, a piecewise constant spline. I might also have used the term knot, another common word in the world of splines.)
If you absolutely know that you will never evaluate the function outside of [0,1], then there is no problem. So then just define a piecewise function with ONE break point, at x = 0.5. The simple way to define a piecewise constant function like yours is to use a logical operator. Thus the test (x > 0.5) returns a constant, either 0 or 1. By scaling and translating that result, it is easy to generate a function that does what you wish.
constfun = #(x) (x > 0.5)*2 - 1;
An inline function does a similar thing, but inline functions are VERY slow compared to an anonymous function. I would strongly recommend use of the anonymous form. As a test, try this:
infun = inline('(x > 0.5)*2 - 1','x');
x = 0:.001:1;
tic,y = constfun(x);toc
Elapsed time is 0.002192 seconds.
tic,y = infun(x);toc
Elapsed time is 0.136311 seconds.
Yes, the inline function took wildly more time to execute than did the anonymous form.
A problem with the simple piecewise constant form I've used here is it is difficult to expand to when you have more break points. For example, suppose you wished to define a function that took on three different values depending on what interval the point fell in? While this can be done too with creative use of tests, carefully shifting and scaling them, it can get nasty. For example, how might one define the piecewise function that returns
-1 when x < 0,
2 when 0 <= x < 1,
1 when 1 <= x
One solution is to use a unit Heaviside function. So first, define a basic unit Heaviside function.
H = #(x) (x >= 0);
Our piecewise function is now derived from H(x).
P = #(x) -1 + H(x)*3 + H(x-1)*(-1);
See that there are three pieces to P(x). The first term is what happens for x below the first break point. Then we add in a piece that takes effect above zero. Finally, the third piece adds in another offset in above x == 1. It is easily enough plotted.
ezplot(P,[-3,3])
More sophisticated splines are easily generated from this beginning. Se that I've called this construct a spline again. Really, this is where we might be leading. In fact, this is where this leads. A spline is a piecewise function, carefully tied together at a list of knots or break points. Splines in particular often have specified orders of continuity, so for example, a cubic spline will be twice differentiable (C2) across the breaks. There are also piecewise cubic functions that are only C1 functions. My point in all of this is I've described a simple beginning point to form any piecewise function. It works quite well for polynomial splines, although there may be a wee bit of mathematics required to choose the coefficients of these functions.
Another way to create this function is as an explicit piecewise polynomial. In MATLAB, we have the little known function mkpp. Try this out...
pp = mkpp([0 .5 1],[1;-1]);
Had you the splines toolbox, then fnplt will plot this directly for you. Assuming that you don't have that TB, do this:
ppfun = #(x) ppval(pp,x);
ezplot(ppfun,[0 1])
Looking back at the mkpp call, it is rather simple after all. The first argument is the list of break points in the curve (as a ROW vector). The second argument is a COLUMN vector, with the piecewise constant values the curve will take on in these two defined intervals between the breaks.
Several years ago I posted another option, piecewise_eval. It can be downloaded from the MATLAB Central file exchange. This is a function that will allow a user to specify a piecewise function purely as a list of break points, along with functional pieces between those breaks. Thus, for a function with a single break at x = 0.5, we would do this:
fun = #(x) piecewise_eval(x,0.5,{1,-1});
See that the third argument provides the value used in each segment, although those pieces need not be purely constant functions. If you wish the function to return perhaps a NaN outside of the interval of interest, this too is easily accomplished.
fun = #(x) piecewise_eval(x,[0 0.5 1],{NaN,1,-1,NaN});
My point in all of this rather lengthy excursion is to understand what a piecewise function is, and several ways to build one in MATLAB.
Unfortunately, MATLAB doesn't have a ternary operator which would make this sort of thing easier, but to expand slightly on gnovice's approach, you could create an anonymous function like so:
fh = #(x) ( 2 .* ( x <= 0.5 ) - 1 )
In general, anonymous functions are more powerful than inline function objects, and allow you to create closures etc.
If you really want to make an inline function (as opposed to an anonymous function), then the following would probably be the simplest way:
f = inline('2.*(x <= 0.5)-1');
However, as pointed out in the other answers, anonymous functions are more commonly used and are more efficient:
f = #(x) (2.*(x <= 0.5)-1);
I just had to solve that problem, and I think the easiest thing to do is use anonymous functions. Say that you have a piecewise function:
when x<0 : x^2 + 3x
when 0<=x<=4: e^x
when x>4 : log(x)
I'd first define logical masks for each piecewise region:
PIECE1 = #(x) x<0
PIECE2 = #(x) x>=0 & x<=4
PIECE3 = #(x) x>4
Then I'd put them all together:
f = #(x) PIECE1(x).*(x.^2+3*x) + PIECE2(x).*exp(x) + PIECE3(x).*log(x)
x = -10:.1:10
figure;
plot(x,f(x))