I'm trying to do the following exercise:
Define a recursive function inteDef that approximates the definite integral of a function f on the interval [a,b] using the composite trapezoidal rule. The function will take eps as a parameter, which marks the maximum length of the subintervals into which the interval [a,b] will be divided. Thus, if the length of [a,b] is less than or equal to eps, the function will return the
area of the corresponding trapezoid. Otherwise, it will divide the interval [a,b] into two subintervals of the same length (half of the original interval) and it will recursively approximate the integral of both intervals and return their sum.
My solution attempt is:
def inteDef(f: Double => Double, x: Double, y: Double, eps: Double): Double = {
if (abs(y - x) <= eps)
(y - x) * (f(x) + f(y)) / 2.0
else
inteDef(f, x, (y-x)/2.0, eps) + inteDef(f, (y-x)/2.0, y, eps)
}
It works when eps is greater than or equal to abs(y - x), but gives a stack overflow error otherwise.
(y - x) / 2.0 is not in the middle between x and y.
(x + y) / 2.0 is.
I'm trying to implement a vectorized version of the regularised logistic regression. I have found a post that explains the regularised version but I don't understand it.
To make it easy I will copy the code below:
hx = sigmoid(X * theta);
m = length(X);
J = (sum(-y' * log(hx) - (1 - y') * log(1 - hx)) / m) + lambda * sum(theta(2:end).^2) / (2*m);
grad =((hx - y)' * X / m)' + lambda .* theta .* [0; ones(length(theta)-1, 1)] ./ m ;
I understand the first part of the Cost equation, If I'm correct it could be represented as:
J = ((-y' * log(hx)) - ((1-y)' * log(1-hx)))/m;
The problem it's the regularization term. Let's take more detail:
Dimensions:
X = (m x (n+1))
theta = ((n+1) x 1)
I don't understand why he let the first term of theta (theta_0) outside of the equation, when in theory the regularized term it's:
and it has to take into account all the thetas
For the gradient descent, I think that this equation it's equivalent:
L = eye(length(theta));
L(1,1) = 0;
grad = (1/m * X'* (hx - y)+ (lambda*(L*theta)/m).
In Matlab indexes begin from 1, and in mathematic indexes begin from 0 (the indexes on the formula which you mentioned are also beginning from 0).
So, in theory, the first term of theta also needs to be let outside of the equation.
And as for your second question, you right! It is an equivalent clean equation!
I am creating a function that takes in data (x,y,z) and an anonymous function (M) as inputs. M's inputs are data (x,y,z) and a parameter (theta).
I need to determine the dimension of the parameter theta inside FUNC
EDIT: (To add some context)
I have data that follows a known data generating process (DGP). For example, I can generate data using a linear instrumental variable DGP with 1 endogenous variable (hence theta will be dimension 1):
n = 100; q = 10;
theta0 = 1; % true param value
e = randn(n, 1); % 2nd stage error
u = randn(n, 1); % 1st stage error
z = randn(n, q); % instrument
x = z * ones(q, 1) + u; % endog variable
y = x * theta0 + e; % dependent variable
Then I want to estimate theta0 using my own variation of generalized linear methods (FUNC)
M = #(x,y,z,theta) z' * (y - x * theta); % moment condition
thetahat = FUNC(M, x, y, z); % estimate theta0
and FUNC.m is
function out = FUNC(M, x, y, z)
k = ; % (!!!) <-- this is what I need to find out!
objFunc = #(theta) M(x, y, z, theta)' * M(x, y, z, theta);
out = fminunc(objFunc, ones(1, k)); % <-- this is where its used
end
In the above example, the DGP is a linear IV model. However, I should be able to use my function for any other DGP.
Other DGPs could, for example, define M as follows:
% E.g. 1) theta is dimension 1
M=#(x,y,z,theta) z' * (y - x * theta);
% E.g. 2) theta is dimension 2
M=#(x,y,z,theta) z' * (y - (x * theta(1))^theta(2));
% E.g. 3) theta is dimension 3
M=#(x,y,z,theta) z' * (y - (theta(1) + x * theta(2))^theta(3));
The (super bad) hack that I am currently using for (!!!) is:
for ktest = [3,2,1] % the dimension of theta will never be higher than 3
try
M(x, y, z, ones(1, ktest);
k = ktest;
end
end
Since you know already what the form and requirements of your function M will be when you pass it to FUNC, it doesn't make much sense to then require FUNC to determine it based only on M. It would make much more sense to pass flag values or needed information to FUNC when you pass it M. I would write FUNC in one of two ways:
function out = FUNC(M, x, y, z, k) % Accept k as an argument
...
end
function out = FUNC(M, x, y, z, theta0) % Pass the initial guess, of the correct size
...
end
If you really want to let FUNC do the extra work, then the answer from excaza is how I would do it.
Old answer below. not really valid since the question was clarified, but I'm leaving it temporarily...
I think you have two better options here...
Make M a cell array of anonymous functions:
You could make your input M a cell array of possible anonymous functions and use the number of values in theta as the index. You would pass this M to FUNC:
M = {#(x,y,z,theta) z' * (y - x * theta), ...
#(x,y,z,theta) z' * (y - (x * theta(1))^theta(2)), ...
#(x,y,z,theta) z' * (y - (theta(1) + x * theta(2))^theta(3))};
Then somewhere inside FUNC:
out = M{numel(theta)}(x, y, z, theta);
Make M a normal function instead of an anonymous one:
An anonymous function is good for quick, simple formulas. Add in conditional logic and you should probably just make it a fully-fledged function. Here's an example with a switch statement (good for if you have a number of different formulas):
function out = M(x, y, x, theta)
switch numel(theta)
case 1
out = z' * (y - x * theta);
case 2
out = z' * (y - (x * theta(1))^theta(2));
case 3
out = z' * (y - (theta(1) + x * theta(2))^theta(3));
end
end
And here's an example that sets some defaults for parameters (good for if you have one formula with different ways to set its parameters, like you seem to have):
function out = M(x, y, x, theta)
switch numel(theta)
case 1
p1 = 0;
p2 = theta;
p3 = 1;
case 2
p1 = 0;
p2 = theta(1);
p3 = theta(2);
case 3
p1 = theta(1);
p2 = theta(2);
p3 = theta(3);
end
out = z' * (y - (p1 + x * p2)^p3);
end
MATLAB doesn't store any information about the size of the inputs to an anonymous function. While a better idea would be to modify your code so you don't have to do these kinds of gymnastics, if your function definition is known to fit a narrow band of possibilities you could use a regular expression to parse the function definition itself. You can get this string from the return of functions.
For example:
function [nelements] = findsizetheta(fh)
defstr = func2str(fh);
test = regexp(defstr, 'theta\((\d+)\)', 'tokens');
if isempty(test)
% Assume we have theta instead of theta(1)
nelements = 1;
else
nelements = max(str2double([test{:}]));
end
end
Which returns 1, 2, and 3 for your example definitions of M.
This assumes that theta is present in you anonymous function and that it is defined as a vector.
Also note that MATLAB cautions against utilizing functions in a programmatic manner, as its behavior may change in future releases. This was tested to function in R2017b.
I am trying to compute log(N(x | mu, sigma)) in MATLAB where
x is the data vector(Dimensions D x 1) , mu(Dimensions D x 1) is mean and sigma(Dimensions D x D) is covariance.
My present implementation is
function [loggaussian] = logmvnpdf(x,mu,Sigma)
[D,~] = size(x);
const = -0.5 * D * log(2*pi);
term1 = -0.5 * ((x - mu)' * (inv(Sigma) * (x - mu)));
term2 = - 0.5 * logdet(Sigma);
loggaussian = const + term1 + term2;
end
function y = logdet(A)
y = log(det(A));
end
For some cases I get an error
Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND =
NaN
I know you will point out that my data is not consistent, but I need to implement the function so that I can get the best approximate instead of throwing an warning. . How do I ensure that I always get a value.
I think the warning comes from using inv(Sigma). According to the documentation, you should avoid using inv where its use can be replaced by \ (mldivide). This will give you both better speed and accuracy.
For your code, instead of inv(Sigma) * (x - mu) use Sigma \ (x - mu).
The following approach should be (a little) less sensitive to ill-conditioning of the covariance matrix:
function logpdf = logmvnpdf (x, mu, K)
n = length (x);
R = chol (K);
const = 0.5 * n * log (2 * pi);
term1 = 0.5 * sum (((R') \ (x - mu)) .^ 2);
term2 = sum (log (diag (R)));
logpdf = - (const + term1 + term2);
end
If K is singular or near-singular, you can still have warnings (or errors) when calling chol.
I want to simplify this symbolic expression, and then take the limit of it (this is not too hard on a paper) using Matlab
syms n x
sn = 8^n * n^2 * (sin(x))^(3*n)
simplify(sn^(1/n))
which results in
ans =
8*n^(2/n)*(sin(x)^(3*n))^(1/n)
This must be like 8 * n^(2/n) * (sin(x))^3. However, if I use
x = sym('x', 'positive'); n = sym('n', 'positive');
sn = 8^n * n^2 * (sin(x))^(3*n)
simplify(sn^(1/n))
to obtain a similar answer and then take limit, I get:
limit(ans, n, inf)
ans =
8*limit(n^(2/n)*(sin(x)^(3*n))^(1/n), n == Inf)
instead of 8*sin(x)^3.
If I simplify this on paper and then take limit, everything works:
limit(8*n^(2/n)*sin(x)^3, n, inf)
ans =
8*sin(x)^3
Is it possible to solve this using Matlab?
MuPAD doesn't simplify the expression, and thus can't take the limit, because you haven't provided the appropriate assumptions. It's not true that an expression such as (sin(x)^n)^(1/n) simplifies to sin(x) if n and x are positive. You need to fully restrict the range of x as it the argument of a periodic function:
x = sym('x','positive');
n = sym('n','real');
assumeAlso(x<=sym(pi));
sn = 8^n * n^2 * (sin(x))^(3*n);
sn2 = simplify(sn^(1/n))
limit(sn2, n, Inf)
which returns the answer you were expecting
ans =
8*sin(x)^3
If you have access to Mathematica, this sort of thing can be accomplished very easily:
Limit[(8^n n^2 Sin[x]^(3 n))^(1/n), n -> \[Infinity], Assumptions -> {n \[Element] Reals, x >= 0, x <= \[Pi]}]