I see that it is possible to use regress/regstats for OLS, and I found an online implementation of L1-Regression (Laplace), but I can't quite seem to figure out how to implement t distributed error terms. I have tried maximizing the log-likelihood of the residuals, but don't seem to be coming up with the right answer.
classdef student < handle
methods (Static)
% Find the sigma that maximizes the Log Liklihood function given a B
function s = findLonS(r,df)
n = length(r);
% if x ~ t location, scale distribution with df
% degrees of freedom, then (x-u)/sigma ~ t(df)
f = #(s) -sum(log(tpdf(r ./ s, df)));
s = fminunc(f, (r'*r)/n);
end
function B = regress(X,Y,df)
[n,m] = size(X);
bInit = ones(m, 1);
r = (Y - X*bInit);
s = student.findLonS(r, df);
% if x ~ t location, scale distribution with df
% degrees of freedom, then (x-u)/sigma ~ t(df)
f = #(b) -sum(log(tpdf((Y - X*b) ./ s, df)));
options = optimset('MaxFunEvals', 10000, 'TolX', 1e-16, 'TolFun', 1e-16);
[B, fval] = fminunc(f, bInit, options);
end
end
end
Comparing to an R implementation (which I know has been tested and is accurate), the solutions I am getting to this is wrong.
Any suggestions for fixing or ideas where I could find a solution already available?
my guess would be you have to adjust the scale s for the given b. This would either mean doing something like alternatively optimizing b, then adjusting s, and optimizing b again, or possibly rewriting your objective as
f = #(b)(-sum(log(tpdf((Y-X*b) ./ student.findLonS(Y-X*b,df),df))));
Related
For example, FX = x ^ 2 + sin (x)
Just for curiosity, I don't want to use the CVX toolbox to do this.
You can check this within some interval [a,b] by checking if the second derivative is nonnegative. For this you have to define a vector of x-values, find the numerical second derivative and check whether it is not too negative:
a = 0;
b = 1;
margin = 1e-5;
point_count = 100;
f=#(x) x.^2 + sin(x);
x = linspace(a, b, point_count)
is_convex = all(diff(x, 2) > -margin);
Since this is a numerical test, you need to adjust the parameter to the properties of the function, that is if the function does wild things on a small scale we might not be able to pick it up. E.g. with the parameters above the test will falsely report the function f=#(x)sin(99.5*2*pi*x-3) as convex.
clear
syms x real
syms f(x) d(x) d1(x)
f = x^2 + sin(x)
d = diff(f,x,2)==0
d1 = diff(f,x,2)
expSolution = solve(d, x)
if size(expSolution,1) == 0
if eval(subs(d1,x,0))>0
disp("condition 1- the graph is concave upward");
else
disp("condition 2 - the graph is concave download");
end
else
disp("condition 3 -- not certain")
end
Hi i've been asked to solve SIR model using fsolve command in MATLAB, and Euler 3 point backward. I'm really confused on how to proceed, please help. This is what i have so far. I created a function for 3BDF scheme but i'm not sure how to proceed with fsolve and solve the system of nonlinear ODEs. The SIR model is shown as and 3BDF scheme is formulated as
clc
clear all
gamma=1/7;
beta=1/3;
ode1= #(R,S,I) -(beta*I*S)/(S+I+R);
ode2= #(R,S,I) (beta*I*S)/(S+I+R)-I*gamma;
ode3= #(I) gamma*I;
f(t,[S,I,R]) = [-(beta*I*S)/(S+I+R); (beta*I*S)/(S+I+R)-I*gamma; gamma*I];
R0=0;
I0=10;
S0=8e6;
odes={ode1;ode2;ode3}
fun = #root2d;
x0 = [0,0];
x = fsolve(fun,x0)
function [xs,yb] = ThreePointBDF(f,x0, xmax, h, y0)
% This function should return the numerical solution of y at x = xmax.
% (It should not return the entire time history of y.)
% TO BE COMPLETED
xs=x0:h:xmax;
y=zeros(1,length(xs));
y(1)=y0;
yb(1)=y0+f(x0,y0)*h;
for i=1:length(xs)-1
R =R0;
y1(i+1,:) = fsolve(#(u) u-2*h/3*f(t(i+1),u) - R, y1(i-1,:)+2*h*F(i,:))
S = S0;
y2(i+1,:) = fsolve(#(u) u-2*h/3*f(t(i+1),u) - S, y2(i-1,:)+2*h*F(i,:))
I= I0;
y3(i+1,:) = fsolve(#(u) u-2*h/3*f(t(i+1),u) - I, y3(i-1,:)+2*h*F(i,:))
end
end
You have an implicit equation
y(i+1) - 2*h/3*f(t(i+1),y(i+1)) = G = (4*y(i) - y(i-1))/3
where the right-side term G is constant in the call to fsolve, that is, during the solution of the implicit step equation.
Note that this is for the vector valued system y'(t)=f(t,y(t)) where
f(t,[S,I,R]) = [-(beta*I*S)/(S+I+R); (beta*I*S)/(S+I+R)-I*gamma; gamma*I];
To solve this write
G = (4*y(i,:) - y(i-1,:))/3
y(i+1,:) = fsolve(#(u) u-2*h/3*f(t(i+1),u) - G, y(i-1,:)+2*h*F(i,:))
where a midpoint step is used to get an order 2 approximation as initial guess, F(i,:)=f(t(i),y(i,:)). Add solver options for error tolerances as necessary, you want the error in the implicit equation smaller than the truncation error O(h^3) of the step. One can also keep only a short array of function values, then one has to be careful for the correspondence of the position in the short array to the time index.
Using all that and a reference solution by a higher order standard solver produces the following error graphs for the components
where one can see that the first order error of the constant first step results in a first order global error, while with a second order error in the first step using the Euler method results in a clear second order global error.
Implement the method in general terms
from scipy.optimize import fsolve
def BDF2(f,t,y0,y1):
N, h = len(t)-1, t[1]-t[0];
y = (N+1)*[np.asarray(y0)];
y[1] = y1;
for i in range(1,N):
t1, G = t[i+1], (4*y[i]-y[i-1])/3
y[i+1] = fsolve(lambda u: u-2*h/3*f(t1,u)-G, y[i-1]+2*h*f(t[i],y[i]), xtol=1e-3*h**3)
return np.vstack(y)
Set up the model to be solved
gamma=1/7;
beta=1/3;
print beta, gamma
y0 = np.array([8e6, 10, 0])
P = sum(y0); y0 = y0/P
def f(t,y): S,I,R = y; trns = beta*S*I/(S+I+R); recv=gamma*I; return np.array([-trns, trns-recv, recv])
Compute a reference solution and method solutions for the two initialization variants
from scipy.integrate import odeint
tg = np.linspace(0,120,25*128)
yg = odeint(f,y0,tg,atol=1e-12, rtol=1e-14, tfirst=True)
M = 16; # 8,4
t = tg[::M];
h = t[1]-t[0];
y1 = BDF2(f,t,y0,y0)
e1 = y1-yg[::M]
y2 = BDF2(f,t,y0,y0+h*f(0,y0))
e2 = y2-yg[::M]
Plot the errors, computation as above, but embedded in the plot commands, could be separated in principle by first computing a list of solutions
fig,ax = plt.subplots(3,2,figsize=(12,6))
for M in [16, 8, 4]:
t = tg[::M];
h = t[1]-t[0];
y = BDF2(f,t,y0,y0)
e = (y-yg[::M])
for k in range(3): ax[k,0].plot(t,e[:,k],'-o', ms=1, lw=0.5, label = "h=%.3f"%h)
y = BDF2(f,t,y0,y0+h*f(0,y0))
e = (y-yg[::M])
for k in range(3): ax[k,1].plot(t,e[:,k],'-o', ms=1, lw=0.5, label = "h=%.3f"%h)
for k in range(3):
for j in range(2): ax[k,j].set_ylabel(["$e_S$","$e_I$","$e_R$"][k]); ax[k,j].legend(); ax[k,j].grid()
ax[0,0].set_title("Errors: first step constant");
ax[0,1].set_title("Errors: first step Euler")
I'm suppose to make a model for an algae population. Here's the code I have so far (all written from examples online). When i run Solve_algaepop, it just hangs for a long time.
Any ideas why? Is there any obvious thing I'm doing wrong? The equations are from a research paper.
This is Solve_algaepop.m. In the equations for r1 and r2, P10 and P20 are supposed to be the values
P1 = x(1) and P2 = x(2) defined in algaepop_model.m. I don't know how to access the values when I'm in Solve_algaepop.m
% Initial conditions
P10 = 560000; %from Chattopadhyay; estimated from graph
P20 = 250000; %same as above
Z0 = 280000; %
N0 = 0.6; %from Edwards
%some variables that the expressions of the parameters use
lambda = .6;
mu = .035;
k = 0.05;
%define parameters (start with estimates from Edwards paper):
r1 = (N0/(.03+N0))*((.2*P10)/(.2 + .4*P10));
r2 = (N0/(.03+N0))*((.2*P20)/(.2 + .4*P20));
a = Z0*((lambda*P10^2)/(mu^2 + P10^2));%G1: zooplankton growth function from Edwards paper
% m1 = .15; %r in Edwards paper
m1 = .075; % q in Edwards
m2 = .15;% r in Edwards paper
m3 = .15; % r in Edwards paper
d = 0.5;
cN = k;%*(N-N0);
par = [r1 r2 a m1 m2 m3 d cN]; % Creates vector of parameter values to pass to the ode solver
tspan = 0:1:300; %(Note: can also use the function linspace)
x0 = [P10 P20 Z0 N0]; % Creates vector of initial conditions
[t,x] = ode45(#algaepop_model,tspan,x0,[],par);
plot(t,x)
And here is algaepop_model.m
function dxdt = algaepop_model(t,x,par)
P1 = x(1);
P2 = x(2);
Z = x(3);
N = x(4);
r1 = par(1);
r2 = par(2);
a = par(3);
m1 = par(4);
m2 = par(5);
m3 = par(6);
d = par(7);
cN = par(8);
dxdt = zeros(4,1);
dxdt(1) = r1*N*P1 - m3*P1 - a*P1*Z;
dxdt(2) = r2*N*P2 - a*P2*Z - m2*P2;
dxdt(3) = a*P2*Z + a*P1*Z - m1*Z;
dxdt(4) = d*m2*P2 + d*m1*Z + d*m3*P1 + cN - r2*N*P2 - r1*N*P1;
end
Thanks for the help.
Let's debug. One of the simplest things that you can do is print out t and x inside of your integration function, algaepop_model. As soon as you do this, you'll probably notice what's happening: ode45 is taking extremely small steps. They're on the order of 1.9e-9. With steps that small, it will take forever to simulate to t = 300 (and even longer if you print stuff out on each step).
This might be caused by a poor choice of initial conditions, poor scaling or dimensionalization, a typo resulting in the wrong equations, or simply that you're using an inappropriate solver (and/or tolerances) for the particular problem. I can't really address the first two situations and must assume that you don't have any errors. Thus, in this case you have what is effectively a stiff system and ode45 is not a particularly good choice in such cases. Simply changing the solver to ode15s results in the following plot almost immediately:
As you can see, there are very large changes over a short period of time in the initial portion of the plot. If you zoom in you'' see that the huge spike happens in the first unit of time (you might output more time points or just let tspan = [0 300]). Some state variables are changing rapidly while others are varying more gradually. Such high frequencies and differences in time scales are the hallmarks of stiff systems. I'd suggest that, in addition to confirming that your code is correct, you also try adjusting the integration tolerances as well via odeset. Make sure that tighter tolerances produce qualitatively similar results. You can also try the other stiff solvers in the ODE suite if you like.
Lastly, it's more efficient and up-to-date to pass your parameters via the function handle itself rather than how you're doing it. Here's how:
[t,x] = ode15s(#(t,x)algaepop_model(t,x,par),tspan,x0);
In an attempt to vectorize a particular piece of Matlab code, I could not find a straightforward function to generate a list of the binomial coefficients. The best I could find was nchoosek, but for some inexplicable reason this function only accepts integers (not vectors of integers). My current solution looks like this:
mybinom = #(n) arrayfun(#nchoosek, n*ones(1,n), 1:n)
This generates the set of binomial coefficients for a given value of n. However, since the binomial coefficients are always symmetric, I know that I am doing twice as much work as necessary. I'm sure that I could create a solution that exploits the symmetry, but I'm sure that it would be at the expense of readability.
Is there a more elegant solution than this, perhaps using a Matlab function that I am not aware of? Note that I am not interested in using the symbolic toolbox.
If you want to minimize operations you can go along these lines:
n = 6;
k = 1:n;
result = [1 cumprod((n-k+1)./k)]
>> result
result =
1 6 15 20 15 6 1
This requires very few operations per coefficient, because each cofficient is obtained exploiting the previously computed one.
You can reduce the number of operations by approximately half if you take into account the symmetry:
m1 = floor(n/2);
m2 = ceil(n/2);
k = 1:m2;
result = [1 cumprod((n-k+1)./k)];
result(n+1:-1:m1+2) = result(1:m2);
What about a modified version of Luis Mendo's solution - but in logarithms:
n = 1e4;
m1 = floor(n/2);
m2 = ceil(n/2);
k = 1:m2;
% Attempt to compute real value
out0 = [1 cumprod((n-k+1)./k)];
out0(n+1:-1:m1+2) = out0(1:m2);
% In logarithms
out1 = [0 cumsum((log(n-k+1)) - log(k))];
out1(n+1:-1:m1+2) = out1(1:m2);
plot(log(out0) - out1, 'o-')
The advantage of working with logarithms is that you can set n = 1e4; and still obtain a good approximation of the real value (nchoosek(1e4, 5e3) returns Inf and this is not a good approximation at all!).
EDIT following horchler's comment
You can use the gammaln function to obtain the same result but it's not faster. The two approximations seem to be quite different:
n = 1e7;
m1 = floor(n/2);
m2 = ceil(n/2);
k = 1:m2;
% In logarithms
tic
out1 = [0 cumsum((log(n-k+1)) - log(k))];
out1(n+1:-1:m1+2) = out1(1:m2);
toc
% Elapsed time is 0.912649 seconds.
tic
k = 0:m2;
out2 = gammaln(n + 1) - gammaln(k + 1) - gammaln(n - k + 1);
out2(n+1:-1:m1+2) = out2(1:m2);
toc
% Elapsed time is 1.020188 seconds.
tmp = out2 - out1;
plot(tmp, '.')
prctile(tmp, [0 2.5 25 50 75 97.5 100])
% 1.0e-006 *
% -0.2217 -0.1462 -0.0373 0.0363 0.1225 0.2943 0.3846
Is adding three gammaln worse than adding n logarithms? Or viceversa?
This works for Octave only
You can use bincoeff function.
Example: bincoeff(5, 0:5)
EDIT :
Only improvement I can think of goes like this. Maybe you already thought this trivial solution and didn't like it.
# Calculate only the first half
mybinomhalf = #(n) arrayfun(#nchoosek, n*ones(1,n/2+1), 0:n/2)
# pad your array symmetrically
mybinom = #(n) padarray(mybinomhalf(n), [0 n/2], 'symmetric', 'post')
# I couldn't test it and this line may not work
I have to determine, which filter was used on a random picture - is there a common way to detect the right one (gaussian, prewitt, sobel, average, ...) or would it be clever to code some sort of 'brute force'-detection ?
I tried to find it with Matlab, but I have no glue how to search more efficiently. At the moment it`s like finding a needle in a Haystack. I also thought of using some bash-script and imagemagick, but this would be to resource hungry.
I tought this wouldn't be a problem, but it's very time-consuming to guess a filter and try it like this
f = fspecial('gaussian', [3 3], 1);
res = imfilter(orginal, f);
corr2(res, pic);
Let f be the original image, g be the filtered one, and h the filter applied to f, so that:
f * h = g
Passing that to the frequency domain:
F.H = G, so H = G/F
The problem is that inverting F is VERY sensitive to noise.
How to implement that in MATLAB:
close all;
f = imread('cameraman.tif');
[x,y] = size(f);
figure,imshow(f);
h = fspecial('motion', 20, 40); % abitrary filter just for testing the algorithm
F = fft2(f);
H = fft2(h,x,y);
G = F.*H;
g = ifft2(G); % the filtered image
figure, imshow(g/max(g(:)));
% Inverting the original image
epsilon = 10^(-10);
small_values = find(abs(F)<epsilon);
F(small_values) = epsilon;
F_i = ones(x,y)./F;
H_calculated = G.*F_i;
h_calculated = ifft2(H_calculated);
% remove really small values to try to infer the original size of h
r = sum(h_calculated,1)<epsilon;
c = sum(h_calculated,2)<epsilon;
h_real = h_calculated(~r,~c);
% Calculate error
% redo the filtering with the found filter
figure,g_comp = ifft2(fft2(f).*fft2(h_real,x,y));
imshow(g_comp/max(g_comp(:)));
rmse = sqrt(mean(mean((double(g_comp) - double(g)).^2,2),1))
edit: Just to explain the epsilon part:
It can be that some values in F are zero, or very close to zero. If we try to invert F with these small values, we would have problems with infinity. The easy way to solve that is to truncate every value in F that is smaller than an arbitrarily small limit, epsilon on the code.
Mathematically, what was done is this:
For all F < epsilon, F = epsilon