I am not sure if what I have done so far is correct, and I need help with the iterative step as I don't understand what is going on in the algorithm. Here is my code. Help to finish this would be much appreciated. Thank you
function x_opt = out(A,b)
%UNTITLED2 Summary of this function goes here
% Detailed explanation goes here
b_vect = b';
m = size(A,1);
n = size(1,A);
set_B = 1:n;
set_B_Comp = n+1:m;
M = inv(A(set_B, :));
is_opt = 0;
x_temp = M*b_vect(set_B);
h = A*x_temp - b_vect;
y_vect = zeros(m, 1);
y_vect(set_B_Comp) = sign(h(set_B_Comp));
y_vect(set_B) = -inv(A(set_B, :))*(A(set_B_Comp, :))'*y_vect(set_B_Comp);
abs_y_B = abs(y_vect(set_B));
if all(abs_y_B <= 1)
x_opt = x_temp;
...
else
all_index_y_vect_more_than_1 = find(abs(y_vect) >= 1);
set_B_index_y_vect_more_than_1 = intersect(set_B, all_index_y_vect_more_than_1);
s = set_B_index_y_vect_more_than_1(1);
y_s = y(s)
t_vect = zeros(m, 1);
temp = inv(A(set_B,:));
t_vect(set_B_Comp) = -(sign(y_s))*(y(set_B_Comp)).*(A(set_B_Comp, :)*temp(:, s));
cur_min = h(set_B_Comp(1))/t_vect(set_B_Comp(1)) + 1;
cur_r = set_B_Comp(1);
for j = set_B_Comp
h_j = h(j);
t_j = t_vect(j);
temp1 = abs(h_j)/t_j;
if (temp1 < cur_min) && (temp1 > 0)
cur_min = temp1;
cur_r = j;
end
end
r = cur_r;
set_B_new = union(setdiff(set_B, s), r);
set_B_Comp_new = setdiff(1:m,set_B_new);
x_new = inv(A(set_B_new, :))*b_vect(set_B_new);
end
x_opt = x_temp;
end
I don't understand what's going on in your algorithm either. It's written without comments or explanations.
However, you can model your problem as a convex optimization problem. A formulation in Python using cvxpy is then quite simple and readable:
#!/usr/bin/env python3
import cvxpy as cp
import numpy as np
# Coefficient of regularization
alpha = 1e-4
# Dimensionality
N = 400
d = 20
# Synthetic data
A = np.random.randn(N, d)
b = np.random.randn(N)
# Define and solve the CVXPY problem.
x = cp.Variable(d)
objective = cp.sum_squares(A # x - b) + alpha * cp.norm1(x)
prob = cp.Problem(cp.Minimize(objective))
optval = prob.solve()
# Print result.
print("Optimal value ", optval)
print("The optimal x is")
print(x.value)
print("The norm of the residual is ", cp.norm(A # x - b, p=2).value)
and gives
Optimal value 328.41957961297607
The optimal x is
[-0.02041302 -0.16156503 0.07215877 0.00505087 0.01188415 -0.01029848
-0.0237066 0.0370556 0.02205413 0.00137185 0.04055319 -0.01157271
0.00369032 0.06349145 0.07494259 -0.04172275 0.04376864 0.02698337
-0.04696984 0.05245699]
The norm of the residual is 18.122348149231115
I have a joint probability density f(x,y,z) and I wish to find the conditional distribution X|Y=y,Z=z, which is equivalent to treating x as data and y and z as parameters (constants).
For example, if I have X|Y=y,Z=z being the pdf of a N(1-2y,3z^2+2), the function would be:
syms x y z
f(y,z) = 1/sqrt(2*pi*(3*z^2+2)) * exp(-1/(2*(3*z^2+2)) * (x-(1-2*y))^2);
I would like to compare it to the following:
syms mu s L a b
Normal(mu,s) = (1/sqrt(2*pi*s^2)) * exp(-1/(2*s^2) * (x-mu)^2);
Exponential(L) = L * exp(-L*x);
Gamma(a,b) = (b^a / gamma(a)) * x^(a-1)*exp(-b*x);
Beta(a,b) = (1/beta(a,b)) * x^(a-1)*(1-x)^(b-1);
Question
How do I make a program whichDistribution that would be able to print which of these four, f is equivalent to (up to proportionality) with respect to the variable x, and what are the parameters? E.g. f and x as above, the distribution is Normal, mu=1-2*y, s=3*z^2+2.
NB: there would not always be a unique solution, since some distributions are are equivalent (e.g. Gamma(1,L)==Exponential(L))
Desired outputs
syms x y z
f = 1/sqrt(2*pi*(3*z^2+2)) * exp(-1/(2*(3*z^2+2)) * (x-(1-2*y))^2)
whichDistribution(f,x) %Conditional X|Y,Z
% Normal(1-2*y,3*z^2+2)
syms x y
f = y^(1/2)*exp(-(x^2)/2 - y/2 * (1+(4-x)^2+(6-x)^2)) % this is not a pdf because it is missing a constant of proportionality, but it should still work
whichDistribution(f,x) %Conditional X|Y
% Normal(10*y/(2*y+1), 1/(2*y+1))
whichDistribution(f,y) %Conditional Y|X
% Gamma(3/2, x^2 - 10*x + 53/2)
f = exp(-x) %also missing a constant of proportionality
whichDistribution(f,x)
% Exponential(1)
f = 1/(2*pi)*exp(-(x^2)/2 - (y^2)/2)
whichDistribution(f,x)
% Normal(0,1)
whichDistribution(f,y)
% Normal(0,1)
What I have tried so far:
Using solve():
q = solve(f(y,z) == Normal(mu,s), mu, s)
Which gives wrong results, since parameters can't depend on x:
>> q.mu
ans =
(z1^2*(log((2^(1/2)*exp(x^2/(2*z1^2) - (x + 2*y - 1)^2/(6*z^2 + 4)))/(2*pi^(1/2)*(3*z^2 + 2)^(1/2))) + pi*k*2i))/x
>> q.s
ans =
z1
Attempting to simplify f(y,z) up to proportionality (in x variable) using a propto() function that I wrote:
>> propto(f(y,z),x)
ans =
exp(-(x*(x + 4*y - 2))/(2*(3*z^2 + 2)))
>> propto(Normal(mu,s),x)
ans =
exp((x*(2*mu - x))/(2*s^2))
This is almost on the money, since it is easy to spot that s^2=3*z^2 + 2 and 2*mu=-(4*y - 2), but I don't know how to deduce this programmatically.
In case it is useful: propto(f,x) attempts to simplify f by dividing f by children of f which don't involve x, and then output whichever form has the least number of children. Here is the routine:
function out = propto(f,x)
oldf = f;
newf = propto2(f,x);
while (~strcmp(char(oldf),char(newf))) % if the form of f changed, do propto2 again. When propto2(f) == f, stop
oldf = newf;
newf = propto2(oldf,x);
end
out = newf;
end
function out = propto2(f,x)
t1 = children(expand(f)); % expanded f
i1 = ~has([t1{:}],x);
out1 = simplify(f/prod([t1{i1}])); % divides expanded f by terms that do not involve x
t2 = children(f); % unexpanded f
i2 = ~has([t2{:}],x);
out2 = simplify(f/prod([t2{i2}])); % divides f by terms that do not involve x
A = [f, symlength(f); out1, symlength(out1); out2, symlength(out2)];
A = sortrows(A,2); % outputs whichever form has the fewest number of children
out = A(1,1);
end
function L = symlength(f)
% counts the number of children of f by repeatingly applying children() to itself
t = children(f);
t = [t{:}];
L = length(t);
if (L == 1)
return
end
oldt = f;
while(~strcmp(char(oldt),char(t)))
oldt = t;
t = children(t);
t = [t{:}];
t = [t{:}];
end
L = length(t);
end
edit: added desired outputs
edit2: clarified the desired function
I have managed to solve my own problem using solve() from Symbolic Toolbox. There were two issues with my original approach: I needed to set up n simultaneous equations for n parameters, and the solve() doesn't cope well with exponentials:
solve(f(3) == g(3), f(4) == g(4), mu,s)
yields no solutions, but
logf(x) = feval(symengine,'simplify',log(f),'IgnoreAnalyticConstraints');
logg(x) = feval(symengine,'simplify',log(g),'IgnoreAnalyticConstraints');
solve(logf(3) == logg(3), logf(4) == logg(4), mu,s)
yields good solutions.
Solution
Given f(x), for each PDF g(x) we attempt to solve simultaneously
log(f(r1)) == log(g(r1)) and log(f(r2)) == log(g(r2))
for some simple non-equal numbers r1, r2. Then output g for which the solution has the lowest complexity.
The code is:
function whichDist(f,x)
syms mu s L a b x0 x1 x2 v n p g
f = propto(f,x); % simplify up to proportionality
logf(x) = feval(symengine,'simplify',log(f),'IgnoreAnalyticConstraints');
Normal(mu,s,x) = propto((1/sqrt(2*pi*s)) * exp(-1/(2*s) * (x-mu)^2),x);
Exponential(L,x) = exp(-L*x);
Gamma(a,b,x) = x^(a-1)*exp(-b*x);
Beta(a,b,x) = x^(a-1)*(1-x)^(b-1);
ChiSq(v,x) = x^(v/2 - 1) * exp(-x/2);
tdist(v,x) = (1+x^2 / v)^(-(v+1)/2);
Cauchy(g,x0,x) = 1/(1+((x-x0)/g)^2);
logf = logf(x);
best_sol = {'none', inf};
r1 = randi(10); r2 = randi(10); r3 = randi(10);
while (r1 == r2 || r2 == r3 || r1 == r3) r1 = randi(10); r2 = randi(10); r3 = randi(10); end
%% check Exponential:
if (propto(logf,x) == x) % pdf ~ exp(K*x), can read off Lambda directly
soln = -logf/x;
if (~has(soln,x)) % any solution can't depend on x
fprintf('\nExponential: rate L = %s\n\n', soln);
return
end
end
%% check Chi-sq:
if (propto(logf + x/2, log(x)) == log(x)) % can read off v directly
soln = 2*(1+(logf + x/2) / log(x));
if (~has(soln,x))
dof = feval(symengine,'simplify',soln,'IgnoreAnalyticConstraints');
fprintf('\nChi-Squared: v = %s\n\n', dof);
return
end
end
%% check t-dist:
h1 = propto(logf,x);
h = simplify(exp(h1) - 1);
if (propto(h,x^2) == x^2) % pdf ~ exp(K*x), can read off Lambda directly
soln = simplify(x^2 / h);
if (~has(soln,x))
fprintf('\nt-dist: v = %s\n\n', soln);
return
end
end
h = simplify(exp(-h1) - 1); % try again if propto flipped a sign
if (propto(h,x^2) == x^2) % pdf ~ exp(K*x), can read off Lambda directly
soln = simplify(x^2 / h);
if (~has(soln,x))
fprintf('\nt-dist: v = %s\n\n', soln);
return
end
end
%% check Normal:
logn(x) = feval(symengine,'simplify',log(Normal(mu,s,x)),'IgnoreAnalyticConstraints');
% A = (x - propto(logf/x, x))/2;
% B = simplify(-x/(logf/x - mu/s)/2);
% if (~has(A,x) && ~has(B,x))
% fprintf('Normal: mu = %s, s^2 = %s', A, B);
% return
% end
logf(x) = logf;
try % attempt to solve the equation
% solve simultaneously for two random non-equal integer values r1,r2
qn = solve(logf(r1) == logn(r1), logf(r2) == logn(r2), mu, s);
catch error
end
if (exist('qn','var')) % if solve() managed to run
if (~isempty(qn.mu) && ~isempty(qn.s) && ~any(has([qn.mu,qn.s],x))) % if solution exists
complexity = symlength(qn.mu) + symlength(qn.s);
if complexity < best_sol{2} % store best solution so far
best_sol{1} = sprintf('Normal: mu = %s, s^2 = %s', qn.mu, qn.s);
best_sol{2} = complexity;
end
end
end
%% check Cauchy:
logcau(x) = feval(symengine,'simplify',log(Cauchy(g,x0,x)),'IgnoreAnalyticConstraints');
f(x) = f;
try
qcau = solve(f(r1) == Cauchy(g,x0,r1), f(r2) == Cauchy(g,x0,r2), g, x0);
catch error
end
if (exist('qcau','var'))
if (~isempty(qcau.g) && ~isempty(qcau.x0) && ~any(has([qcau.g(1),qcau.x0(1)],x)))
complexity = symlength(qcau.g(1)) + symlength(qcau.x0(1));
if complexity < best_sol{2}
best_sol{1} = sprintf('Cauchy: g = %s, x0 = %s', qcau.g(1), qcau.x0(1));
best_sol{2} = complexity;
end
end
end
f = f(x);
%% check Gamma:
logg(x) = feval(symengine,'simplify',log(Gamma(a,b,x)),'IgnoreAnalyticConstraints');
t = children(logf); t = [t{:}];
if (length(t) == 2)
if (propto(t(1),log(x)) == log(x) && propto(t(2),x) == x)
soln = [t(1)/log(x) + 1, -t(2)/x];
if (~any(has(soln,x)))
fprintf('\nGamma: shape a = %s, rate b = %s\n\n',soln);
return
end
elseif (propto(t(2),log(x)) == log(x) && propto(t(1),x) == x)
soln = [t(2)/log(x) + 1, -t(1)/x];
if (~any(has(soln,x)))
fprintf('\nGamma: shape a = %s, rate b = %s\n\n',soln);
return
end
end
end
logf(x) = logf;
try % also try using solve(), just in case.
qg = solve(logf(r1) == logg(r1), logf(r2) == logg(r2), a, b);
catch error
end
if (exist('qg','var'))
if (~isempty(qg.a) && ~isempty(qg.b) && ~any(has([qg.a,qg.b],x)))
complexity = symlength(qg.a) + symlength(qg.b);
if complexity < best_sol{2}
best_sol{1} = sprintf('Gamma: shape a = %s, rate b = %s', qg.a, qg.b);
best_sol{2} = complexity;
end
end
end
logf = logf(x);
%% check Beta:
B = feval(symengine,'simplify',log(propto(f,x-1)),'IgnoreAnalyticConstraints');
if (propto(B,log(x-1)) == log(x-1))
B = B / log(x-1) + 1;
A = f / (x-1)^(B-1);
A = feval(symengine,'simplify',log(abs(A)),'IgnoreAnalyticConstraints');
if (propto(A,log(abs(x))) == log(abs(x)))
A = A / log(abs(x)) + 1;
if (~any(has([A,B],x)))
fprintf('\nBeta1: a = %s, b = %s\n\n', A, B);
return
end
end
elseif (propto(B,log(1-x)) == log(1-x))
B = B / log(1-x);
A = simplify(f / (1-x)^(B-1));
A = feval(symengine,'simplify',log(A),'IgnoreAnalyticConstraints');
if (propto(A,log(x)) == log(x))
A = A / log(x) + 1;
if (~any(has([A,B],x)))
fprintf('\nBeta1: a = %s, b = %s\n\n', A, B);
return
end
end
end
%% Print solution with lowest complexity
fprintf('\n%s\n\n', best_sol{1});
end
Tests:
>> syms x y z
>> f = y^(1/2)*exp(-(x^2)/2 - y/2 * (1+(4-x)^2+(6-x)^2))
>> whichDist(f,x)
Normal: mu = (10*y)/(2*y + 1), s^2 = 1/(2*y + 1)
>> whichDist(f,y)
Gamma: a = 3/2, b = x^2 - 10*x + 53/2
>> Beta(a,b,x) = propto((1/beta(a,b)) * x^(a-1)*(1-x)^(b-1), x);
>> f = Beta(1/z + 7*y/(1-sqrt(z)), z/y + 1/(1-z), x)
Beta: a = -(7*y*z - z^(1/2) + 1)/(z*(z^(1/2) - 1)), b = -(y + z - z^2)/(y*(z - 1))
All correct.
Sometimes bogus answers if the parameters are numeric:
whichDist(Beta(3,4,x),x)
Beta: a = -(pi*log(2)*1i + pi*log(3/10)*1i - log(2)*log(3/10) + log(2)*log(7/10) - log(3/10)*log(32) + log(2)*log(1323/100000))/(log(2)*(log(3/10) - log(7/10))), b = (pi*log(2)*1i + pi*log(7/10)*1i + log(2)*log(3/10) - log(2)*log(7/10) - log(7/10)*log(32) + log(2)*log(1323/100000))/(log(2)*(log(3/10) - log(7/10)))
So there is room for improvement and I will still award bounty to a better solution than this.
Edit: Added more distributions. Improved Gamma and Beta distribution identifications by spotting them directly without needing solve().
Posting here vs math.stackexchange because I think my issue is syntax:
I'm trying to analyze the 2nd order ODE: y'' + 2y' + 2y = e^(-x) * sin(x) using MATLAB code for the midpoint method. I first converted the ODE to a system of 1st order equations and then tried to apply it below, but as the discretizations [m] are increased, the output is stopping at .2718. For example, m=11 yields:
ans =
0.2724
and m=101:
ans =
0.2718
and m=10001
ans =
0.2718
Here's the code:
function [y,t] = ODEsolver_midpointND(F,y0,a,b,m)
if nargin < 5, m = 11; end
if nargin < 4, a = 0; b = 1; end
if nargin < 3, a = 0; b = 1; end
if nargin < 2, error('invalid number of inputs'); end
t = linspace(a,b,m)';
h = t(2)-t(1);
n = length(y0);
y = zeros(m,n);
y(1,:) = y0;
for i=2:m
Fty = feval(F,t(i-1),y(i-1,:));
th = t(i-1)+h/2;
y(i,:) = y(i-1,:) + ...
h*feval(F,th,y(i-1,:)+(h/2)*Fty );
end
Separate file:
function F = Fexample1(t,y)
F1 = y(2);
F2 = exp(-t).*sin(t)-2.*y(2)-2.*y(1);
F = [F1,F2];
Third file:
[Y,t] = ODEsolver_midpointND('Fexample1',[0 0],0,1,11);
Ye = [(1./2).*exp(-t).*(sin(t)-t.*cos(t)) (1./2).*exp(-t).*((t-1).*sin(t)- t.*cos(t))];
norm(Y-Ye,inf)
Your ODE solver looks to me like it should work - however there's a typo in the analytic solution you're comparing to. It should be
Ye = [(1./2).*exp(-t).*(sin(t)-t.*cos(t)) (1./2).*exp(-t).*((t-1).*sin(t)+ t.*cos(t))];
i.e. with a + sign before the t.*cos(t) term in the derivative.
The following is my full code: (Most of it isn't useful for what I'm asking, but I just put in the entire code for context, the part of the code that is causing me trouble is towards the end)
clc
clear
P = xlsread('b3.xlsx', 'P');
d = xlsread('b3.xlsx', 'd');
CM = xlsread('b3.xlsx', 'Cov');
Original_PD = P; %Store original PD
LM_rows = size(P,1)+1; %Expected LM rows
LM_columns = size(P,2); %Expected LM columns
LM_FINAL = zeros(LM_rows,LM_columns); %Dimensions of LM_FINAL
% Start of the outside loop
for k = 1:size(P,2)
P = Original_PD(:,k);
interval = cell(size(P,1)+2,1);
for i = 1:size(P,1)
interval{i,1} = NaN(size(P,1),2);
interval{i,1}(:,1) = -Inf;
interval{i,1}(:,2) = d;
interval{i,1}(i,1) = d(i,1);
interval{i,1}(i,2) = Inf;
end
interval{i+1,1} = [-Inf*ones(size(P,1),1) d];
interval{i+2,1} = [d Inf*ones(size(P,1),1)];
c = NaN(size(interval,1),1);
for i = 1:size(c,1)
c(i,1) = mvncdf(interval{i,1}(:,1),interval{i,1}(:,2),0,CM);
end
c0 = c(size(P,1)+1,1);
f = c(size(P,1)+2,1);
c = c(1:size(P,1),:);
b0 = exp(1);
b = exp(1)*P;
syms x;
eqn = f*x;
for i = 1:size(P,1)
eqn = eqn*(c0/c(i,1)*x + (b(i,1)-b0)/c(i,1));
end
eqn = c0*x^(size(P,1)+1) + eqn - b0*x^size(P,1);
x0 = solve(eqn);
for i = 1:size(x0)
id(i,1) = isreal(x0(i,1));
end
x0 = x0(id,:);
x0 = x0(x0 > 0,:);
clear x;
for i = 1:size(P,1)
x(i,:) = (b(i,1) - b0)./(c(i,1)*x0) + c0/c(i,1);
end
x = [x0'; x];
x = double(x);
x = x(:,sum(x <= 0,1) == 0)
lamda = -log(x);
LM_FINAL(:,k) = lamda;
end
% end of the outside loop
The important part of the above loop is towards the end:
x = x(:,sum(x <= 0,1) == 0)
This condition is sometimes not satisfied and hence the variable x is empty, which means LM_FINAL(:,k) = lamda is also empty. When this happens, I get the error:
x =
Empty matrix: 43-by-0
Improper assignment with rectangular empty matrix.
Error in Solution (line 75)
LM_FINAL(:,k) = lamda;
How can I skip this error so that the column for LM_FINAL remains as empty, but the loop continues (so that the rest of LM_FINAL's columns are filled) rather than terminating?
You can use try and catch phrase to explicitly handle errors inside loop (or elsewhere in your code).