I have a joint probability density f(x,y,z) and I wish to find the conditional distribution X|Y=y,Z=z, which is equivalent to treating x as data and y and z as parameters (constants).
For example, if I have X|Y=y,Z=z being the pdf of a N(1-2y,3z^2+2), the function would be:
syms x y z
f(y,z) = 1/sqrt(2*pi*(3*z^2+2)) * exp(-1/(2*(3*z^2+2)) * (x-(1-2*y))^2);
I would like to compare it to the following:
syms mu s L a b
Normal(mu,s) = (1/sqrt(2*pi*s^2)) * exp(-1/(2*s^2) * (x-mu)^2);
Exponential(L) = L * exp(-L*x);
Gamma(a,b) = (b^a / gamma(a)) * x^(a-1)*exp(-b*x);
Beta(a,b) = (1/beta(a,b)) * x^(a-1)*(1-x)^(b-1);
Question
How do I make a program whichDistribution that would be able to print which of these four, f is equivalent to (up to proportionality) with respect to the variable x, and what are the parameters? E.g. f and x as above, the distribution is Normal, mu=1-2*y, s=3*z^2+2.
NB: there would not always be a unique solution, since some distributions are are equivalent (e.g. Gamma(1,L)==Exponential(L))
Desired outputs
syms x y z
f = 1/sqrt(2*pi*(3*z^2+2)) * exp(-1/(2*(3*z^2+2)) * (x-(1-2*y))^2)
whichDistribution(f,x) %Conditional X|Y,Z
% Normal(1-2*y,3*z^2+2)
syms x y
f = y^(1/2)*exp(-(x^2)/2 - y/2 * (1+(4-x)^2+(6-x)^2)) % this is not a pdf because it is missing a constant of proportionality, but it should still work
whichDistribution(f,x) %Conditional X|Y
% Normal(10*y/(2*y+1), 1/(2*y+1))
whichDistribution(f,y) %Conditional Y|X
% Gamma(3/2, x^2 - 10*x + 53/2)
f = exp(-x) %also missing a constant of proportionality
whichDistribution(f,x)
% Exponential(1)
f = 1/(2*pi)*exp(-(x^2)/2 - (y^2)/2)
whichDistribution(f,x)
% Normal(0,1)
whichDistribution(f,y)
% Normal(0,1)
What I have tried so far:
Using solve():
q = solve(f(y,z) == Normal(mu,s), mu, s)
Which gives wrong results, since parameters can't depend on x:
>> q.mu
ans =
(z1^2*(log((2^(1/2)*exp(x^2/(2*z1^2) - (x + 2*y - 1)^2/(6*z^2 + 4)))/(2*pi^(1/2)*(3*z^2 + 2)^(1/2))) + pi*k*2i))/x
>> q.s
ans =
z1
Attempting to simplify f(y,z) up to proportionality (in x variable) using a propto() function that I wrote:
>> propto(f(y,z),x)
ans =
exp(-(x*(x + 4*y - 2))/(2*(3*z^2 + 2)))
>> propto(Normal(mu,s),x)
ans =
exp((x*(2*mu - x))/(2*s^2))
This is almost on the money, since it is easy to spot that s^2=3*z^2 + 2 and 2*mu=-(4*y - 2), but I don't know how to deduce this programmatically.
In case it is useful: propto(f,x) attempts to simplify f by dividing f by children of f which don't involve x, and then output whichever form has the least number of children. Here is the routine:
function out = propto(f,x)
oldf = f;
newf = propto2(f,x);
while (~strcmp(char(oldf),char(newf))) % if the form of f changed, do propto2 again. When propto2(f) == f, stop
oldf = newf;
newf = propto2(oldf,x);
end
out = newf;
end
function out = propto2(f,x)
t1 = children(expand(f)); % expanded f
i1 = ~has([t1{:}],x);
out1 = simplify(f/prod([t1{i1}])); % divides expanded f by terms that do not involve x
t2 = children(f); % unexpanded f
i2 = ~has([t2{:}],x);
out2 = simplify(f/prod([t2{i2}])); % divides f by terms that do not involve x
A = [f, symlength(f); out1, symlength(out1); out2, symlength(out2)];
A = sortrows(A,2); % outputs whichever form has the fewest number of children
out = A(1,1);
end
function L = symlength(f)
% counts the number of children of f by repeatingly applying children() to itself
t = children(f);
t = [t{:}];
L = length(t);
if (L == 1)
return
end
oldt = f;
while(~strcmp(char(oldt),char(t)))
oldt = t;
t = children(t);
t = [t{:}];
t = [t{:}];
end
L = length(t);
end
edit: added desired outputs
edit2: clarified the desired function
I have managed to solve my own problem using solve() from Symbolic Toolbox. There were two issues with my original approach: I needed to set up n simultaneous equations for n parameters, and the solve() doesn't cope well with exponentials:
solve(f(3) == g(3), f(4) == g(4), mu,s)
yields no solutions, but
logf(x) = feval(symengine,'simplify',log(f),'IgnoreAnalyticConstraints');
logg(x) = feval(symengine,'simplify',log(g),'IgnoreAnalyticConstraints');
solve(logf(3) == logg(3), logf(4) == logg(4), mu,s)
yields good solutions.
Solution
Given f(x), for each PDF g(x) we attempt to solve simultaneously
log(f(r1)) == log(g(r1)) and log(f(r2)) == log(g(r2))
for some simple non-equal numbers r1, r2. Then output g for which the solution has the lowest complexity.
The code is:
function whichDist(f,x)
syms mu s L a b x0 x1 x2 v n p g
f = propto(f,x); % simplify up to proportionality
logf(x) = feval(symengine,'simplify',log(f),'IgnoreAnalyticConstraints');
Normal(mu,s,x) = propto((1/sqrt(2*pi*s)) * exp(-1/(2*s) * (x-mu)^2),x);
Exponential(L,x) = exp(-L*x);
Gamma(a,b,x) = x^(a-1)*exp(-b*x);
Beta(a,b,x) = x^(a-1)*(1-x)^(b-1);
ChiSq(v,x) = x^(v/2 - 1) * exp(-x/2);
tdist(v,x) = (1+x^2 / v)^(-(v+1)/2);
Cauchy(g,x0,x) = 1/(1+((x-x0)/g)^2);
logf = logf(x);
best_sol = {'none', inf};
r1 = randi(10); r2 = randi(10); r3 = randi(10);
while (r1 == r2 || r2 == r3 || r1 == r3) r1 = randi(10); r2 = randi(10); r3 = randi(10); end
%% check Exponential:
if (propto(logf,x) == x) % pdf ~ exp(K*x), can read off Lambda directly
soln = -logf/x;
if (~has(soln,x)) % any solution can't depend on x
fprintf('\nExponential: rate L = %s\n\n', soln);
return
end
end
%% check Chi-sq:
if (propto(logf + x/2, log(x)) == log(x)) % can read off v directly
soln = 2*(1+(logf + x/2) / log(x));
if (~has(soln,x))
dof = feval(symengine,'simplify',soln,'IgnoreAnalyticConstraints');
fprintf('\nChi-Squared: v = %s\n\n', dof);
return
end
end
%% check t-dist:
h1 = propto(logf,x);
h = simplify(exp(h1) - 1);
if (propto(h,x^2) == x^2) % pdf ~ exp(K*x), can read off Lambda directly
soln = simplify(x^2 / h);
if (~has(soln,x))
fprintf('\nt-dist: v = %s\n\n', soln);
return
end
end
h = simplify(exp(-h1) - 1); % try again if propto flipped a sign
if (propto(h,x^2) == x^2) % pdf ~ exp(K*x), can read off Lambda directly
soln = simplify(x^2 / h);
if (~has(soln,x))
fprintf('\nt-dist: v = %s\n\n', soln);
return
end
end
%% check Normal:
logn(x) = feval(symengine,'simplify',log(Normal(mu,s,x)),'IgnoreAnalyticConstraints');
% A = (x - propto(logf/x, x))/2;
% B = simplify(-x/(logf/x - mu/s)/2);
% if (~has(A,x) && ~has(B,x))
% fprintf('Normal: mu = %s, s^2 = %s', A, B);
% return
% end
logf(x) = logf;
try % attempt to solve the equation
% solve simultaneously for two random non-equal integer values r1,r2
qn = solve(logf(r1) == logn(r1), logf(r2) == logn(r2), mu, s);
catch error
end
if (exist('qn','var')) % if solve() managed to run
if (~isempty(qn.mu) && ~isempty(qn.s) && ~any(has([qn.mu,qn.s],x))) % if solution exists
complexity = symlength(qn.mu) + symlength(qn.s);
if complexity < best_sol{2} % store best solution so far
best_sol{1} = sprintf('Normal: mu = %s, s^2 = %s', qn.mu, qn.s);
best_sol{2} = complexity;
end
end
end
%% check Cauchy:
logcau(x) = feval(symengine,'simplify',log(Cauchy(g,x0,x)),'IgnoreAnalyticConstraints');
f(x) = f;
try
qcau = solve(f(r1) == Cauchy(g,x0,r1), f(r2) == Cauchy(g,x0,r2), g, x0);
catch error
end
if (exist('qcau','var'))
if (~isempty(qcau.g) && ~isempty(qcau.x0) && ~any(has([qcau.g(1),qcau.x0(1)],x)))
complexity = symlength(qcau.g(1)) + symlength(qcau.x0(1));
if complexity < best_sol{2}
best_sol{1} = sprintf('Cauchy: g = %s, x0 = %s', qcau.g(1), qcau.x0(1));
best_sol{2} = complexity;
end
end
end
f = f(x);
%% check Gamma:
logg(x) = feval(symengine,'simplify',log(Gamma(a,b,x)),'IgnoreAnalyticConstraints');
t = children(logf); t = [t{:}];
if (length(t) == 2)
if (propto(t(1),log(x)) == log(x) && propto(t(2),x) == x)
soln = [t(1)/log(x) + 1, -t(2)/x];
if (~any(has(soln,x)))
fprintf('\nGamma: shape a = %s, rate b = %s\n\n',soln);
return
end
elseif (propto(t(2),log(x)) == log(x) && propto(t(1),x) == x)
soln = [t(2)/log(x) + 1, -t(1)/x];
if (~any(has(soln,x)))
fprintf('\nGamma: shape a = %s, rate b = %s\n\n',soln);
return
end
end
end
logf(x) = logf;
try % also try using solve(), just in case.
qg = solve(logf(r1) == logg(r1), logf(r2) == logg(r2), a, b);
catch error
end
if (exist('qg','var'))
if (~isempty(qg.a) && ~isempty(qg.b) && ~any(has([qg.a,qg.b],x)))
complexity = symlength(qg.a) + symlength(qg.b);
if complexity < best_sol{2}
best_sol{1} = sprintf('Gamma: shape a = %s, rate b = %s', qg.a, qg.b);
best_sol{2} = complexity;
end
end
end
logf = logf(x);
%% check Beta:
B = feval(symengine,'simplify',log(propto(f,x-1)),'IgnoreAnalyticConstraints');
if (propto(B,log(x-1)) == log(x-1))
B = B / log(x-1) + 1;
A = f / (x-1)^(B-1);
A = feval(symengine,'simplify',log(abs(A)),'IgnoreAnalyticConstraints');
if (propto(A,log(abs(x))) == log(abs(x)))
A = A / log(abs(x)) + 1;
if (~any(has([A,B],x)))
fprintf('\nBeta1: a = %s, b = %s\n\n', A, B);
return
end
end
elseif (propto(B,log(1-x)) == log(1-x))
B = B / log(1-x);
A = simplify(f / (1-x)^(B-1));
A = feval(symengine,'simplify',log(A),'IgnoreAnalyticConstraints');
if (propto(A,log(x)) == log(x))
A = A / log(x) + 1;
if (~any(has([A,B],x)))
fprintf('\nBeta1: a = %s, b = %s\n\n', A, B);
return
end
end
end
%% Print solution with lowest complexity
fprintf('\n%s\n\n', best_sol{1});
end
Tests:
>> syms x y z
>> f = y^(1/2)*exp(-(x^2)/2 - y/2 * (1+(4-x)^2+(6-x)^2))
>> whichDist(f,x)
Normal: mu = (10*y)/(2*y + 1), s^2 = 1/(2*y + 1)
>> whichDist(f,y)
Gamma: a = 3/2, b = x^2 - 10*x + 53/2
>> Beta(a,b,x) = propto((1/beta(a,b)) * x^(a-1)*(1-x)^(b-1), x);
>> f = Beta(1/z + 7*y/(1-sqrt(z)), z/y + 1/(1-z), x)
Beta: a = -(7*y*z - z^(1/2) + 1)/(z*(z^(1/2) - 1)), b = -(y + z - z^2)/(y*(z - 1))
All correct.
Sometimes bogus answers if the parameters are numeric:
whichDist(Beta(3,4,x),x)
Beta: a = -(pi*log(2)*1i + pi*log(3/10)*1i - log(2)*log(3/10) + log(2)*log(7/10) - log(3/10)*log(32) + log(2)*log(1323/100000))/(log(2)*(log(3/10) - log(7/10))), b = (pi*log(2)*1i + pi*log(7/10)*1i + log(2)*log(3/10) - log(2)*log(7/10) - log(7/10)*log(32) + log(2)*log(1323/100000))/(log(2)*(log(3/10) - log(7/10)))
So there is room for improvement and I will still award bounty to a better solution than this.
Edit: Added more distributions. Improved Gamma and Beta distribution identifications by spotting them directly without needing solve().
So I'm trying to figure out how to solve a given equation y'=-y+2te^(-t+2) for t in [0,10], step of 0.01 and y(0)=0.
I am supposed to solve it using the Lobatto IIIA method following a Butcher tableau:
Coefficients table
So far, this is what I got:
function lob = Lobatto_solver()
h = 0.01;
t = 0:h:10;
y = zeros(size(t));
y(1) = 0;
f = #(t,y) -y + (2*t*(exp(-t+2)));
% Lobatto IIIA Method
for i=1:numel(y)-1
f1 = f(t(i), y(i));
f2 = f(t(i)+(1/2)*h, y(i) + (5/24)*h*f1 + (1/3)*h*f2 + (-1/24)*h*f3);
f3 = f(t(i)+h, y(i) + (1/6)*h*f1 + (2/3)*h*f2 + (1/6)*h*f3);
y(x) = y(i) + h*((-1/2)*f1 + (2)*f2 + (-1/2)*f3);
end
end
It obviously makes no sense from the point when I equal f2 to itself, when the variable is still undefined.
Any help would be much appreciated :)
Cheers
You will need a predictor-corrector loop, in the simple case the corrector uses the slope-equations as basis of a fixed-point iteration. In the code below I use also the value of an explicit Euler step, in principle you could initialize all slopes with f1.
function lob = Lobatto_solver()
h = 0.01;
t = 0:h:10;
y = zeros(size(t));
y(1) = 0;
f = #(t,y) -y + (2*t*(exp(-t+2)));
% Lobatto IIIA Method
for i=1:numel(y)-1
f1 = f(t(i), y(i));
f3 = f(t(i)+h, y(i)+h*f1)
f2 = (f1+f3)/2;
for k=1:3
f2 = f(t(i)+(1/2)*h, y(i) + (5/24)*h*f1 + (1/3)*h*f2 + (-1/24)*h*f3);
f3 = f(t(i)+h, y(i) + (1/6)*h*f1 + (2/3)*h*f2 + (1/6)*h*f3);
end;
y(i+1) = y(i) + h*((-1/2)*f1 + (2)*f2 + (-1/2)*f3);
end
plot(t,y,t,0.05+t.^2.*exp(-t+2))
end
The graph shows that the result (blue) is qualitatively correct, the exact solution curve (green) is shifted so that two distinct curves can be seen.
This is my matlab code , I got Not enough input argument error in line 2 and i don't know how to fix it. Anyhelp ? Thanks in advance .
function [] = Integr1( F,a,b )
i = ((b - a)/500);
x = a;k = 0; n = 0;
while x <= b
F1 = F(x);
x = x + i;
F2 = F(x);
m = ((F1+F2)*i)/2;
k = k +m;
end
k
x = a; e = 0; o = 0;
while x <= (b - 2*i)
x = x + i;
e = e + F(x);
x = x + i;
o = o + F(x);
end
n = (i/3)*(F(a) + F(b) + 2*o + 4*e)
This code performs integration by the trapezoidal rule. The last line of code gave it away. Please do not just push the Play button in your MATLAB editor. Don't even think about it, and ignore that it's there. Instead, go into your Command Prompt, and you need to define the inputs that go into this function. These inputs are:
F: A function you want to integrate:
a: The starting x point
b: The ending x point
BTW, your function will not do anything once you run it. You probably want to return the integral result, and so you need to modify the first line of your code to this:
function n = Integr1( F,a,b )
The last line of code assigns n to be the area under the curve, and that's what you want to return.
Now, let's define your parameters. A simple example for F is a linear function... something like:
F = #(x) 2*x + 3;
This defines a function y = 2*x + 3. Next define the starting and ending points:
a = 1; b = 4;
I made them 1 and 4 respectively. Now you can call the code:
out = Integr1(F, a, b);
out should contain the integral of y = 2*x + 3 from x = 1 to x = 4.
I use function with multiple outputs farina4 that computes coefficients a, b, e, f and a vector out_p5tads_final (1 x n array) through a minimization of a system of equations using the data input set p5tads (1 x n array):
function [a b e f fval out_p5tads_final] = farina4(p5tads)
f = #(coeff)calculs_farina4(coeff,p5tads);
[ans,fval] = fminsearchcon(f,coeff0,[0 0 0 0],[1 1 1 1]);% fminsearch with constrains
a = ans(1);
b = ans(2);
e = ans(3);
f = ans(4);
out_p5tads_final = p5tads_farina4(a,b,e,f);
function out_coeff = calculs_farina4(coeff0,p5tads)
%bla-bla-bla
end
function out_p5tads = p5tads_farina4(a,b,e,f)
%bla-bla-bla
end
end
After calculating a, b, e, f and out_p5tads_final I need to calculate/minimize the RMS function with respect to out_p5tads_f4.
RMS = sqrt(mean((p5tads(:) - out_p5tads_f4(:)).^2))*100
and to repeat function farina4 in order to find the optimal set of the parameters a, b, e, f and out_p5tads_final.
I am trying to build up an algorithm of such optimization and do not see a way so far.
For instance, it seems to be not possible to introduce a function with multiple output inside the above RMS equation unless there is a way to index somehow the output of this function farina4.
If there can be an alternative optimization algorithm for RMS without fminsearch (or similar) ?
a b e and f are values between 0 and 1
out_p5tads_final is an (1 x 10) array
%
function out_coeff = calculs_farina4(coeff0,p5tads)
%
mmmm = p5tads(1);
mmmr = p5tads(2);
rmmr = p5tads(3);
mmrm = p5tads(4);
mmrr = p5tads(5);
rmrm = p5tads(6);
rmrr = p5tads(7);
mrrm = p5tads(8);
mrrr = p5tads(9);
rrrr = p5tads(10);
%
a = coeff0(1);
b = coeff0(2);
e = coeff0(3);
f = coeff0(4);
%
f_mmmm = mmmm - ((a^2*b^2*(a + b) + e^2*f^2*(e + f))/2);
f_mmmr = mmmr - (a^2*b^2*(e + f) + e^2*f^2*(a + b));
f_rmmr = rmmr - ((a^2*f^2*(b + e) + b^2*e^2*(a + f))/2);
f_mmrm = mmrm - 2*a*b*e*f;
f_mmrr = mmrr - b*f*(a^3 + e^3) + a*e*(a^3 + f^3);
f_rmrm = rmrm - 2*a*b*e*f;
f_rmrr = rmrr - 2*a*b*e*f;
f_mrrm = mrrm - ((a^2*b^2*(e + f) + e^2*f^2*(a + b))/2);
f_mrrr = mrrr - (a^2*f^2*(b + e) + b^2*e^2*(a + f));
f_rrrr = rrrr - ((a^2*f^2*(a + f) + b^2*e^2*(b + e))/2);
%
out_coeff = f_mmmm^2 + f_mmmr^2 + f_rmmr^2 + f_mmrm^2 + f_mmrr^2 + f_rmrm^2 + f_rmrr^2 + f_mrrm^2 + f_mrrr^2 + f_rrrr^2;
end
%
function out_p5tads = p5tads_farina4(a,b,e,f)
%
p_mmmm = ((a^2*b^2*(a + b) + e^2*f^2*(e + f))/2);
p_mmmr = (a^2*b^2*(e + f) + e^2*f^2*(a + b));
p_rmmr = ((a^2*f^2*(b + e) + b^2*e^2*(a + f))/2);
p_mmrm = 2*a*b*e*f;
p_mmrr = b*f*(a^3 + e^3) + a*e*(a^3 + f^3);
p_rmrm = 2*a*b*e*f;
p_rmrr = 2*a*b*e*f;
p_mrrm = ((a^2*b^2*(e + f) + e^2*f^2*(a + b))/2);
p_mrrr = (a^2*f^2*(b + e) + b^2*e^2*(a + f));
p_rrrr = ((a^2*f^2*(a + f) + b^2*e^2*(b + e))/2);
%
out_p5tads = [p_mmmm,p_mmmr,p_rmmr,p_mmrm,p_mmrr,p_rmrm,p_rmrr,p_mrrm,p_mrrr,p_rrrr];
end
end
Thanks much in advance !
19/08/2014 3:35 pm
I need to get an optimal set of coefficients a b e f that the RMS value , which is calculated from
RMS = sqrt(mean((p5tads(:) - out_p5tads_f4(:)).^2))*100
is minimal. Here, the vector p5tads is used to calculate/optimize the set of a b e f coefficients, which are in turn used to calculate the vector out_p5tads_f4. The code should run a desired number of optimizations cycles (e.g. by default 100) and then select the series of a b e f and out_p5tads_f4 afforded the minimal RMS error value (with respect to out_p5tads_f4).
I would like to preface this by saying, I know some functions, including RGB2HSI could do this for me, but I would like to do it manually for a deeper understanding.
So my goal here is to change my RGB image to HSI color scheme. The image is in .raw format, and i am using the following formulas on the binary code to try and convert it.
theta = arccos((.5*(R-G) + (R-B))/((R-G).^2 + (R-B).*(G-B)).^.5);
S = 1 - 3./(R + G + B)
I = 1/3 * (R + G + B)
if B <= G H = theta if B > G H = 360 - theta
So far I have tried two different things, that have resulted in two different errors. The first attempted was the following,
for iii = 1:196608
C(iii) = acosd((.5*(R-G) + (R-B))/((R-G).^2 + (R-B).*(G-B)).^.5);
S(iii) = 1 - 3./(R + G + B);
I(iii) = 1/3 * (R + G + B);
end
Now in attempting this I knew it was grossly inefficent, but I wanted to see if it was a viable option. It was not, and the computer ran out of memory and refused to even run it.
My second attempt was this
fid = fopen('color.raw');
R = fread(fid,512*384*3,'uint8', 2);
fseek(fid, 1, 'bof');
G = fread(fid, 512*384*3, 'uint8', 2);
fseek(fid, 2, 'bof');
B = fread(fid, 512*384*3, 'uint8', 2);
fclose(fid);
R = reshape(R, [512 384]);
G = reshape(G, [512 384]);
B = reshape(B, [512 384]);
C = acosd((.5*(R-G) + (R-B))/((R-G).^2 + (R-B).*(G-B)).^.5);
S = 1 - 3./(R + G + B);
I = 1/3 * (R + G + B);
if B <= G
H = B;
if B > G
H = 360 - B;
end
end
H = H/360;
figure(1);
imagesc(H * S * I)
There were several issues with this that I need help with. First of all, the matrix 'C' has different dimensions than S and I so multiplication is impossible, so my first question is, how would I call up each pixel so I could perform the operations on them individually to avoid this dilemma.
Secondly the if loops refused to work, if I put them after "imagesc" nothing would happen, and if i put them before "imagesc" then the computer would not recognize what variable H was. Where is the correct placement of the ends?
Normally, the matrix 'C' have same dimensions as S and I because:
C = acosd((.5*(R-G) + (R-B))/((R-G).^2 + (R-B).*(G-B)).^.5);
should be
C = acosd((.5*(R-G) + (R-B))./((R-G).^2 + (R-B).*(G-B)).^.5);
elementwise division in the middle was missing . Another point is:
if B <= G
H = B;
if B > G
H = 360 - B;
end
end
should be
H = zeros(size(B));
H(find(B <= G)) = B(find(B <= G));
H(find(B > G)) = 360 - B(find(B > G));