Squared covariance function of Gaussian process using matlab? - matlab

This is my first attempt to write the covariance function. I have following values,
x = [-1.50 -1.0 -.75 -.40 -.25 0.00];
sf = 1.27;
ell = 1;
sn = 0.3;
The formula for squared exponential covariance function is
The matlab code for that I have written as :
K = sf^2*exp(-0.5*(squareform(pdist(x)).^2)/ell^2)+(sn)^2*eye(Ntr,Ntr);
where sf is the signal standard deviation, ell the characteristic length scale, sn the noise standard deviation and Ntr the length of the training input data x.
But this giving me no result. Is there is any mistake in my coding ?
And once I calculate, I want to summarize into matrix form as shown below ,
Any help please ?
if x_ = 0.2 then how we can calculate :
a) K_ =[k(x_,x1) k(x_,x2)..........k(x_,xn)] and
b) K__ = k(x_,x_)
Using matlab ?

I obtain the following error message:
Error using +
Matrix dimensions must agree.
Error in untitled3 (line 7)
K = sf^2*exp(-0.5*(squareform(pdist(x)).^2)/ell^2)+(sn)^2*eye(Ntr,Ntr);
indicating that your matrix dimensions do not agree. If you evaluate the parts of your code separately, you will notice that pdist(x) returns a 1x0 vector. In the documentation of pdist the expected format of x is explained:
Rows of X correspond to observations, and columns correspond to
variables
So, instead of calculating pdist(x), you should calculate it for the transpose of x, i.e. pdist(x.'):
K = sf^2*exp(-0.5*(squareform(pdist(x.')).^2)/ell^2)+(sn)^2*eye(Ntr,Ntr);
As a conclusion, always read the error messages and documentation carefully, especially the expected format of your input arguments.
Subquestions
To calculate K for a specific value of x_ (x' in your mentioned formula), you can convert your given formula almost literally to MATLAB:
K_ = sf^2*exp(-0.5*(x-x_).^2/ell^2)+(sn)^2*(x == x_);
To calculate K__, you can use the formula above and setting x = x_, or you can simplify your formula to:
K__ = sf^2+sn^2;

Related

Draw random numbers from a custom probability density function in Matlab

I want to sample R random numbers from a custom probability density function in Matlab.
This is the expression of the probability density function evaluated at x.
I thought about using slicesample
R=10^6;
f = #(x) 1/(2*pi^(1/2))*(1/(x^(3/2)))*exp(-1/(4*x));
epsilon= slicesample(0.3,R,'pdf',f,'thin',1,'burnin',1000);
However, it does not work because I get the error
Error using slicesample (line 175)
The step-out procedure failed.
I tried to change starting value and values of thin and burning parameters but it does not seem to work. Could you advise, either on how to make slicesample work or on alternative solutions to sample random numbers from a custom probability density function in Matlab?
Let X be a random variable distributed according to your target pdf. Applying the change of variable y = 1/x and using the well-known theorem for a function of a random variable, the distribution of Y = 1/X is recognized to be a Gamma distribution with parameters α = 1/2, β = 1/4.
Therefore, it suffices to generate a Gamma random variable (using gamrnd) with those parameters and take the inverse. Note that Matlab's definition of the Gamma distribution uses parameters A = α, B = 1/β.
R = 1e5; % desired sample size
x = 1./gamrnd(1/2, 4, [1 R]); % result
Check:
histogram(x, 'Normalization', 'pdf', 'BinEdges', 0:.1:10)
hold on
f = #(x) 1/2/sqrt(pi)./x.^(3/2).*exp(-1/4./x); % target pdf
fplot(f, 'linewidth', .75)

Plot log(n over k)

I've never used Matlab before and I really don't know how to fix the code. I need to plot log(1000 over k) with k going from 1 to 1000.
y = #(x) log(nchoosek(1000,x));
fplot(y,[1 1000]);
Error:
Warning: Function behaves unexpectedly on array inputs. To improve performance, properly
vectorize your function to return an output with the same size and shape as the input
arguments.
In matlab.graphics.function.FunctionLine>getFunction
In matlab.graphics.function.FunctionLine/updateFunction
In matlab.graphics.function.FunctionLine/set.Function_I
In matlab.graphics.function.FunctionLine/set.Function
In matlab.graphics.function.FunctionLine
In fplot>singleFplot (line 241)
In fplot>#(f)singleFplot(cax,{f},limits,extraOpts,args) (line 196)
In fplot>vectorizeFplot (line 196)
In fplot (line 166)
In P1 (line 5)
There are several problems with the code:
nchoosek does not vectorize on the second input, that is, it does not accept an array as input. fplot works faster for vectorized functions. Otherwise it can be used, but it issues a warning.
The result of nchoosek is close to overflowing for such large values of the first input. For example, nchoosek(1000,500) gives 2.702882409454366e+299, and issues a warning.
nchoosek expects integer inputs. fplot uses in general non-integer values within the specified limits, and so nchoosek issues an error.
You can solve these three issues exploiting the relationship between the factorial and the gamma function and the fact that Matlab has gammaln, which directly computes the logarithm of the gamma function:
n = 1000;
y = #(x) gammaln(n+1)-gammaln(x+1)-gammaln(n-x+1);
fplot(y,[1 1000]);
Note that you get a plot with y values for all x in the specified range, but actually the binomial coefficient is only defined for non-negative integers.
OK, since you've gotten spoilers for your homework exercise anyway now, I'll post an answer that I think is easier to understand.
The multiplicative formula for the binomial coefficient says that
n over k = producti=1 to k( (n+1-i)/i )
(sorry, no way to write proper formulas on SO, see the Wikipedia link if that was not clear).
To compute the logarithm of a product, we can compute the sum of the logarithms:
log(product(xi)) = sum(log(xi))
Thus, we can compute the values of (n+1-i)/i for all i, take the logarithm, and then sum up the first k values to get the result for a given k.
This code accomplishes that using cumsum, the cumulative sum. Its output at array element k is the sum over all input array elements from 1 to k.
n = 1000;
i = 1:1000;
f = (n+1-i)./i;
f = cumsum(log(f));
plot(i,f)
Note also ./, the element-wise division. / performs a matrix division in MATLAB, and is not what you need here.
syms function type reproduces exactly what you want
syms x
y = log(nchoosek(1000,x));
fplot(y,[1 1000]);
This solution uses arrayfun to deal with the fact that nchoosek(n,k) requires k to be a scalar. This approach requires no toolboxes.
Also, this uses plot instead of fplot since this clever answer already addresses how to do with fplot.
% MATLAB R2017a
n = 1000;
fh=#(k) log(nchoosek(n,k));
K = 1:1000;
V = arrayfun(fh,K); % calls fh on each element of K and return all results in vector V
plot(K,V)
Note that for some values of k greater than or equal to 500, you will receive the warning
Warning: Result may not be exact. Coefficient is greater than 9.007199e+15 and is only accurate to 15 digits
because nchoosek(1000,500) = 2.7029e+299. As pointed out by #Luis Mendo, this is due to realmax = 1.7977e+308 which is the largest real floating-point supported. See here for more info.

logistic regression in matlab

i would like to implement logistic regression in matlab, i have following few code for this
function B=logistic_regression(x,y)
f=#(a)(sum(y.*log((exp(a(1)+a(2)*x)/(1+exp(a(1)+a(2)*x))))+(1-y).*log((1-((exp(a(1)+a(2)*x)/(1+exp(a(1)+a(2)*x))))))));
a=[0.1, 0.1];
options = optimset('PlotFcns',#optimplotfval);
B = fminsearch(f,a, options);
end
logistic regression is following :
first we are calculating logit which is equal to
L=b0+b1*x
then we are calculating probability which is equal to
p=e^L/(1+e^L)
and finally we are calculating
y*ln(p)+(1-y)*ln(1-p)
i decided to write all those stuff in one line, but when i am running code , it gives me following error
>> B=logistic_regression(x,y)
Assignment has more non-singleton rhs dimensions than non-singleton subscripts
Error in fminsearch (line 200)
fv(:,1) = funfcn(x,varargin{:});
Error in logistic_regression (line 6)
B = fminsearch(f,a, options);
how can i fix this problem? thanks in advance
In order to implement a logistic regression model, I usually call the glmfit function, which is the simpler way to go. The syntax is:
b = glmfit(x,y,'binomial','link','logit');
b is a vector that contains the coefficients for the linear portion of the logistic regression (the first element is the constant term alpha of the regression). x contains the predictors data, with one row for each observation and one column for each variable. y contains the target variable, usually a vector of boolean (0 or 1) values representing the outcome.
Once you obtain the coefficients, you have to apply the linear part of the regression to your predictors:
z = b(1) + (x * b(2));
To finish, you must apply the logistic function to the output of the linear part:
z = 1 ./ (1 + exp(-z));
If you need more tinkering on your data or on your output, and you require more flexibility and control over your model, I suggest you to look at this implementation:
https://github.com/mohammadaltaleb/Logistic-Regression

How do I write correlation coefficient manually in matlab?

The following is a function that takes two equal sized vectors X and Y, and is supposed to return a vector containing single correlation coefficients for image correspondence. The function is supposed to work similarly to the built in corr(X,Y) function in matlab if given two equal sized vectors. Right now my code is producing a vector containing multiple two-number vectors instead of a vector containing single numbers. How do I fix this?
function result = myCorr(X, Y)
meanX = mean(X);
meanY = mean(Y);
stdX = std(X);
stdY = std(Y);
for i = 1:1:length(X),
X(i) = (X(i) - meanX)/stdX;
Y(i) = (Y(i) - meanY)/stdY;
mult = X(i) * Y(i);
end
result = sum(mult)/(length(X)-1);
end
Edit: To clarify I want myCorr(X,Y) above to produce the same output at matlab's corr(X,Y) when given equal sized vectors of image intensity values.
Edit 2: Now the format of the output vector is correct, however the values are off by a lot.
I recommend you use r=corrcoef(X,Y) it will give you a normalized r value you are looking for in a 2x2 matrix and you can just return the r(2,1) entry as your answer. Doing this is equivalent to
r=(X-mean(X))*(Y-mean(Y))'/(sqrt(sum((X-mean(X)).^2))*sqrt(sum((Y-mean(Y)).^2)))
However, if you really want to do what you mentioned in the question you can also do
r=(X)*(Y)'/(sqrt(sum((X-mean(X)).^2))*sqrt(sum((Y-mean(Y)).^2)))

Matlab Convolution regarding the conv() function and length()/size() function

I'm kind've new to Matlab and stack overflow to begin with, so if I do something wrong outside of the guidelines, please don't hesitate to point it out. Thanks!
I have been trying to do convolution between two functions and I have been having a hard time trying to get it to work.
t=0:.01:10;
h=exp(-t);
x=zeros(size(t)); % When I used length(t), I would get an error that says in conv(), A and B must be vectors.
x(1)=2;
x(4)=5;
y=conv(h,x);
figure; subplot(3,1,1);plot(t,x); % The discrete function would not show (at x=1 and x=4)
subplot(3,1,2);plot(t,h);
subplot(3,1,3);plot(t,y(1:length(t))); %Nothing is plotted here when ran
I commented my issues with the code. I don't understand the difference of length and size in this case and how it would make a difference.
For the second comment, x=1 should have an amplitude of 2. While x=4 should have an amplitude of 5. When plotted, it only shows nothing in the locations specified but looks jumbled up at x=0. I'm assuming that's the reason why the convoluted plot won't be displayed.
The original problem statement is given if it helps to understand what I was thinking throughout.
Consider an input signal x(t) that consists of two delta functions at t = 1 and t = 4 with amplitudes A1 = 5 and A2 = 2, respectively, to a linear system with impulse response h that is an exponential pulse (h(t) = e ^−t ). Plot x(t), h(t) and the output of the linear system y(t) for t in the range of 0 to 10 using increments of 0.01. Use the MATLAB built-in function conv.
The initial question regarding size vs length
length yields a scalar that is equal to the largest dimension of the input. In the case of your array, the size is 1 x N, so length yields N.
size(t)
% 1 1001
length(t)
% 1001
If you pass a scalar (N) to ones, zeros, or a similar function, it will create a square matrix that is N x N. This results in the error that you see when using conv since conv does not accept matrix inputs.
size(ones(length(t)))
% 1001 1001
When you pass a vector to ones or zeros, the output will be that size so since size returns a vector (as shown above), the output is the same size (and a vector) so conv does not have any issues
size(ones(size(t)))
% 1 1001
If you want a vector, you need to explicitly specify the number of rows and columns. Also, in my opinion, it's better to use numel to the number of elements in a vector as it's less ambiguous than length
z = zeros(1, numel(t));
The second question regarding the convolution output:
First of all, the impulses that you create are at the first and fourth index of x and not at the locations where t = 1 and t = 4. Since you create t using a spacing of 0.01, t(1) actually corresponds to t = 0 and t(4) corresponds to t = 0.03
You instead want to use the value of t to specify where to put your impulses
x(t == 1) = 2;
x(t == 4) = 5;
Note that due to floating point errors, you may not have exactly t == 1 and t == 4 so you can use a small epsilon instead
x(abs(t - 1) < eps) = 2;
x(abs(t - 4) < eps) = 5;
Once we make this change, we get the expected scaled and shifted versions of the input function.