Knowing that X is a mx3 matrix and theta is a 3x1 matrix, I calculated the cost function of logistic regression as follows:
h = sigmoid(theta'*X');
J = ((-y)*log(h)-(1-y)*log(1-h))/m;
grad(1) = (h'-y)'*X(:,1);
grad(2) = (h'-y)'*X(:,2);
grad(3) = (h'-y)'*X(:,3);
The output is the picture attached:
That's explicitly not the correct result.
When I do
h = sigmoid(X*theta);
J = ((-y)'*log(h)-(1-y)'*log(1-h))/m;
grad = (X'*(h - y))/m;
I get the right result:
For me, these two codes are the same - and yes, I checked the matrices sizes in the first code.
Could somebody help me understand while one is giving one input and the other a different output? Somehow, the first code is giving lots of cost at theta values...
This is because you're not paying attention to the dimensionality of your inputs and outputs. (which in turn is because your code is not properly commented/structured.). Assuming y has the same orientation as X in terms of observations, then:
In the first case you have:
h = sigmoid(theta'*X'); # h is a 1xm horizontal vector
J = ((-y)*log(h)-(1-y)*log(1-h))/m; # J is an mxm matrix
In the second case you have:
h = sigmoid(X*theta); # h is an mx1 vector
J = ((-y)'*log(h)-(1-y)'*log(1-h))/m; # J is a 1xm horizontal vector
This is also the reason you get multiple printouts of that "Cost at test theta" printout. My guess is you're calling "sum" somewhere down the line, to sum over m observations, but because J was an mxm matrix instead of a vector, you ended up with a vector in an fprintf statement, which has the effect of printing that statement as many times as there are elements in your vector. Is m=12 by any chance?
Related
I am working on MATLAB problems from my textbook and one of the problems asks me to use the eig command in MATLAB, compute the matrices V and D such that A = V * D * inv(V). Knowing that the first column of V corresponds to the first eigenvalue D(1,1) and so on, I need to reorder the diagonal entries of D so that the real part is increasing down the diagonal and reorder the columns of V accordingly so that A = V * D * inv(V) still holds. Here's what I have written so far:
r = RandStream('mt19937ar','Seed',1234);
A = r.randn(10,10)+1j*r.randn(10,10);
[V,D] = eig(A);
for tt = 1:9
if (real(D(tt,tt)) > real(D(tt+1,tt+1)))
temp = D(tt,tt);
D(tt,tt) = D(tt+1,tt+1);
D(tt+1,tt+1) = temp;
tempV = V(1,tt);
V(1,tt) = V(1,tt+1);
V(1,tt+1) = tempV;
if (A == V*D*inv(V))
break
end
end
end
When I tested it, the diagonal elements of D did not change from the original order, I know it might be due to the conditionals I set, but I am not sure what specifically is causing it to not do anything. I also think there might be issues in the way I am reordering the diagonal elements and corresponding eigenvectors. Any feedback or suggestions is appreciated, thank you in advance.
Your code has multiple problems:
You need two for-loops for sorting.
You are only swapping first element of eigenvectors, use V(:, tt) for whole column.
V*D*inv(V) will never be exactly equal to A (see this).
To sort eigenvalues by their real parts, try this:
clc;
r = RandStream('mt19937ar','Seed',1234);
n = 10;
A = r.randn(n)+1j*r.randn(n);
[V,D] = eig(A);
d = diag(D, 0); % get eigenvalues in a vector
[~, I] = sort(real(d)); % get index of eigenvalues after sorting
D2 = diag(d(I)); % get sorted eigenvalues as diagonal matrix
V2 = V(:, I); % reorder eigenvectors to match sorted eigenvalues
any(any(abs(A - V2*D2*inv(V2)) > 1e-14)) % test sorted eigenvalues and eigenvectors
The following is a function that takes two equal sized vectors X and Y, and is supposed to return a vector containing single correlation coefficients for image correspondence. The function is supposed to work similarly to the built in corr(X,Y) function in matlab if given two equal sized vectors. Right now my code is producing a vector containing multiple two-number vectors instead of a vector containing single numbers. How do I fix this?
function result = myCorr(X, Y)
meanX = mean(X);
meanY = mean(Y);
stdX = std(X);
stdY = std(Y);
for i = 1:1:length(X),
X(i) = (X(i) - meanX)/stdX;
Y(i) = (Y(i) - meanY)/stdY;
mult = X(i) * Y(i);
end
result = sum(mult)/(length(X)-1);
end
Edit: To clarify I want myCorr(X,Y) above to produce the same output at matlab's corr(X,Y) when given equal sized vectors of image intensity values.
Edit 2: Now the format of the output vector is correct, however the values are off by a lot.
I recommend you use r=corrcoef(X,Y) it will give you a normalized r value you are looking for in a 2x2 matrix and you can just return the r(2,1) entry as your answer. Doing this is equivalent to
r=(X-mean(X))*(Y-mean(Y))'/(sqrt(sum((X-mean(X)).^2))*sqrt(sum((Y-mean(Y)).^2)))
However, if you really want to do what you mentioned in the question you can also do
r=(X)*(Y)'/(sqrt(sum((X-mean(X)).^2))*sqrt(sum((Y-mean(Y)).^2)))
So my question refers to the regress() function in matlab. Click here for the Matlab documentation
If I want to run multiple regressions using this function and output both the coefficients and the confidence intervals, what's the best way to do this in a For loop?
Matlab's own syntax for this is [b,bint] = regress(y,X). But when I try to implement this in a for loop it tells me that the dimension mismatch. My code is the following:
for i=1:6
[a, b]=regress(Dataset(:,i),capm_factors);
capm_coefs(i,:)=a;
capm_ci(i,:)=b;
end
Please help, thanks!
regress outputs a column vector of coefficients that minimize the least squared error between your input data (capm_factors) and your predicted values (Dataset(:,i)). However, in your for loop, you are assuming that a and b are row vectors.
Also, the first output of regress is the solution to your system, but the second output contains a matrix of confidence values where the first column denotes the lower end of the confidence interval for each variable and the second column denotes the upper end of the confidence interval.
Specifically, your input capm_factors should be a M x N matrix where M is the total number of input samples and N is the total number of features. In your code, a would thus give you a N x 1 vector and b would give you a N x 2 matrix.
If you'd like use a loop, make sure capm_coefs is a N x l matrix where l is the total number of times you want to loop and capm_ci should either be a N x 2 x l 3D matrix or perhaps a l element cell array. Either way is acceptable.... but I'll show you how to do both.
Something like this comes to mind:
Confidences as a 3D matrix
l = 6; %// Define # of trials
[M,N] = size(capm_factors); %// Get dimensions of data
capm_coefs = zeros(N, l);
capm_ci = zeros(N, 2, l);
for ii = 1 : l
[a,b] = regress(Dataset(:,i), capm_factors);
capm_coefs(:,ii) = a;
capm_ci(:,:,ii) = b;
end
You'd then access the coefficients for a trial via capm_coefs(:,ii) where ii is the iteration you want. Similarly, the confidence matrix can be accessed via capm_ci(:,:,ii)
Confidences as a cell array
l = 6; %// Define # of trials
[M,N] = size(capm_factors); %// Get dimensions of data
capm_coefs = zeros(N, l);
capm_ci = cell(l); %// Cell array declaration
for ii = 1 : l
[a,b] = regress(Dataset(:,i), capm_factors);
capm_coefs(:,ii) = a;
capm_ci{ii} = b; %// Assign confidences to cell array
end
Like above, you'd access the coefficients for a trial via capm_coefs(:,ii) where ii is the iteration you want. However, the confidence matrix can be accessed via capm_ci{ii} as we are now dealing with cell arrays.
I have a simple function below (I omitted the allocations, etc. for brevity) that I have been tryig to plot against it's x value for specific values of N and T but I keep getting a dimensions error. I think that when I try to plot this I am defining an array for x and then plotting Psum(N', x, T') for certain N' and T' against these x, however MATLAB doesn't seem to like this. Can someone give me some direction please.
function U = Psum(N, X, T)
for m = 1:N
A(1,m) = (1/(m*pi))*sin(m*pi*X)*T*exp(-(m^2)*(pi^2)*T);
% array terms of partial sum
end
M = -sum(A); % evaluate Nth partial sum
U = T*(1-X) + M; % output U(X,T) = T(1-X) + V(X,T)
end
I'm getting a similar error when I try to plot the following, I think there must be something wrong with my general approach
syms x;
f = #(x)((x/(100*pi))*(exp(-(100^2)*(pi^2)*x)));
x = 0:0.1:10000;
plot(x,f(x),'r')
title('PartialSum convergence');
xlabel('T');
ylabel('a_n');
the error I get here reads:
Error using *
Inner matrix dimensions must agree.
Here's the analysis of why you're getting a dimension mismatch error. From this line:
A(1,m) = (1/(m*pi))*sin(m*pi*X)*T*exp(-(m^2)*(pi^2)*T)
The element A(1, m) is supposed to be a scalar value in a two-dimensional matrix. Now let's see what are the dimensions of each of the multiplicands:
(1/(m*pi)) is a scalar (that is, a 1×1 matrix).
sin(m*pi*X) has the same dimensions as X. Let's assume its dimensions are q×n.
exp(-(m^2)*(pi^2)*T) has the same dimensions as T, and is multiplied by T.
Therefore T must be a square matrix, so let's assume its dimensions are p×p.
What we get is a q×n matrix multiplied by a square p×p matrix, and the result must be a scalar (that is, 1×1 matrix). This forces q=1 and n=p.
Now let's look at this line:
U = T*(1-X) + M
We are forced to conclude that p=1, otherwise T cannot be multiplied by X from the right.
This means that your code forces T and X to be scalar! No wonder you're getting a error :)
The remedy is simple: revise the computation in Psum so that it can produce correct results for both a scalar X and a vector X. A possible fix would be adding another loop to iterate over all values of X:
function U = Psum(N, X, T)
U = zeros(size(X));
for k = 1:numel(X) %// Iterate over all values of X
for m = 1:N
A(1,m) = (1/(m*pi))*sin(m*pi*X(k))*T*exp(-(m^2)*(pi^2)*T);
%// array terms of partial sum
end
M = -sum(A); % evaluate Nth partial sum
U(k) = T*(1-X(k)) + M; % output U(X,T) = T(1-X) + V(X,T)
end
end
The output of this function has the same dimensions as X.
By the way, did you verify that Psum produces that correct result for scalar inputs?
I don't fully understand what you are trying to accomplish, but just an observation for you: if your input X is a vector, line 3 can not be computed correctly
A(1,m) = (1/(m*pi))*sin(m*pi*X)*T*exp(-(m^2)*(pi^2)*T);
because the right hand side of the equation give you a vector, but the right hand side A(1,m) is one element, not vector. so you have dimension mismatch.
Hope this helps!
I am working towards comparing multiple images. I have these image data as column vectors of a matrix called "images." I want to assess the similarity of images by first computing their Eucledian distance. I then want to create a matrix over which I can execute multiple random walks. Right now, my code is as follows:
% clear
% clc
% close all
%
% load tea.mat;
images = Input.X;
M = zeros(size(images, 2), size (images, 2));
for i = 1:size(images, 2)
for j = 1:size(images, 2)
normImageTemp = sqrt((sum((images(:, i) - images(:, j))./256).^2));
%Need to accurately select the value of gamma_i
gamma_i = 1/10;
M(i, j) = exp(-gamma_i.*normImageTemp);
end
end
My matrix M however, ends up having a value of 1 along its main diagonal and zeros elsewhere. I'm expecting "large" values for the first few elements of each row and "small" values for elements with column index > 4. Could someone please explain what is wrong? Any advice is appreciated.
Since you're trying to compute a Euclidean distance, it looks like you have an error in where your parentheses are placed when you compute normImageTemp. You have this:
normImageTemp = sqrt((sum((...)./256).^2));
%# ^--- Note that this parenthesis...
But you actually want to do this:
normImageTemp = sqrt(sum(((...)./256).^2));
%# ^--- ...should be here
In other words, you need to perform the element-wise squaring, then the summation, then the square root. What you are doing now is summing elements first, then squaring and taking the square root of the summation, which essentially cancel each other out (or are actually the equivalent of just taking the absolute value).
Incidentally, you can actually use the function NORM to perform this operation for you, like so:
normImageTemp = norm((images(:, i) - images(:, j))./256);
The results you're getting seem reasonable. Recall the behavior of the exp(-x). When x is zero, exp(-x) is 1. When x is large exp(-x) is zero.
Perhaps if you make M(i,j) = normImageTemp; you'd see what you expect to see.
Consider this solution:
I = Input.X;
D = squareform( pdist(I') ); %'# euclidean distance between columns of I
M = exp(-(1/10) * D); %# similarity matrix between columns of I
PDIST and SQUAREFORM are functions from the Statistics Toolbox.
Otherwise consider this equivalent vectorized code (using only built-in functions):
%# we know that: ||u-v||^2 = ||u||^2 + ||v||^2 - 2*u.v
X = sum(I.^2,1);
D = real( sqrt(bsxfun(#plus,X,X')-2*(I'*I)) );
M = exp(-(1/10) * D);
As was explained in the other answers, D is the distance matrix, while exp(-D) is the similarity matrix (which is why you get ones on the diagonal)
there is an already implemented function pdist, if you have a matrix A, you can directly do
Sim= squareform(pdist(A))