Related
Here is the problem and current code I am using.
Use the svd() function in MATLAB to compute , the rank-2 approximation of . Clearly state what is, rounded to 4 decimal places. Also, compute the root-mean square error (RMSE) between and . Which approximation is better, or ? Explain.
Solution:
%code
[U , S, V] = svd(A)
k = 2
V = V(:,1:k)
V = transpose(V)
Ak = U(:,1:k) .* S(1:k,1:k) .* V
diffA = A - A2
fro_norm = norm(diffA,'fro')
RMSE2 = (fro_norm)/sqrt(m*n)
However, when running, the line AK = . . . keeps giving an error because the matrix sizes are not compatible. So I understand that the matrix sizes need to match in order to do the multiplication, but I also know that the problem requires the following calculation requirements, that when k = 2, U has to use the first 2 columns, S has to use the first 2 rows and first 2 columns, and V has to be the transpose of V only using the first two columns.
I must be missing something in my understanding of the calculation, or creation of the sub k matrices. The matrix I have to use is a 3 x 3.
I am trying to use mvregress with the data I have with dimensionality of a couple of hundreds. (3~4). Using 32 gb of ram, I can not compute beta and I get "out of memory" message. I couldn't find any limitation of use for mvregress that prevents me to apply it on vectors with this degree of dimensionality, am I doing something wrong? is there any way to use multivar linear regression via my data?
here is an example of what goes wrong:
dim=400;
nsamp=1000;
dataVariance = .10;
noiseVariance = .05;
mixtureCenters=randn(dim,1);
X=randn(dim, nsamp)*sqrt(dataVariance ) + repmat(mixtureCenters,1,nsamp);
N=randn(dim, nsamp)*sqrt(noiseVariance ) + repmat(mixtureCenters,1,nsamp);
A=2*eye(dim);
Y=A*X+N;
%without residual term:
A_hat=mvregress(X',Y');
%wit residual term:
[B, y_hat]=mlrtrain(X,Y)
where
function [B, y_hat]=mlrtrain(X,Y)
[n,d] = size(Y);
Xmat = [ones(n,1) X];
Xmat_sz=size(Xmat);
Xcell = cell(1,n);
for i = 1:n
Xcell{i} = [kron([Xmat(i,:)],eye(d))];
end
[beta,sigma,E,V] = mvregress(Xcell,Y);
B = reshape(beta,d,Xmat_sz(2))';
y_hat=Xmat * B ;
end
the error is:
Error using bsxfun
Out of memory. Type HELP MEMORY for your options.
Error in kron (line 36)
K = reshape(bsxfun(#times,A,B),[ma*mb na*nb]);
Error in mvregress (line 319)
c{j} = kron(eye(NumSeries),Design(j,:));
and this is result of whos command:
whos
Name Size Bytes Class Attributes
A 400x400 1280000 double
N 400x1000 3200000 double
X 400x1000 3200000 double
Y 400x1000 3200000 double
dataVariance 1x1 8 double
dim 1x1 8 double
mixtureCenters 400x1 3200 double
noiseVariance 1x1 8 double
nsamp 1x1 8 double
Okay, I think I have a solution for you, short version first:
dim=400;
nsamp=1000;
dataVariance = .10;
noiseVariance = .05;
mixtureCenters=randn(dim,1);
X=randn(dim, nsamp)*sqrt(dataVariance ) + repmat(mixtureCenters,1,nsamp);
N=randn(dim, nsamp)*sqrt(noiseVariance ) + repmat(mixtureCenters,1,nsamp);
A=2*eye(dim);
Y=A*X+N;
[n,d] = size(Y);
Xmat = [ones(n,1) X];
Xmat_sz=size(Xmat);
Xcell = cell(1,n);
for i = 1:n
Xcell{i} = kron(Xmat(i,:),speye(d));
end
[beta,sigma,E,V] = mvregress(Xcell,Y);
B = reshape(beta,d,Xmat_sz(2))';
y_hat=Xmat * B ;
Strangely, I could not access the function's workspace, it did not appear in the call stack. This is why I put the function after the script here.
Here's the explanation that might also help you in the future:
Looking at the kron definition, the result when inserting an m by n and a p by q matrix has size mxp by nxq, in your case 400 by 1001 and 1000 by 1000, that makes a 400000 by 1001000 matrix, which has 4*10^11 elements. Now you have four hundred of them, and each element takes up 8 bytes for double precision, that is a total size of about 1.281 Petabytes of memory (or 1.138 Pebibytes, if you prefer), well out of reach even with your grand 32 Gibibyte.
Seeing that one of your matrices, the eye one, contains mostly zeros, and the resulting matrix contains all possible element product combinations, most of them will be zero, too. For such cases specifically, MATLAB offers the sparse matrix format, which saves a lot of memory depending on the number of zero elements in a matrix by only storing nonzero ones. You can convert a full matrix to a sparse representation with sparse(X), or you get an eye matrix directly by using speye(n), which is what I did above. The sparse property propagates to the result, which you should now have enough memory for (I have with 1/4 of your memory available, and it works).
However, what remains is the problem Matthew Gunn mentioned in a comment. I get an error saying:
Error using mvregress (line 260)
Insufficient data to estimate either full or least-squares models.
Preface
If your regressors are all the same across each regression equation and you're interested in the OLS estimate, you can replace a call to mvregress with a simple call to \.
It appears in the call to mlrtrain you had a matrix transposition error (since corrected). In the language of mvregress, n is the number of observations, d is the number of outcome variables. You generate a matrix Y that is d by n. But THEN when you should call mlrtrain(X', Y') not mlrtrain(X, Y).
If below isn't specifically, what you're looking for, I suggest you precisely define what you're trying to estimate.
What I would have written if I were you
So much that's been said here is completely off base that I'm posting code of what I would have written if I were you. I've reduced the dimensionality to show the equivalence in your special case to simply calling \. I've also written stuff in a more standard way (i.e. having observations run down the rows and not making matrix transposition errors).
dim=5; % These can go way higher but only if you use my code
nsamp=20; % rather than call mvregress
dataVariance = .10;
noiseVariance = .05;
mixtureCenters=randn(dim,1);
X = randn(nsamp, dim)*sqrt(dataVariance ) + repmat(mixtureCenters', nsamp, 1); %'
E = randn(nsamp, dim)*sqrt(noiseVariance); %noise should be mean zero
B = 2*eye(dim);
Y = X*B+E;
% without constant:
B_hat = mvregress(X,Y); %<-------- slow, blows up with high dimension
B_hat2 = X \ Y; %<-------- fast, fine with higher dimensions
norm(B_hat - B_hat2) % show numerical equivalent if basically 0
% with constant:
B_constant_hat = mlrtrain(X,Y) %<-------- slow, blows up with high dimension
B_constant_hat2 = [ones(nsamp, 1), X] \ Y; % <-- fast, and fine with higher dimensions
norm(B_constant_hat - B_constant_hat2) % show numerical equivalent if basically 0
Explanation
I'll assume you have:
An nsamp by dim sized data matrix X.
An nsamp by ny sized matrix of outcome variables Y
You want the results from regressing each column of Y on data matrix X. That is, we're doing multivariate regression but there's a common data matrix X.
That is, we're estimating:
y_{ij} = \sum_k b_k * x_{ik} + e_{ijk} for i=1...nsamp, j = 1...ny, k=1...dim
If you're trying to do something different than this, you need to clearly state what you're trying to do!
To regress Y on X you could do:
[beta_mvr, sigma_mvr, resid_mvr] = mvregress(X, Y);
This appears to be horribly slow. The following should match mvregress for the case where you're using the same data matrix for each regression.
beta_hat = X \ Y; % estimate beta using least squares
resid = Y - X * beta_hat; % calculate residual
If you want to construct a new data matrix with a vector of ones, you would do:
X_withones = [ones(nsamp, 1), X];
Further clarification for some that are confused
Let's say we want to run the regression
y_i = \sum_j x_{ij} + e_i i=1...n, j=1...k
We can construct the data matrix n by k datamatrix X and an n by 1 outcome vector y. The OLS estimate is bhat = pinv(X' * X) * X' * y which can also be computed in MATLAB with bhat = X \ y.
If you want to do this multiple times (i.e. run multivariate regression on the same data matrix X), you can construct an outcome matrix Y where EACH column represents a separate outcome variable. Y = [ya, yb, yc, ...]. Trivially, the OLS solution is B = pinv(X'*X)*X'*Y which can be computed as B = X \ Y. The first column of B is the result of regressing Y(:,1) on X. The second column of B is the result of regressing Y(:,2) on X, etc... Under these conditions, this is equivalent to a call to B = mvregress(X, Y)
Even more test code
If regressors are the same and estimation is by simple OLS, there is an equivalence between multivariate regression and equation by equation ordinary least squares.
d = 10;
k = 15;
n = 100;
C = RandomCorr(d + k, 1); %Use any method you like to generate a random correlation matrix
s = randn(d+k , 1) * 10;
S = (s * s') .* C; % generate covariance matrix
mu = randn(d+k,1);
data = mvnrnd(ones(n, 1) * mu', S);
Y = data(:,1:d);
X = data(:,d+1:end);
[b1, sigma] = mvregress(X, Y);
b2 = X \ Y;
norm(b1 - b2)
You will notice b1 and b2 are numerically equivalent. They are equivalent even though sigma is EXTREMELY different from zero.
I am trying out one of the matlab programming question.
Question:
Write a function called hulk that takes a row vector v as an input and
returns a matrix H whose first column consist of the elements of v,
whose second column consists of the squares of the elements of v, and
whose third column consists of the cubes of the elements v. For
example, if you call the function likes this, A = hulk(1:3) , then A
will be [ 1 1 1; 2 4 8; 3 9 27 ].
My Code:
function H = hulk(v)
H = [v; v.^2; v.^3];
size(H) = (n,3);
end
When I test my code using A = hulk(1:3), it throws an error on console.
Your function made an error for argument(s) 0
Am I doing something incorrect? Have I missed anything?
Remove the line size(H) = (n,3);
and add the line H = H';
Final code should be as follows
function H = hulk(v)
H = [v; v.^2; v.^3];
H = H';
end
Your code giving error in matlab editor on the size(H) = (n,3); line
That's why you should use the matlabeditor itself
For your future reference, you can very easily generalise this function in Matlab to allow the user to specify the number of cols in your output matrix. I also recommend that you make this function a bit more defensive by ensuring that you are working with column vectors even if your user submits a row vector.
function H = hulk(v, n)
%//Set default value for n to be 3 so it performs like your current function does when called with the same signature (i.e. only 1 argument)
if nargin < 2 %// nargin stands for "Number of ARGuments IN"
n = 3;
end if
%// Next force v to be a row vector using this trick (:)
%// Lastly use the very useful bsxfun function to perform the power calcs
H = bsxfun(#power, v(:), 1:n);
end
You could reduce the number of operations using cumprod. That way, each v.^k is computed as the previous v.^k times v:
function H = hulk(v, n)
H = cumprod(repmat(v,n,1),1);
The first input argument is the vector, and the second is the maximum exponent.
I have a custom function to calculate the weight between two pixels (that represent nodes on a graph) of an image
function [weight] = getWeight(a,b,img, r, L)
ac = num2cell(a);
bc = num2cell(b);
imgint1 = img(sub2ind(size(img),ac{:}));
imgint2 = img(sub2ind(size(img),bc{:}));
weight = (sum((a - b) .^ 2) + (r^2/L) * abs(imgint2 - imgint1)) / (2*r^2);
where a = [x1 y1] and b = [x2 y2] are coordinates that represents pixels of the image, img is a gray-scale image and r and L are constants. Within the function imgint1 and imgint2 are gray intensities of the pixels on a and b.
I need to calculate the weight among set of points of the image.
Instead of two nested loops, I want to use the pdist function because it is WAY FASTER!
For instance, let nodes a set of pixel coordinates
nodes =
1 1
1 2
2 1
2 2
And img = [ 128 254; 0 255], r = 3, L = 255
In order to get these weights, I am using an intermediate function.
function [weight] = fxIntermediate(a,b, img, r, L)
weight = bsxfun(#(a,b) getWeight(a,b,img,r,L), a, b);
In order to finally get the whole set of weights
distNodes = pdist(nodes,#(XI,XJ) fxIntermediate(XI,XJ,img,r,L));
But it always get me an error
Error using pdist (line 373)
Error evaluating distance function '#(XI,XJ)fxIntermediate(XI,XJ,img,r,L)'.
Error in obtenerMatriz (line 27)
distNodes = pdist(nodes,#(XI,XJ) fxIntermediate(XI,XJ,img,r,L));
Caused by:
Error using bsxfun
Invalid output dimensions.
EDIT 1
This is a short example of my code that it should work, but I got the error mentioned above. If you copy/paste the code on MATLAB and run the code you will see the error
function [adjacencyMatrix] = problem
img = [123, 229; 0, 45]; % 2x2 Image as example
nodes = [1 1; 1 2; 2 2]; % I want to calculate distance function getWeight()
% between pixels img(1,1), img(1,2), img(2,2)
r = 3; % r is a constant, doesn't matter its meaning
L = 255; % L is a constant, doesn't matter its meaning
distNodes = pdist(nodes,#(XI,XJ) fxIntermediate(XI,XJ,img,r,L));
adjacencyMatrix = squareform(distNodes );
end
function [weight] = fxIntermediate(a,b, img, r, L)
weight = bsxfun(#(a,b) getWeight(a,b,img,r,L), a, b);
end
function [weight] = getWeight(a,b,img, r, L)
ac = num2cell(a);
bc = num2cell(b);
imgint1 = img(sub2ind(size(img),ac{:}));
imgint2 = img(sub2ind(size(img),bc{:}));
weight = (sum((a - b) .^ 2) + (r^2/L) * abs(imgint2 - imgint1)) / (2*r^2);
end
My goal is to obtain an adjacency matrix that represents the distance between pixels. For the above example, the desired adjacency matrix is:
adjacencyMatrix =
0 0.2634 0.2641
0.2634 0 0.4163
0.2641 0.4163 0
The problem is that you are neither fulfilling the expectations for a function to be used with pdist, nor those for a function to be used with bsxfun.
– From the documentation of pdist:
A distance function must be of form
d2 = distfun(XI,XJ)
taking as arguments a 1-by-n vector XI, corresponding to a single row
of X, and an m2-by-n matrix XJ, corresponding to multiple rows of X.
distfun must accept a matrix XJ with an arbitrary number of rows.
distfun must return an m2-by-1 vector of distances d2, whose kth
element is the distance between XI and XJ(k,:).
However, by using bsxfun in fxIntermediate, this function always returns a matrix of values whose size is the larger of the sizes of the two inputs.
– From the documentation of bsxfun:
A binary element-wise function of the form C
= fun(A,B) accepts arrays A and B of arbitrary but equal size and returns output of the same size. Each element in the output array C is
the result of an operation on the corresponding elements of A and B
only. fun must also support scalar expansion, such that if A or B is a
scalar, C is the result of applying the scalar to every element in the
other input array.
However, your getWeight appears to always return a scalar.
I do not understand your problem well enough in order to repair this. Moreover, I think if speed is what you are after, feeding pdist with a function handle is not the way to go. pdist does not perform magic; it is only fast because its built-in distance functions are implemented efficiently. Also, you are using anonymous function handles and conversions to and from cell arrays, all of which slow the process down. I think you should post a new question where you start with a description of what you are trying to compute, include some code that does the job even if inefficiently, and ask how to improve that.
Just a simple question today. If I have an m*n matrix and I want to cycle through every value in it and apply a probability based function.
Basically, if the probability is p, then each value in the matrix has p chance of having the function applied to it.
I have the loop and the function itself all worked out, but I haven't found how to actually apply the probability itself.
Any advice would be greatly appreciated! Thanks in advance.
Here's your data matrix:
>> X = reshape(1:9, 3, 3);
and you want to (possibly) apply the following function to every element (note how I've vectorized it, so that it can take a matrix as an argument)
>> f = #(x) x.^2;
You want to apply the function with probability p
>> p = 0.25;
So generate some random numbers between 0 and 1, and see which ones are less than p
>> idx = rand(3,3) < p;
And now apply the function to the relevant indexes
>> X(idx) = f(X(idx));
Here's your result:
>> X
X =
1 16 7
2 5 64
3 6 81
The trick is that you can generate the random numbers first, and then apply the other formulas.
For example:
R = rand(m,n) < p
Now each value of R(row,col) corresponds to the outcome that you need to process your original matrix(row,col).
So I suggest applying your function to every cell and then setting the values to a default value based on some probability. So lets assume M is the result of applying to function to everycell:
default = NaN % Or 0 or whatever
p = 0.8;
M(rand(size(M)) > p) = default;
I think you might have to reshape m after this... not sure
M = reshape(M, m, n);