I am trying to understand how MATLAB's imfilter function works.
im = imread("cameraman.tif");
% Kernel for sharpening the image
kernel = [
0 -1 0;
-1 5 -1;
0 -1 0];
im2 = zeros(size(im));
for y = 1 : size(im,1) - 3
for x = 1 : size(im,2) - 3
sum = 0;
for ky = 1:3
for kx = 1:3
xx = x + kx - 1;
yy = y + ky - 1;
sum = sum + im(yy,xx)*kernel(ky,kx);
end
end
im2(y,x) = sum;
end
end
% Map im2 to 0 - 255
im2 = im2 - min(im2(:));
im2 = im2 / max(im2(:)) * 255;
im2 = uint8(im2);
subplot(131), imshow(im), title('Original Image')
subplot(132), imshow(imfilter(im,kernel)), title('Matlab imfilter')
subplot(133), imshow(im2), title('My filter')
Differences at the boundaries are not my concern but my result (the subplot at the right) is clearly different than what MATLAB produces (the middle subplot) although the same kernel was used.
May I know what could be the deviation? As far as I know, kernel patch will be applied to the image element-wise and be summed over the result. Could someone let me know what I am missing? Thanks.
Your bug is in this line:
sum = sum + im(yy,xx)*kernel(ky,kx);
This is not obvious, but MATLAB makes a strange choice when doing arithmetic with different data types. In MATLAB, by default all numeric arrays (all values) are doubles. sum is double* when you initialize it (sum = 0), as are kernel and im2. But im is an 8-bit unsigned integer array. So im(yy,xx)*kernel(ky,kx) does a multiplication of a uint8 with a double. This is the strange case of combining different data types in an operation.
When doing arithmetic with an integer array, the other operand must be of the same type, unless it is a double scalar (a 1x1 double array). In this case, the scalar value is converted to the integer type, then the operation is applied. Also, integer arithmetic is saturated, meaning that any result outside of the integer range is clamped to the range (there is no overflow as in other languages).
So im(yy,xx)*kernel(ky,kx) results in a uint8. Next, sum + <uint8 result> is also a uint8 value, which is assigned to sum. Now sum is uint8!
To fix this, do
sum = sum + double(im(yy,xx)) * kernel(ky,kx);
* Also note that everything is an array. 0 is a 1x1 array.
Unsolicited advice:
You should not scale the im2 image, you should directly cast it to uint8. MATLAB will clamp the values to the [0,255] range for you. Scaling causes loss of contrast.
Don't use sum as a variable name. sum is a built-in function, which you shadow (make unavailable) with this variable. After running your code, you can no longer do sum(im(:)), for example.
The inner two loops are very easily vectorized:
tmp = double(im(x + (0:2), y + (0:2))) .* kernel;
im2(y,x) = sum(tmp(:));
or, in very recent versions of MATLAB,
im2(y,x) = sum(double(im(x + (0:2), y + (0:2))) .* kernel, 'all');
(And note the need for the sum function here!)
Related
I am trying to use mvregress with the data I have with dimensionality of a couple of hundreds. (3~4). Using 32 gb of ram, I can not compute beta and I get "out of memory" message. I couldn't find any limitation of use for mvregress that prevents me to apply it on vectors with this degree of dimensionality, am I doing something wrong? is there any way to use multivar linear regression via my data?
here is an example of what goes wrong:
dim=400;
nsamp=1000;
dataVariance = .10;
noiseVariance = .05;
mixtureCenters=randn(dim,1);
X=randn(dim, nsamp)*sqrt(dataVariance ) + repmat(mixtureCenters,1,nsamp);
N=randn(dim, nsamp)*sqrt(noiseVariance ) + repmat(mixtureCenters,1,nsamp);
A=2*eye(dim);
Y=A*X+N;
%without residual term:
A_hat=mvregress(X',Y');
%wit residual term:
[B, y_hat]=mlrtrain(X,Y)
where
function [B, y_hat]=mlrtrain(X,Y)
[n,d] = size(Y);
Xmat = [ones(n,1) X];
Xmat_sz=size(Xmat);
Xcell = cell(1,n);
for i = 1:n
Xcell{i} = [kron([Xmat(i,:)],eye(d))];
end
[beta,sigma,E,V] = mvregress(Xcell,Y);
B = reshape(beta,d,Xmat_sz(2))';
y_hat=Xmat * B ;
end
the error is:
Error using bsxfun
Out of memory. Type HELP MEMORY for your options.
Error in kron (line 36)
K = reshape(bsxfun(#times,A,B),[ma*mb na*nb]);
Error in mvregress (line 319)
c{j} = kron(eye(NumSeries),Design(j,:));
and this is result of whos command:
whos
Name Size Bytes Class Attributes
A 400x400 1280000 double
N 400x1000 3200000 double
X 400x1000 3200000 double
Y 400x1000 3200000 double
dataVariance 1x1 8 double
dim 1x1 8 double
mixtureCenters 400x1 3200 double
noiseVariance 1x1 8 double
nsamp 1x1 8 double
Okay, I think I have a solution for you, short version first:
dim=400;
nsamp=1000;
dataVariance = .10;
noiseVariance = .05;
mixtureCenters=randn(dim,1);
X=randn(dim, nsamp)*sqrt(dataVariance ) + repmat(mixtureCenters,1,nsamp);
N=randn(dim, nsamp)*sqrt(noiseVariance ) + repmat(mixtureCenters,1,nsamp);
A=2*eye(dim);
Y=A*X+N;
[n,d] = size(Y);
Xmat = [ones(n,1) X];
Xmat_sz=size(Xmat);
Xcell = cell(1,n);
for i = 1:n
Xcell{i} = kron(Xmat(i,:),speye(d));
end
[beta,sigma,E,V] = mvregress(Xcell,Y);
B = reshape(beta,d,Xmat_sz(2))';
y_hat=Xmat * B ;
Strangely, I could not access the function's workspace, it did not appear in the call stack. This is why I put the function after the script here.
Here's the explanation that might also help you in the future:
Looking at the kron definition, the result when inserting an m by n and a p by q matrix has size mxp by nxq, in your case 400 by 1001 and 1000 by 1000, that makes a 400000 by 1001000 matrix, which has 4*10^11 elements. Now you have four hundred of them, and each element takes up 8 bytes for double precision, that is a total size of about 1.281 Petabytes of memory (or 1.138 Pebibytes, if you prefer), well out of reach even with your grand 32 Gibibyte.
Seeing that one of your matrices, the eye one, contains mostly zeros, and the resulting matrix contains all possible element product combinations, most of them will be zero, too. For such cases specifically, MATLAB offers the sparse matrix format, which saves a lot of memory depending on the number of zero elements in a matrix by only storing nonzero ones. You can convert a full matrix to a sparse representation with sparse(X), or you get an eye matrix directly by using speye(n), which is what I did above. The sparse property propagates to the result, which you should now have enough memory for (I have with 1/4 of your memory available, and it works).
However, what remains is the problem Matthew Gunn mentioned in a comment. I get an error saying:
Error using mvregress (line 260)
Insufficient data to estimate either full or least-squares models.
Preface
If your regressors are all the same across each regression equation and you're interested in the OLS estimate, you can replace a call to mvregress with a simple call to \.
It appears in the call to mlrtrain you had a matrix transposition error (since corrected). In the language of mvregress, n is the number of observations, d is the number of outcome variables. You generate a matrix Y that is d by n. But THEN when you should call mlrtrain(X', Y') not mlrtrain(X, Y).
If below isn't specifically, what you're looking for, I suggest you precisely define what you're trying to estimate.
What I would have written if I were you
So much that's been said here is completely off base that I'm posting code of what I would have written if I were you. I've reduced the dimensionality to show the equivalence in your special case to simply calling \. I've also written stuff in a more standard way (i.e. having observations run down the rows and not making matrix transposition errors).
dim=5; % These can go way higher but only if you use my code
nsamp=20; % rather than call mvregress
dataVariance = .10;
noiseVariance = .05;
mixtureCenters=randn(dim,1);
X = randn(nsamp, dim)*sqrt(dataVariance ) + repmat(mixtureCenters', nsamp, 1); %'
E = randn(nsamp, dim)*sqrt(noiseVariance); %noise should be mean zero
B = 2*eye(dim);
Y = X*B+E;
% without constant:
B_hat = mvregress(X,Y); %<-------- slow, blows up with high dimension
B_hat2 = X \ Y; %<-------- fast, fine with higher dimensions
norm(B_hat - B_hat2) % show numerical equivalent if basically 0
% with constant:
B_constant_hat = mlrtrain(X,Y) %<-------- slow, blows up with high dimension
B_constant_hat2 = [ones(nsamp, 1), X] \ Y; % <-- fast, and fine with higher dimensions
norm(B_constant_hat - B_constant_hat2) % show numerical equivalent if basically 0
Explanation
I'll assume you have:
An nsamp by dim sized data matrix X.
An nsamp by ny sized matrix of outcome variables Y
You want the results from regressing each column of Y on data matrix X. That is, we're doing multivariate regression but there's a common data matrix X.
That is, we're estimating:
y_{ij} = \sum_k b_k * x_{ik} + e_{ijk} for i=1...nsamp, j = 1...ny, k=1...dim
If you're trying to do something different than this, you need to clearly state what you're trying to do!
To regress Y on X you could do:
[beta_mvr, sigma_mvr, resid_mvr] = mvregress(X, Y);
This appears to be horribly slow. The following should match mvregress for the case where you're using the same data matrix for each regression.
beta_hat = X \ Y; % estimate beta using least squares
resid = Y - X * beta_hat; % calculate residual
If you want to construct a new data matrix with a vector of ones, you would do:
X_withones = [ones(nsamp, 1), X];
Further clarification for some that are confused
Let's say we want to run the regression
y_i = \sum_j x_{ij} + e_i i=1...n, j=1...k
We can construct the data matrix n by k datamatrix X and an n by 1 outcome vector y. The OLS estimate is bhat = pinv(X' * X) * X' * y which can also be computed in MATLAB with bhat = X \ y.
If you want to do this multiple times (i.e. run multivariate regression on the same data matrix X), you can construct an outcome matrix Y where EACH column represents a separate outcome variable. Y = [ya, yb, yc, ...]. Trivially, the OLS solution is B = pinv(X'*X)*X'*Y which can be computed as B = X \ Y. The first column of B is the result of regressing Y(:,1) on X. The second column of B is the result of regressing Y(:,2) on X, etc... Under these conditions, this is equivalent to a call to B = mvregress(X, Y)
Even more test code
If regressors are the same and estimation is by simple OLS, there is an equivalence between multivariate regression and equation by equation ordinary least squares.
d = 10;
k = 15;
n = 100;
C = RandomCorr(d + k, 1); %Use any method you like to generate a random correlation matrix
s = randn(d+k , 1) * 10;
S = (s * s') .* C; % generate covariance matrix
mu = randn(d+k,1);
data = mvnrnd(ones(n, 1) * mu', S);
Y = data(:,1:d);
X = data(:,d+1:end);
[b1, sigma] = mvregress(X, Y);
b2 = X \ Y;
norm(b1 - b2)
You will notice b1 and b2 are numerically equivalent. They are equivalent even though sigma is EXTREMELY different from zero.
I have a custom function to calculate the weight between two pixels (that represent nodes on a graph) of an image
function [weight] = getWeight(a,b,img, r, L)
ac = num2cell(a);
bc = num2cell(b);
imgint1 = img(sub2ind(size(img),ac{:}));
imgint2 = img(sub2ind(size(img),bc{:}));
weight = (sum((a - b) .^ 2) + (r^2/L) * abs(imgint2 - imgint1)) / (2*r^2);
where a = [x1 y1] and b = [x2 y2] are coordinates that represents pixels of the image, img is a gray-scale image and r and L are constants. Within the function imgint1 and imgint2 are gray intensities of the pixels on a and b.
I need to calculate the weight among set of points of the image.
Instead of two nested loops, I want to use the pdist function because it is WAY FASTER!
For instance, let nodes a set of pixel coordinates
nodes =
1 1
1 2
2 1
2 2
And img = [ 128 254; 0 255], r = 3, L = 255
In order to get these weights, I am using an intermediate function.
function [weight] = fxIntermediate(a,b, img, r, L)
weight = bsxfun(#(a,b) getWeight(a,b,img,r,L), a, b);
In order to finally get the whole set of weights
distNodes = pdist(nodes,#(XI,XJ) fxIntermediate(XI,XJ,img,r,L));
But it always get me an error
Error using pdist (line 373)
Error evaluating distance function '#(XI,XJ)fxIntermediate(XI,XJ,img,r,L)'.
Error in obtenerMatriz (line 27)
distNodes = pdist(nodes,#(XI,XJ) fxIntermediate(XI,XJ,img,r,L));
Caused by:
Error using bsxfun
Invalid output dimensions.
EDIT 1
This is a short example of my code that it should work, but I got the error mentioned above. If you copy/paste the code on MATLAB and run the code you will see the error
function [adjacencyMatrix] = problem
img = [123, 229; 0, 45]; % 2x2 Image as example
nodes = [1 1; 1 2; 2 2]; % I want to calculate distance function getWeight()
% between pixels img(1,1), img(1,2), img(2,2)
r = 3; % r is a constant, doesn't matter its meaning
L = 255; % L is a constant, doesn't matter its meaning
distNodes = pdist(nodes,#(XI,XJ) fxIntermediate(XI,XJ,img,r,L));
adjacencyMatrix = squareform(distNodes );
end
function [weight] = fxIntermediate(a,b, img, r, L)
weight = bsxfun(#(a,b) getWeight(a,b,img,r,L), a, b);
end
function [weight] = getWeight(a,b,img, r, L)
ac = num2cell(a);
bc = num2cell(b);
imgint1 = img(sub2ind(size(img),ac{:}));
imgint2 = img(sub2ind(size(img),bc{:}));
weight = (sum((a - b) .^ 2) + (r^2/L) * abs(imgint2 - imgint1)) / (2*r^2);
end
My goal is to obtain an adjacency matrix that represents the distance between pixels. For the above example, the desired adjacency matrix is:
adjacencyMatrix =
0 0.2634 0.2641
0.2634 0 0.4163
0.2641 0.4163 0
The problem is that you are neither fulfilling the expectations for a function to be used with pdist, nor those for a function to be used with bsxfun.
– From the documentation of pdist:
A distance function must be of form
d2 = distfun(XI,XJ)
taking as arguments a 1-by-n vector XI, corresponding to a single row
of X, and an m2-by-n matrix XJ, corresponding to multiple rows of X.
distfun must accept a matrix XJ with an arbitrary number of rows.
distfun must return an m2-by-1 vector of distances d2, whose kth
element is the distance between XI and XJ(k,:).
However, by using bsxfun in fxIntermediate, this function always returns a matrix of values whose size is the larger of the sizes of the two inputs.
– From the documentation of bsxfun:
A binary element-wise function of the form C
= fun(A,B) accepts arrays A and B of arbitrary but equal size and returns output of the same size. Each element in the output array C is
the result of an operation on the corresponding elements of A and B
only. fun must also support scalar expansion, such that if A or B is a
scalar, C is the result of applying the scalar to every element in the
other input array.
However, your getWeight appears to always return a scalar.
I do not understand your problem well enough in order to repair this. Moreover, I think if speed is what you are after, feeding pdist with a function handle is not the way to go. pdist does not perform magic; it is only fast because its built-in distance functions are implemented efficiently. Also, you are using anonymous function handles and conversions to and from cell arrays, all of which slow the process down. I think you should post a new question where you start with a description of what you are trying to compute, include some code that does the job even if inefficiently, and ask how to improve that.
Suppose I want to find the size of a matrix, but can't use any functions such as size, numel, and length. Are there any neat ways to do this? I can think of a few versions using loops, such as the one below, but is it possible to do this without loops?
function sz = find_size(m)
sz = [0, 0]
for ii = m' %' or m(1,:) (probably faster)
sz(1) = sz(1) + 1;
end
for ii = m %' or m(:,1)'
sz(2) = sz(2) + 1;
end
end
And for the record: This is not a homework, it's out of curiosity. Although the solutions to this question would never be useful in this context, it is possible that they provide new knowledge in terms of how certain functions/techniques can be used.
Here is a more generic solution
function sz = find_size(m)
sz = [];
m(f(end), f(end));
function r = f(e)
r=[];
sz=[sz e];
end
end
Which
Works for arrays, cell arrays and arrays of objects
Its time complexity is constant and independent of matrix size
Does not use any MATLAB functions
Is easy to adapt to higher dimensions
For non-empty matrices you can use:
sz = [sum(m(:,1)|1) sum(m(1,:)|1)];
But to cover empty matrices we need more function calls
sz = sqrt([sum(sum(m*m'|1)) sum(sum(m'*m|1))]);
or more lines
n=m&0;
n(end+1,end+1)=1;
[I,J]=find(n);
sz=[I,J]-1;
Which both work fine for m=zeros(0,0), m=zeros(0,10) and m=zeros(10,0).
Incremental indexing and a try-catch statement works:
function sz = find_size(m)
sz = [0 0];
isError = false;
while ~isError
try
b = m(sz(1) + 1, :);
sz(1) = sz(1) + 1;
catch
isError = true;
end
end
isError = false;
while ~isError
try
b = m(:, sz(2) + 1);
sz(2) = sz(2) + 1;
catch
isError = true;
end
end
end
A quite general solution is:
[ sum(~sum(m(:,[]),2)) sum(~sum(m([],:),1)) ]
It accepts empty matrices (with 0 columns, 0 rows, or both), as well as complex, NaN or inf values.
It is also very fast: for a 1000 × 1000 matrix it takes about 22 microseconds in my old laptop (a for loop with 1e5 repetitions takes 2.2 seconds, measured with tic, toc).
How this works:
The keys to handling empty matrices in a unified way are:
empty indexing (that is, indexing with []);
the fact that summing along an empty dimension gives zeros.
Let r and c be the (possibly zero) numbers of rows and columns of m. m(:,[]) is an r × 0 empty vector. This holds even if r or c are zero. In addition, this empty indexing automatically provides insensitivity to NaN, inf or complex values in m (and probably accounts for the small computation time as well).
Summing that r × 0 vector along its second dimension (sum(m(:,[]),2)) produces a vector of r × 1 zeros. Negating and summing this vector gives r.
The same procedure is applied for the number of columns, c, by empty-indexing in the first dimension and summing along that dimension.
The find command has a neat option to get the last K elements:
I = find(X,K,'last') returns at most the last K indices corresponding to the nonzero entries of the arrayX`.
To get the size, ask for the last k=1 elements. For example,
>> x=zeros(256,4);
>> [numRows,numCols] = find(x|x==0, 1, 'last')
numRows =
256
numCols =
4
>> numRows0 = size(x,1), numCols0 = size(x,2)
numRows0 =
256
numCols0 =
4
You can use find with the single output argument syntax, which will give you numel:
>> numEl = find(x|x==0, 1, 'last')
numEl =
1024
>> numEl0 = numel(x)
numEl0 =
1024
Another straightforward, but less interesting solution uses whos (thanks for the reminder Navan):
s=whos('x'); s.size
Finally, there is format debug.
Can I use not as a discrete dirac delta function in Matlab?
The definition for the discrete dirac delta function is that
for argument 0 it returns 1, and otherwise it returns 0.
But that is exactly what the not function does in Matlab also!
Do you see any problems if I use not instead of writing my own
dirac delta function? I am aware that Matlab has a dirac
function, but that one is the continuous version - it returns infinity
for 0 instead of 1.
I think it's OK, but note that the output of not is an array of logicals:
Example:
a = [0, 1, pi]
b = not(a)
c = double(b)
whos
Output:
a =
0.00000 1.00000 3.14159
b =
1 0 0
c =
1 0 0
Variables in the current scope:
Attr Name Size Bytes Class
==== ==== ==== ===== =====
a 1x3 24 double
b 1x3 3 logical
c 1x3 24 double
Total is 9 elements using 51 bytes
So if the inputs are doubles, I would define the discrete Dirac delta function this way:
ddirac = #(x) double(not(x));
or
function y = ddelta(x)
y = double(not(x));
For clarity, I would define
diracdelta = #not;
to make it clear in your code that you mean to be using the dirac delta function, rather than doing logical negation. When you revisit your code in six months time, you'll thank me for the extra clarity.
Just as a quick aside, if you try to evaluate this (or any dirac function) for numbers with large amount of significant figures, it will not evaluate properly owing to the (im)precision of floating point numbers.
x = [-2e-6:0.5e-6:2e-6]
y = (x - 0.5e-6)
~y
However, the element of x that appears to be zero will not be true zero.
y(6) = 1.0588e-22
Therefore, I propose a function that will return 1 if x is really, really close to zero; closer than the precision of the floating point number:
epsdirac = #(x) double(abs(x) < eps)
I'm trying to find the max machine number x that satisfies the following equation: x+a=a, where a is a given integer. (I'm not allowed to use eps.)
Here's my code (which is not really working):
function [] = Largest_x()
a=2184;
x=0.0000000001
while (x+a)~=a
x=2*x;
end
fprintf('The biggest value of x in order that x+a=a \n (where a is equal to %g) is : %g \n',a,x);
end
Any help would be much appreciated.
The answer is eps(a)/2.
eps is the difference to the next floating point number, so if you add half or less than that to a float, it won't change. For example:
100+eps(100)/2==100
ans =
1
%# divide by less than two
100+eps(100)/1.9==100
ans =
0
%# what is that number x?
eps(100)/2
ans =
7.1054e-15
If you don't want to rely on eps, you can calculate the number as
2^(-53+floor(log2(a)))
You're small algorithm is certainly not correct. The only conditions where A = X + A are when X is equal to 0. By default matlab data types are doubles with 64 bits.
Lets pretend that matlab were instead using 8 bit integers. The only way to satisfy the equation A = X + A is for X to have the binary representation of [0 0 0 0 0 0 0 0]. So any number between 1 and 0 would work as decimal points are truncated from integers. So again if you were using integers A = A + X would resolve to true if you were to set the value of X to any value between [0,1). However this value is meaningless because X would not take on this value but rather it would take on the value of 0.
It sounds like you are trying to find the resolution of matlab data types. See this: http://www.mathworks.com/help/matlab/matlab_prog/floating-point-numbers.html
The correct answer is that, provided by Jonas: 0.5 * eps(a)
Here is an alternative for the empirical and approximate solution:
>> a = 2184;
>> e = 2 .^ (-100 : 100); % logarithmic scale
>> idx = find(a + e == a, 1, 'last')
idx =
59
>> e(idx)
ans =
2.2737e-013