I'm currently trying to use convolution to average blocks in my data and turn my current, 336x264x25x27, grid into a new, 100x100x27, grid.
To achieve this I've been trying to use convolution.
In a 2-D setting I've been able to convert a 336x264 matrix to 100x100 using the conv2 function in matlab. I'm now trying to use convn to accomplish a similar task in 4-D.
As stated, currently I'm using the convn function. I'm trying to average out cells over the first two dimensions so that I end up with a 100x100x27 matrix. My code is as follows:
A = rand(336,264,25,27); % Sample Data
A = A(:,:,13,:); % This line and the following line eliminate the third dimension (time) which will be constant throughout my output. Now "A" is 336x264x27 after using "squeeze".
A = squeeze(A);
B = ones(100,100,27); % This is the size of matrix that I would like to achieve. I was under the impression that "B" was the size matrix that you want to inevitably end up with but I believe I am mistaken.
C = convn(A,B); % C would hopefully by my 100x100x27 matrix.
Currently, this is resulting in a 435x363x53 matrix. If you could help me with my logic and show me how I might turn "A" into a 100x100x27 matrix using convolution it would be much appreciated!
Related
I am trying trying to graph the polynomial fit of a 2D dataset in Matlab.
This is what I tried:
rawTable = readtable('Test_data.xlsx','Sheet','Sheet1');
x = rawTable.A;
y = rawTable.B;
figure(1)
scatter(x,y)
c = polyfit(x,y,2);
y_fitted = polyval(c,x);
hold on
plot(x,y_fitted,'r','LineWidth',2)
rawTable.A and rawTable.A are randomly generated numbers. (i.e. the x dataset cannot be represented in the following form : x=0:0.1:100)
The result:
second-order polynomial
But the result I expect looks like this (generated in Excel):
enter image description here
How can I graph the second-order polynomial fit in MATLAB?
I sense some confusion regarding what the output of each of those Matlab function mean. So I'll clarify. And I think we need some details as well. So expect some verbosity. A quick answer, however, is available at the end.
c = polyfit(x,y,2) gives the coefficient vectors of the polynomial fit. You can get the fit information such as error estimate following the documentation.
Name this polynomial as P. P in Matlab is actually the function P=#(x)c(1)*x.^2+c(2)*x+c(3).
Suppose you have a single point X, then polyval(c,X) outputs the value of P(X). And if x is a vector, polyval(c,x) is a vector corresponding to [P(x(1)), P(x(2)),...].
Now that does not represent what the fit is. Just as a quick hack to see something visually, you can try plot(sort(x),polyval(c,sort(x)),'r','LineWidth',2), ie. you can first sort your data and try plotting on those x-values.
However, it is only a hack because a) your data set may be so irregularly spaced that the spline doesn't represent function or b) evaluating on the whole of your data set is unnecessary and inefficient.
The robust and 'standard' way to plot a 2D function of known analytical form in Matlab is as follows:
Define some evenly-spaced x-values over the interval you want to plot the function. For example, x=1:0.1:10. For example, x=linspace(0,1,100).
Evaluate the function on these x-values
Put the above two components into plot(). plot() can either plot the function as sampled points, or connect the points with automatic spline, which is the default.
(For step 1, quadrature is ambiguous but specific enough of a term to describe this process if you wish to communicate with a single word.)
So, instead of using the x in your original data set, you should do something like:
t=linspace(min(x),max(x),100);
plot(t,polyval(c,t),'r','LineWidth',2)
I need to generate some random 2D points (for example 30 points) near the y=x line, insert them in a matrix, plot it and then calculate the SVD of the matrix. But since I'm new to MATLAB I don't know how can I generate my desired matrix.
Since this looks like homework I'll just post some general ideas here.
randi can be used to get semi-random integers. Using that you can create a 2D matrix by duplicating the array and putting them together. Thus: generate a 30x1 column and duplicate it to a 30x2 column. All rows will have the same two entries, i.e. x=y.
Noise can be added to this by creating a 30x2 matrix of random numbers, use rand for that and simply add that to the previously created matrix.
Check the documentation on svd to see how the singular-value decomposition works, it's fairly straight-forward if you know your linear algebra.
Finally for plotting you can use various tools such as image, imagesc, plot, surf and scatter, try them and see which works best for you.
Here is a quick example I made: https://saturnapi.com/fullstack/2d-points-randomly-near-line
%// Welcome to Saturn's MATLAB-Octave API.
%// Delete the sample code below these comments and write your own!'
x = 13 + 6.*rand(20,1);
y = x*0.7 + 0.5*rand(20,1);
[X,Y] = meshgrid(x,y)
figure(1);
plot(x,y,'.');
%// Print plot as PNG with resultion of 60 pixels per inch
print("MyPNG.png", "-dpng", "-r60");
I have a problem in matrix (A) 9*9, I need to extract median for each block 3*3.
Also, I need to know the position of the median in each block.
Note:
I know how can extract the median by using
m=median(median(B));
and B is block in matrix
B = A(i:i+L-1, j:j+L-1);
thank you.
If you have the image processing toolbox installed you can use:
medianBlocks = blockproc(A,[3,3],#(X) ones(size(X.data))*median(X.data(:)))
A == medianBlocks
If you don't have the Image Processing Toolbox, I suggest transforming each neighbourhood so that it fits into a single column then you stack all of these columns together to create a single matrix. Once you have this new matrix, find the median of each column, then use this result for further processing.
The best way to do this would be to use im2col to create this temporary matrix and do the median on this matrix over the columns. The output will give you the median of each neighbourhood in your image.
Ironically, im2col is part of the Image Processing Toolbox. However, Divakar has made a vectorized and more efficient version of it that doesn't require the toolbox and only relies on native MATLAB commands.
See here: Efficient Implementation of `im2col` and `col2im`
As such, depending on whether you want overlapping blocks or sliding blocks, use whichever command suits you. Therefore:
out = im2col_distinct(A, [3 3]); %//OR
%out = im2col_sliding(A, [3 3]);
out_median = median(out);
out_median will contain a vector that has the median of each block in the image. Now, if you want to find which location in each window has the median, you can make a bsxfun call:
loc = bsxfun(#eq, out, out_median);
loc will contain the column-major locations of each block. If you want the row and column locations with respect to each block, you can use ind2sub:
[rows,cols] = ind2sub([3 3], loc);
rows and cols will give you the row and column locations of where the median was located in each block neighbourhood of your image.
Is it possible to use pagefun to calculate the norm (and execute other built-in GPU functions - http://www.mathworks.co.uk/help/distcomp/run-built-in-functions-on-a-gpu.html) on multiple pages simultaneously?
For example, I need to calculate the norms of a 3D array.
N = 10000
Sig = gpuArray(2,2000,N) % This is just to get an idea of the dimensions. Its populated elsewhere
% This is what I am currently doing.
for k = 1:N
TNorm(k,:) = [norm(Sig(1,:,k),2) norm(Sig(2,:,k),2)];
end
Is there a way to execute it in one go rather than iterate through the 3rd dimension to calculate the norm each time? Can something like this be done?
pagefun(norm(Sig,2)) % This gives the error: Input data must be a double or single vector or 2D matrix.
Thanks in advance! :)
Say you have a 3D array, in your case lets call it E:
E = rand(3,3,4) % This is a 3D array with size 3x3x4
if you want to calculate the norm of page so from 1 to 4 in this case, you can use:
norm_E = arrayfun(#(idx) norm(E(:,:,idx)), 1:size(E,3))
The output will be:
norm_E = [(norm array 1),(norm array 2),(norm array 3),(norm array 4)]
You are also not limited to the 2-norm you can use any norm you please for example if you wanted to use the infinity norm you could put:
norm_E = arrayfun(#(idx) norm(E(:,:,idx),inf), 1:size(E,3))
Using arrayfun is faster than a for loop. I apologize if you are restricted to using pagefun I do not have that feature with my version of Matlab.
I have a huge sparse matrix (1,000 x 1,000,000) that I cannot load on matlab (not enough RAM).
I want to visualize this matrix to have an idea of its sparsity and of the differences of the values.
Because of the memory constraints, I want to proceed as follows:
1- Divide the matrix into 4 matrices
2- Load each matrix on matlab and visualize it so that the colors give an idea of the values (and of the zeros particularly)
3- "Stick" the 4 images I will get in order to have a global idea for the original matrix
(i) Is it possible to load "part of a matrix" in matlab?
(ii) For the visualization tool, I read about spy (and daspect). However, this function only enables to visualize the non-zero values indifferently of their scales. Is there a way to add a color code?
(iii) How can I "stick" plots in order to make one?
If your matrix is sparse, then it seems that the currently method of storing it (as a full matrix in a text file) is very inefficient, and certainly makes loading it into MATLAB very hard. However, I suspect that as long as it is sparse enough, it can still be leaded into MATLAB as a sparse matrix.
The traditional way of doing this would be to load it all in at once, then convert to sparse representation. In your case, however, it would make sense to read in the text file, one line at a time, and convert to a MATLAB sparse matrix on-the-fly.
You can find out if this is possible by estimating the sparsity of your matrix, and using this to see if the whole thing could be loaded into MATLAB's memory as a sparse matrix.
Try something like: (untested code!)
% initialise sparse matrix
sparse_matrix = sparse(num_rows, num_cols);
row_num = 1;
fid = fopen(filename);
% read each line of text file in turn
while ~feof(fid)
this_line = fscanf(fid, '%f');
% add row to sparse matrix (note transpose, which I think is required)
sparse_matrix(row_num, :) = this_line';
row_num = row_num + 1;
end
fclose(fid)
% visualise using spy
spy(sparse_matrix)
Visualisation
With regards to visualisation: visualising a sparse matrix like this via a tool like imagesc is possible, but I believe it may internally create the full matrix – maybe someone can confirm if this is true or not. If it does, then it's going to cause you memory problems.
All spy is really doing is plotting in 2D the locations of the non-zero elements. You can fairly easily write your own spy function, which can have different coloured or sized points depending on the values at each location. See this answer for some examples.
Saving sparse matrices
As I say above, the method your matrix is saved as is pretty inefficient – for a matrix with 10% sparsity, around 95% of your text file will be a zero or a space. I don't know where this data has come from, but if you have any control over its creation (e.g. it comes from another program you have written) it would make much more sense to save only the non-zero elements in the format row_idx, col_idx, value.
You can then use spconvert to import the sparse matrix directly.
One of the simplest methods (if you can actually store the full sparse matrix in RAM) is to use gnuplot to visualize the sparisty pattern.
I was able to spy matrices of size 10-20GB using gnuplot without problems. But make sure you use png or jpeg formats to output the image. Note that you don't need the value of the non-zero entry only the integers (row, col). And plot them "plot "row_col.dat" using 1:2 with points".
This chooses your row as x axis and cols as your y axis and start plotting the non-zero entries. It is very easy to do this. This is the most scalable solution I know. Gnuplot works at decent speed even for very large datasets (>10GB of [row, cols]), but Matlab just hangs (with due respect)
I use imagesc() to visualise arrays. It scales the values in array to values between 0 and 1, then plots the array like a greyscale bitmap image (of course you can change the colormap to make it easier to see detail).