Is there a way to stack multiple 2x2 matrices in MATLAB into a multidimensional array in a simpler way (e.g. without using "cat" or "reshape")? - matlab

I am receiving a text file with 1000 matrices of size 2x2 each day from someone, in the following format (only 3 matrices are shown here instead of 1000):
0.96875000 0.03125000
0.03125000 0.96875000
0.96875000 0.01562500
0.03125000 0.98437500
0.99218800 0.03125000
0.00781250 0.96875000
I need to make a 2x2x1000 array in MATLAB. Ideally I could do something simple like:
[0.96875000 0.03125000
0.03125000 0.96875000;
0.96875000 0.01562500
0.03125000 0.98437500;
0.99218800 0.03125000
0.00781250 0.96875000]
After reading the MATLAB documentation on multidimensional arrays and the MATLAB documentation for the cat function, I figured out that I could make the required array in the following way (the first argument of cat is 3 because I'm concatenating the 2x2 matrices along the 3rd dimension):
cat(3,...
[0.96875000 0.03125000
0.03125000 0.96875000],...
[0.96875000 0.01562500
0.03125000 0.98437500],...
[0.99218800 0.03125000
0.00781250 0.96875000])
But that does not work if I put spacing between the lines as in my "ideal" example above, and the need for all the commas and dots makes it a bit uglier in my opinion.
While writing this question, I have discovered that I can run my "ideal" example and then use reshape, which I prefer over my solution using the cat function. For this, I don't even need the semi-colons. However Cris Luengo correctly pointed out in the comments that reshape is not enough and permute is also needed, and then Luis Mendo pointed out in chat that the solution is not so simple:
permute(reshape(ideal.',2,2,[]),[2 1 3])
Andras Deak has done what we thought was impossible, which is to remove the transpose, but the solution is still quite complicated, and was not easy to engineer:
permute(reshape(ideal,2,[],2),[1 3 2])
Ideally one would not need to use cat or reshape to make a 3D array, when the original data is already so nicely formatted in what the human eye can already see is a 3D array of several 2x2 matrices.
Is there a simpler way to build the 3D array in MATLAB using the data in the format I have?
So far I have done the following on my own:
Searched online and found the above two MATLAB documentation articles which lead me to the above solution using cat
Came up with the above solution using reshape while writing this question, then it got improved by Cris and Luis in the comments and chat 😊.
Also: I tried saving the data in a .txt file and clicked import in MATLAB, knowing that the import GUI gives some options for how the data is to be organized in the resulting MATLAB array, but there did not seem to be any option to make this a 3D array.

Indeed there is no "direct" way to import this text as a 3D matrix. This is the easiest way I can come up with:
Save the input as a .txt file
Use the import tool (Import Data button in the Variable toolbar) to import the data as a Mx2 matrix. Choose "Numeric Matrix" as "Output Type". And you can "exclude rows with" "blank cells" to avoid the empty rows.
Besides reshape() and permute(), using cell array to format it as below might be more intuitive and less error prone to someones.
% The number of 2x2 matrices
N = size(m,1)/2;
% Split each 2x2 matrix into a cell
c = mat2cell(m, 2*ones(1,N), 1);
% Concatenate along the 3rd dimension
output3DMatrix = cat(3, c{:});

Related

How to read a complex 3D matrix (binary file) in Matlab without using interleaved/reshaping method?

I have a very huge 3D matrix, the data was written into disk for future use. Writing the matrix into a bin is easy, reading it back however have some issue.
Write to bin:
z=repmat(complex(rand(5),rand(5)),[1 1 5])
z_imag = imag(z);
z_real = real(z);
adjacent = [z_real z_imag];
fileID = fopen('complex.bin','w');
fwrite(fileID,adjacent,'double')
And now, I try to read it back using memmapfile:
m = memmapfile('complex.bin', 'Offset', 0, 'Format', {'double' [5,5,5] 'x'});
complexValues = complex(m.Data(:).x(1,:), m.Data(:).x(2,:)); %this line doesn't work though, just for explanation's sake
It gave me an error saying that
Error using memmapfile/subsref (line 764) A subscripting operation on
the Data field attempted to create a comma-separated list. The
memmapfile class does not support the use of comma-separated lists
when subscripting.
I was referring to the solution here, the suggested solution used the reshape to shape the matrix beforehand (as contrast to my method above). I try to avoid using reshape in my code as I'm dealing with very huge data and that might computationally expensive and takes a long time. Is there an alternative/better way to do this?
Thanks in advance!

MATLAB: making a histogram plot from csv files read and put into cells?

Unfortunately I am not too tech proficient and only have a basic MATLAB/programming background...
I have several csv data files in a folder, and would like to make a histogram plot of all of them simultaneously in order to compare them. I am not sure how to go about doing this. Some digging online gave a script:
d=dir('*.csv'); % return the list of csv files
for i=1:length(d)
m{i}=csvread(d(i).name); % put into cell array
end
The problem is I cannot now simply write histogram(m(i)) command, because m(i) is a cell type not a csv file type (I'm not sure I'm using this terminology correctly, but MATLAB definitely isn't accepting the former).
I am not quite sure how to proceed. In fact, I am not sure what exactly is the nature of the elements m(i) and what I can/cannot do with them. The histogram command wants a matrix input, so presumably I would need a 'vector of matrices' and a command which plots each of the vector elements (i.e. matrices) on a separate plot. I would have about 14 altogether, which is quite a lot and would take a long time to load, but I am not sure how to proceed more efficiently.
Generalizing the question:
I will later be writing a script to reduce the noise and smooth out the data in the csv file, and binarise it (the csv files are for noisy images with vague shapes, and I want to distinguish these shapes by setting a cut off for the pixel intensity/value in the csv matrix, such as to create a binary image showing these shapes). Ideally, I would like to apply this to all of the images in my folder at once so I can shift out which images are best for analysis. So my question is, how can I run a script with all of the csv files in my folder so that I can compare them all at once? I presume whatever technique I use for the histogram plots can apply to this too, but I am not sure.
It should probably be better to write a script which:
-makes a histogram plot and/or runs the binarising script for each csv file in the folder
-and puts all of the images into a new, designated folder, so I can sift through these.
I would greatly appreciate pointers on how to do this. As I mentioned, I am quite new to programming and am getting overwhelmed when looking at suggestions, seeing various different commands used to apparently achieve the same thing- reading several files at once.
The function csvread returns natively a matrix. I am not sure but it is possible that if some elements inside the csv file are not numbers, Matlab automatically makes a cell array out of the output. Since I don't know the structure of your csv-files I will recommend you trying out some similar functions(readtable, xlsread):
M = readtable(d(i).name) % Reads table like data, most recommended
M = xlsread(d(i).name) % Excel like structures, but works also on similar data
Try them out and let me know if it worked. If not please upload a file sample.
The function csvread(filename)
always return the matrix M that is numerical matrix and will never give the cell as return.
If you have textual data inside the .csv file, it will give you an error for not having the numerical data only. The only reason I can see for using the cell array when reading the files is if the dimensions of individual matrices read from each file are different, for example first .csv file contains data organised as 3xA, and second .csv file contains data organised as 2xB, so you can place them all into a single structure.
However, it is still possible to use histogram on cell array, by extracting the element as an array instead of extracting it as cell element.
If M is a cell matrix, there are two options for extracting the data:
M(i) and M{i}. M(i) will give you the cell element, and cannot be used for histogram, however M{i} returns element in its initial form which is numerical matrix.
TL;DR use histogram(M{i}) instead of histogram(M(i)).

Dynamically naming and exporting MAT files

I've tried to search stackoverflow for a solution, but the cases I found differ slightly from what I'm trying to do, so I thought I'd ask.
I have a loop in MATLAB, which for every iteration, a large matrix is computed. I want to save each matrix as a separate MAT file, however each file needs to be named according to its position in loop. For example: matrix1, matrix2,...
The method I'm using to save my data (which seemed different from the few examples I found) is the following (where matrix is the generated matrix and matrix1 is the filename to be saved for the matrix corresponding to i = 1)
save matrix1 matrix;
I've seen something similar to
save ['matrix', i] matrix;
But I can't seem to remember the exact syntax.
Sorry if the question is very basic, a simple nod in the right direction for this type of saving would be greatly appreciated.
Use the functional form of save:
save(['matrix', int2str(i)], 'matrix');
Here's my nod:
eval(['save matrix' num2str(i) ' matrix;']);
Good luck! :)

Suppress kinks in a plot matlab

I have a csv file which contains data like below:[1st row is header]
Element,State,Time
Water,Solid,1
Water,Solid,2
Water,Solid,3
Water,Solid,4
Water,Solid,5
Water,Solid,2
Water,Solid,3
Water,Solid,4
Water,Solid,5
Water,Solid,6
Water,Solid,7
Water,Solid,8
Water,Solid,7
Water,Solid,6
Water,Solid,5
Water,Solid,4
Water,Solid,3
The similar pattern is repeated for State: "Solid" replaced with Liquid and Gas.
And moreover the Element "Water" can be replaced by some other element too.
Time as Integer's are in seconds (to simplify) but can be any real number.
Additionally there might by some comment line starting with # in between the file.
Problem Statement: I want to eliminate the first dip in Time values and smooth out using some quadratic or cubic or polynomial interpolation [please notice the first change from 5->2 --->8. I want to replace these numbers to intermediate values giving a gradual/smooth increase from 5--->8].
And I wish this to be done for all the combinations of Elements and States.
Is this possible through some sort of coding in Matlab etc ?
Any Pointers will be helpful !!
Thanks in advance :)
You can use the interp1 function for 1D-interpolation. The syntax is
yi = interp1(x,y,xi,method)
where x are your original coordinates, y are your original values, xi are the coordinates at which you want the values to be interpolated at and yi are the interpolated values. method can be 'spline' (cubic spline interpolation), 'pchip' (piece-wise Hermite), 'cubic' (cubic polynomial) and others (see the documentation for details).
You have alot of options here, it really depends on the nature of your data, but I would start of with a simple moving average (MA) filter (which replaces each data point with the average of the neighboring data points), and see were that takes me. It's easy to implement, and fine-tuning the MA-span a couple of times on some sample data is usually enough.
http://www.mathworks.se/help/curvefit/smoothing-data.html
I would not try to fit a polynomial to the entire data set unless I really needed to compress it, (but to do so you can use the polyfit function).

How to read text fields into MATLAB and create a single matrix

I have a huge CSV file that has a mix of numerical and text datatypes. I want to read this into a single matrix in Matlab. I'll use a simpler example here to illustrate my problem. Let's say I have this CSV file:
1,foo
2,bar
I am trying to read this into MatLab using:
A=fopen('filename.csv');
B=textscan(A,'%d %d', 'delimiter',',');
C=cell2mat(B);
The first two lines work fine, but the problem is that texscan doesn't create a 2x2 matrix; instead it creates a 1x2 matrix with each value being an array. So I try to use the last line to combine the arrays into one big matrix, but it generates an error because the arrays have different datatypes.
Is there a way to get around this problem? Or a better way to combine the arrays?
I am note sure if combining them is a good idea. It is likely that you would be better off with them separate.
I changed your code, so that it works better:
clear
clc
A=fopen('filename.csv');
B=textscan(A,'%d %s', 'delimiter',',')
fclose(A)
Looking at the results
K>> B{1}
ans =
1
2
K>> B{2}
ans =
'foo'
'bar'
Really, I think this is the format that is most useful. If anything, most people would want to break this cell array into smaller chunks
num = B{1}
txt = B{2}
Why are your trying to combine them? They are already together in a cell array, and that is the most combined you are going to get.
There is a natural solution to this, but it requires the Statistics toolbox (version 6.0 or higher). Mixed data types can be read into a dataset array. See the Mathworks help page here.
I believe you can't use textscan for this purpose. I'd use fscanf which always gives you a matrix as specified. If you don't know the layout of the data it gets kind of tricky however.
fscanf works as follows:
fscanf(fid, format, size)
where fid is the fid generated by the fopen
format is the file format & how you are reading the data (['%d' ',' '%s'] would work for your example file)
size is the matrix dimensions ([2 2] would work on your example file).