MATLAB convert from struct to table and ouput as csv - matlab

As part of an image processing pipeline using 'regionprops' in Matlab I generate the struct:
vWFfeatures =
1631x1 struct array with fields:
Area
Centroid
MajorAxisLength
MinorAxisLength
Eccentricity
EquivDiameter
Where 'Centroid' is a Vector containing [x, y] for example [12.4, 26.2]. I would like to convert this struct to a table and save as a CSV file. The objective is to separate the 'Centroid' vector into two columns in the table labelled Centroid_X and Centroid_Y for example. I am not sure how to achieve this.
So far I have investigated using the 'struct2table' function. This ouputs the 'Centroid' as one column. In addition when I try to assign the output to a variable I get an error:
table = struct2table(vWFfeatures)
Error using struct2table
Too many output arguments.
I cannot understand this, any help please?

Since the original struct2table isn't available to you, you might want to implement specifically the behavior you're trying to achieve yourself.
In this case, this means extracting the values you want to save, (split the array,) then save the data:
data_Centroid = vertcat(vWFfeatures.Centroid); %// contains the centroid data
Centroid_X = data_Centroid(:,1); %// The first column is X
Centroid_Y = data_Centroid(:,2); %// the second column is Y
csvwrite('centroid.csv',data_Centroid); %// writes values into csv
If you want column headers in your csv, it gets complicated because csvwrite can only handle numeric arrays:
celldata = num2cell(num2str(data_Centroid)); %// create cell array
celldata(:,3) = celldata(:,4); %// copy col 4 (y data) into col 3 (spaces)
for i=1:length(celldata)
celldata{i,2} = ','; %// col 2 has commas
celldata{i,4} = '\n'; %// col 4 has newlines
end
celldata = celldata'; %'// transpose to make the entries come columnwise
strdata = ['Centroid_X,Centroid_Y\n',celldata{:}]; %// contains all as string
fid = fopen('centroid.csv','w'); % writing the string into the csv
fprintf(fid,strdata);
fclose(fid);

This is how I solved it in the end: extracted each field from struct used horzcat to join into a new array then defined headers and used csvwrite_with_headers, to ouput to csv.
wpbFeatures = regionprops(vWFlabelled, 'Area','Centroid', 'MajorAxisLength', 'MinorAxisLength', 'Eccentricity', 'EquivDiameter');
wpbArea = vertcat(wpbFeatures.Area);
wpbCentroid = vertcat(wpbFeatures.Centroid);
wpbCentroidX = wpbCentroid(:,1);
wpbCentroidY = wpbCentroid(:,2);
wpbFeret = max(imFeretDiameter(vWFlabelled, linspace(0,180,201)), [], 2);
wpbMajorAxisLength = vertcat(wpbFeatures.MajorAxisLength);
wpbMinorAxisLength = vertcat(wpbFeatures.MinorAxisLength);
wpbEccentricity = vertcat(wpbFeatures.Eccentricity);
wpbEquivDiameter = vertcat(wpbFeatures.EquivDiameter);
wpbFeatures = horzcat(wpbArea, wpbCentroidX, wpbCentroidY, wpbFeret, wpbMajorAxisLength, wpbMinorAxisLength, wpbEccentricity, wpbEquivDiameter);
headers = {'Area','CentroidX','CentroidY', 'Feret', 'MajorAxisLength', 'MinorAxisLength', 'Eccentricity', 'EquivDiameter'};
csvwrite_with_headers(strcat(PlateName, '_ResultsFeatures.csv'),wpbFeatures,headers);

Related

How to load tables and save tables from different .mat files and create new table dimension for each file

I am using Matlab 2019 and I saved a large number of different tables with the same name, each in a .mat-file with a different name. I'd like to load them and save them in a new table together.
I need a table that has the same variables (variable1, variable2, variable3), and numbers but an additional dimension for the different .mat-files. I.e. in the dummy code below the size of the table allData should be: 2 x 3 x 2.
If I simply concatenate them like this [Results1.Inpts; Results2.Inpts], then the table is 4 x 3 and I cannot distinguish the data from the different files.
Also I have a large number of .mat-files, so I would like to automate it like done here (https://www.mathworks.com/matlabcentral/answers/233745-loading-mat-files-with-same-variable-name), which I tried to do for my table below, but it didn't work.
Please find my dummy code below.
%% Create data and .mat files
clc;
clear;
close all;
format long;
Names = {'variable1','variable2','variable3'};
Numbers1 = [2, 3, 4; 3, 4, 5];
Numbers2 = [1, 4, 10; 2, 6, 3];
Inpts = array2table(Numbers1,'VariableNames',Names);
save('Results1.mat');
clear Inpts;
Inpts = array2table(Numbers2,'VariableNames',Names);
save('Results2.mat');
clear;
%% Analyse data
files = dir('*.mat');
allData = table(size(files));
for ii=1:length(files)
aFileData = load(files(ii).name);
allData{ii} = aFileData.Inpts;
end
I think you just need to add a column in each table before concatenation
allData = cell(numel(files),1); % Store the tables in a cell
for ii=1:length(files)
aFileData = load(files(ii).name);
% Add a column
aFileData.FileName(:) = {files(ii).name};
% Stick the modified table in the cell
allData{ii} = aFileData.Inpts;
end
% Merge tables
allData = vertcat( allData{:} );
If you really want to do a 3-dimensional array with the third dimension corresponding to files, you can't use a table, because tables are inherently two-dimensional. Instead, convert your table data back to a matrix after reading it, and then concatenate them along the third dimension.
allData = cell(numel(files), 1);
for i = 1:numel(files)
fileTable = load(files(i).name);
fileMat = table2array(fileTable);
allData{i} = fileMat;
end
data = cat(3, allData{:});

Importing a txt file into Matlab and reading the data

I need to import a .txt datafile into Matlab. The file has been made into 3 columns. Each column has specific numbers for a given variable. The script code must be able to do the following,
Requirement
1) import the data from txt into Matlab
2) Matlab should remove the values from the columns if the values are out of a certain range
3) Matlab should tell which line and what type of error.
My Approach
I have tried using the following approach,
function data = insertData(filename)
filename = input('Insert the name of the file: ', 's');
data = load(filename);
Column1 = data(:,1);
Column2 = data(:,2);
Column3 = data(:,3);
%Ranges for each column
nclm1 = Column1(Column1>0);
nclm2 = Column2(Column2 >= 10 & Column2 <= 100);
nclm3 = Column3(Column3>0);
%Final new data columns within the ranges
final = [nclm1, nclm2, nclm3];
end
Problem
The above code has the following problems:
1) Matlab is not saving the imported data as 'data' after the user inserts the name of the file. Hence I don't know why my code is wrong.
filename =input('Insert the name of the file: ', 's');
data = load(filename);
2) The columns in the end do not have the same dimensions because I can see that Matlab removes values from the columns independently. Therefore is there a way in which I can make Matlab remove values/rows from a matrix rather than the three 'vectors', given a range.
1) Not sure what you mean by this. I created a sample text file and Matlab imports the data as data just fine. However, you are only returning the original unfiltered data so maybe that is what you mean??? I modified it to return the original data and the filtered data.
2) You need to or the bad indices together so that they are removed from each column like this. Note I made some other edits ... see comments in the code below:
function [origData, filteredData]= insertData(filename)
% You pass in filename then overwrite it ...
% Modified to only prompt if not passed in.
if ~exist('filename','var') || isempty(filename)
filename = input('Insert the name of the file: ', 's');
end
origData = load(filename);
% Ranges check for each column
% Note: return these if you want to know what data was filter for
% which reason
badIdx1 = origData(:,1) > 0;
badIdx2 = origData(:,2) >= 10 & origData(:,2) <= 100;
badIdx3 = origData(:,3)>0;
totalBad = badIdx1 | badIdx2 | badIdx3;
%Final new data columns within the ranges
filteredData = origData(~totalBad,:);
end
Note: you mentioned you want to know which line for which type of error. That information is now contained in badIDx1,2, 3. So you can return them, print a message to the screen, or whatever you need to display that information.

How to store .csv data and calculate average value in MATLAB

Can someone help me to understand how I can save in matlab a group of .csv files, select only the columns in which I am interested and get as output a final file in which I have the average value of the y columns and standard deviation of y axes? I am not so good in matlab and so I kindly ask if someone to help me to solve this question.
Here what I tried to do till now:
clear all;
clc;
which_column = 5;
dirstats = dir('*.csv');
col3Complete=0;
col4Complete=0;
for K = 1:length(dirstats)
[num,txt,raw] = xlsread(dirstats(K).name);
col3=num(:,3);
col4=num(:,4);
col3Complete=[col3Complete;col3];
col4Complete=[col4Complete;col4];
avgVal(K)=mean(col4(:));
end
col3Complete(1)=[];
col4Complete(1)=[];
%columnavg = mean(col4Complete);
%columnstd = std(col4Complete);
% xvals = 1 : size(columnavg,1);
% plot(xvals, columnavg, 'b-', xvals, columnavg-columnstd, 'r--', xvals, columnavg+columstd, 'r--');
B = reshape(col4Complete,[5000,K]);
m=mean(B,2);
C = reshape (col4Complete,[5000,K]);
S=std(C,0,2);
Now I know that I should compute mean and stdeviation inside for loop, using mean()function, but I am not sure how I can use it.
which_column = 5;
dirstats = dir('*.csv');
col3Complete=[]; % Initialise as empty matrix
col4Complete=[];
avgVal = zeros(length(dirstats),2); % initialise as columnvector
for K = 1:length(dirstats)
[num,txt,raw] = xlsread(dirstats(K).name);
col3=num(:,3);
col4=num(:,4);
col3Complete=[col3Complete;col3];
col4Complete=[col4Complete;col4];
avgVal(K,1)=mean(col4(:)); % 1st column contains mean
avgVal(K,2)=std(col4(:)); % 2nd column contains standard deviation
end
%columnavg = mean(col4Complete);
%columnstd = std(col4Complete);
% xvals = 1 : size(columnavg,1);
% plot(xvals, columnavg, 'b-', xvals, columnavg-columnstd, 'r--', xvals, columnavg+columstd, 'r--');
B = reshape(col4Complete,[5000,K]);
meanVals=mean(B,2);
I didn't change much, just initialised your arrays as empty arrays so you do not have to delete the first entry later on and made avgVal a column vector with the mean in column 1 and the standard deviation in column 1. You can of course add two columns if you want to collect those statistics for your 3rd column in the csv as well.
As a side note: xlsread is rather heavy for reading files, since Excel is horribly inefficient. If you want to read a structured file such as a csv, it's faster to use importdata.
Create some random matrix to store in a file with header:
A = rand(1e3,5);
out = fopen('output.csv','w');
fprintf(out,['ColumnA', '\t', 'ColumnB', '\t', 'ColumnC', '\t', 'ColumnD', '\t', 'ColumnE','\n']);
fclose(out);
dlmwrite('output.csv', A, 'delimiter','\t','-append');
Load it using csvread:
data = csvread('output.csv',1);
data now contains your five columns, without any headers.

Save a sparse array in csv

I have a huge sparse matrix a and I want to save it in a .csv. I can not call full(a) because I do not have enough ram memory. So, calling dlmwrite with full(a) argument is not possible. We must note that dlmwrite is not working with sparse formatted matrices.
The .csv format is depicted below. Note that the first row and column with the characters should be included in the .csv file. The semicolon in the (0,0) position of the .csv file is necessary too.
;A;B;C;D;E
A;0;1.5;0;1;0
B;2;0;0;0;0
C;0;0;1;0;0
D;0;2.1;0;1;0
E;0;0;0;0;0
Could you please help me to tackle this problem and finally save the sparse matrix in the desired form?
You can use csvwrite function:
csvwrite('matrix.csv',a)
You could do this iteratively, as follows:
A = sprand(20,30000,.1);
delimiter = ';';
filename = 'filecontaininghugematrix.csv';
dims = size(A);
N = max(dims);
% create names first
idx = 1:26;
alphabet = dec2base(9+idx,36);
n = ceil(log(N)/log(26));
q = 26.^(1:n);
names = cell(sum(q),1);
p = 0;
for ii = 1:n
temp = repmat({idx},ii,1);
names(p+(1:q(ii))) = num2cell(alphabet(fliplr(combvec(temp{:})')),2);
p = p + q(ii);
end
names(N+1:end) = [];
% formats for writing
headStr = repmat(['%s' delimiter],1,dims(2));
headStr = [delimiter headStr(1:end-1) '\n'];
lineStr = repmat(['%f' delimiter],1,dims(2));
lineStr = ['%s' delimiter lineStr(1:end-1) '\n'];
fid = fopen(filename,'w');
% write header
header = names(1:dims(2));
fprintf(fid,headStr,header{:});
% write matrix rows
for ii = 1:dims(1)
row = full(A(ii,:));
fprintf(fid, lineStr, names{ii}, row);
end
fclose(fid);
The names cell array is quite memory demanding for this example. I have no time to fix that now, so think about this part yourself if it is really a problem ;) Hint: just write the header element wise, first A;, then B; and so on. For the rows, you can create a function that maps the index ii to the desired character, in which case the complete first part is not necessary.

How to import a sequence of Excel Files in matlab as a column vectors or as a cell array?

I want to import a sequence of excel files with a large amount of data in them. The problem that I have is I want to process the data in each file at a time and store the output from this into a variable, but each time I try to process a different file the variable gets overwritten in the variable workspace. Is there anyway I could store these files and process each file at a time?
numFiles = 1;
range = 'A2:Q21';
sheet = 1;
myData = cell(1,numFiles); % Importing data from Excel
for fileNum = 1:numFiles
fileName = sprintf('myfile%02d.xlsx',fileNum);
myData{fileNum} = importfile3(fileName,sheet,range);
end
data = cell2mat(myData);
The actual data import is performed by importfile3 which is, for the most part, a wrapper for the xlsread function that returns a matrix corresponding to the specified range of excel data.
function data = importfile3(workbookFile, sheetName, range)
% If no sheet is specified, read first sheet
if nargin == 1 || isempty(sheetName)
sheetName = 1;
end
% If no range is specified, read all data
if nargin <= 2 || isempty(range)
range = '';
end
%% Import the data
[~, ~, raw] = xlsread(workbookFile, sheetName, range);
%% Replace non-numeric cells with 0.0
R = cellfun(#(x) ~isnumeric(x) || isnan(x),raw); % Find non-numeric cells
raw(R) = {0.0}; % Replace non-numeric cells
%% Create output variable
data = cell2mat(raw);
The issue that you are running in to is a result of cell2mat concatenating all of the data in your cells in to one large 2-dimensional matrix. If you were to import two excel files with 20 rows and 17 columns, each, this would result in a 2-dimensional matrix of size [20 x 34]. The doc for cell2mat has a nice visual describing this.
I see that your importfile3 function returns a matrix, and based on your use of cell2mat in your final line of code, it looks like you would like to have your final result be in the form of a matrix. So I think the easiest way to go about this is to just bypass the intermediate myData cell array.
In the example code below, the resulting data is a 3-dimensional matrix. The 1st dimension indicates row number, 2nd dimension is column number, and 3rd dimension is file number. Cell arrays are very useful for "jagged" data, but based on the code you provided, each excel data set that you import will have the same number of rows and columns.
numFiles = 2;
range = 'A2:Q21';
sheet = 1;
% Number of rows and cols known before data import
numRows = 20;
numCols = 17;
data = zeros(numRows,numCols,numFiles);
for fileNum = 1:numFiles
fileName = sprintf('myfile%02d.xlsx',fileNum);
data(:,:,fileNum) = importfile3(fileName,sheet,range);
end
Accessing this data is now very straight-forward.
data(:,:,1) returns the data imported from your first excel file.
data(:,:,2) returns the data imported from your second excel file.
etc.