This is a problem I am working on in Matlab.
I am looping through a series of *.ALL files and stripping the field name by a delimiter '.'. The first field is the network, second station name, third location code and fourth component. I pre-allocate my structure based on the number of files (3) I run through which for this example is a 3x3x3 structure that I would like to define as struct(station,loc code,component). You can see these definitions in my code example below.
I would like to loop through the station, loc code, and component and fill their values in the structure. The only problem is for some reason the way I've defined the loop it's actually looping through the files more than once. I only want to loop through each file once and extract the station, comp, and loc code from it and put it inside the structure. Because it's looping through the files more than once it's taken like 10 minutes to fill the structure. This is not very efficient at all. Can someone point out the culprit line for me or tell me what I'm doing incorrectly?
Here's my code below:
close all;
clear;
[files]=dir('*.ALL');
for i = 1:length(files)
fields=textscan(files(i).name, '%s', 'Delimiter', '.');
net{i,1}=fields{1}{:};
sta{i,1}=fields{1}{2};
loc{i,1}=fields{1}{3};
comp{i,1}=fields{1}{4};
data = [];
halfhour(1:2) = struct('data',data);
hour(1:24) = struct('halfhour',halfhour);
day(1:366) = struct('hour',hour);
PSD_YE_DBT(1:length(files),1:length(files),1:length(files)) =
struct('sta','','loc','','comp','','allData',[],'day',day);
end
for s=1:1:length(sta)
for l=1:1:length(loc)
for c=1:1:length(comp)
tempFileName = strcat(net{s},'.',sta{s},'.',loc{l},'.',comp{c},'.','ALL');
fid = fopen(tempFileName);
PSD_YE_DBT(s,l,c).sta = sta{s};
PSD_YE_DBT(s,l,c).loc = loc{l};
PSD_YE_DBT(s,l,c).comp = comp{c};
end
end
end
Example file names for the three files I'm working with are:
XO.GRUT.--.HHE.ALL
XO.GRUT.--.HHN.ALL
XO.GRUT.--.HHZ.ALL
Thanks in advance!
Related
So I have an iterative loop that extracts data from .csv files in MATLAB's active folder and plots it. I would like to take it one step further and run the script on two folders, each with their own .csv files.
One folder is called stress and the other strain. As the name implies, they contain .csv files for stress and strain data for several samples, each of which is called E3-01, E3-02, E3-03, etc. In other words, both folders have the same number of files and the same names.
The way I see it, the process would have the following steps:
Look in the stress folder, look inside file E3-01, extract the data in the column labelled Stress
Look in the strain folder, look inside file E3-01, extract the data in the column labelled Strain
Combine the data together for sample E3-01 and plot it
Repeat steps 1-3 for all files in the folders
Like I said, I already have a script that can find the right column and extract the data. What I'm not sure about is how to tell MATLAB to alternate the folder that the script is being run on.
Instead of a script, would a function be better? Something that accepts 4 inputs: the names of the two folders and the columns to extract?
EDIT: Apologies, here's the code I have so far:
clearvars;
files = dir('*.csv');
prompt = {'Plot name:','x label:','y label:','x values:','y values:','Points to eliminate:'};
dlg_title = 'Input';
num_lines = 1;
defaultans = {'Title','x label','y label','Surface component 1.avg(epsY) [True strain]','Stress','0'};
answer = inputdlg(prompt,dlg_title,num_lines,defaultans);
name_plot = answer{1};
x_label = answer{2};
y_label = answer{3};
x_col = answer{4};
y_col = answer{5};
des_cols = {y_col,x_col};
smallest_n = 100000;
points_elim = answer{6};
avg_x_values = [];
avg_y_values = [];
for file = files'
M=xlsread(file.name);
[row,col]=size(M);
if smallest_n > row
smallest_n = row;
end
end
smallest_n=smallest_n-points_elim;
avg_x_values = zeros(smallest_n,size(files,1));
avg_y_values = zeros(smallest_n,size(files,1));
hold on;
set(groot, 'DefaultLegendInterpreter', 'none');
set(gca,'FontSize',20);
ii = 0;
for file = files'
ii = ii + 1;
[n,s,r] = xlsread(file.name);
colhdrs = s(1,:);
[row, col] = find(strcmpi(s,x_col));
x_values = n(1:end-points_elim,col);
[row, col] = find(strcmpi(s,y_col));
y_values = n(1:end-points_elim,col);
plot(x_values,y_values,'DisplayName',s{1,1});
avg_x_values(:,ii)=x_values(1:smallest_n);
avg_y_values(:,ii)=y_values(1:smallest_n);
end
ylabel({y_label});
xlabel({x_label});
title({name_plot});
colormap(gray);
hold off;
avg_x_values = mean(avg_x_values,2);
avg_y_values = mean(avg_y_values,2);
plot(avg_x_values,avg_y_values);
set(gca,'FontSize',20);
ylabel({y_label});
xlabel({x_label});
title({name_plot});
EDIT 2: #Adriaan I tried to write the following function to get a column from a file:
function [out_col] = getcolumn(col,file)
file = dir(file);
[n,s,r] = xlsread(file.name);
colhdrs = s(1,:);
[row, col] = find(strcmpi(s,col));
out_col = n(1:end,col);
end
but I get the error
Function 'subsindex' is not defined for values of class 'struct'.
Error in getcolumn (line 21)
y = x(:,n);
not sure why.
You can do both, of course, and it depends on preference mainly, provided you're the sole user of the script. If others are going to use it as well, use functions instead, as they can contain a proper help file and calling help functionname will then give you that help.
For instance:
folders1 = dir(../strain/*)
folders2 = dir(../stress/*)
for ii 1 = 1:numel(folders)
operand1 = folders1{ii};
operand2 = folders2{ii};
%... rest of script
%
% Or function:
data = YourFunction(folders1{ii},folders2{ii})
end
So all in all you can use both, although from experience I find functions easier to use in the end, as you just pass parameters and don't need to trawl through the complete code to change the parameters each run.
Additionally you can partition off small parts of your program which do a fix task. If you nest your functions, and finally call just a single function in your scripts, you don't have to look at hundreds of lines of code each time you run the script, but rather can just run a single function (which can also be inside a script or function, ad infinitum).
Finally, a function has its own scope; meaning that any variables that are in that function stay within that function unless you explicitly set them as output (apart from global variables, but those are problematic anyway). This can be a good thing, or a bad thing, depending on the rest of your code. If you function would output ~20 variables for further processing, the function probably should include more steps. It'd be a good thing if you create lots of intermediate variables (I always do), because when the function's finished running, the scope of that function will be removed from memory, saving you clear tmpVar1 tmpVar2 tmpVar3 etc every few lines in your script.
For the script the argument in favour would be that it is easier to debug; you don't need dbstop on error and can step a bit easier through the script, keeping check of all your variables. But, after the debugging has been completed, this argument becomes moot, and thus in general I'd start with writing a script, and once it performs as desired, I rework it to a function at minimal extra effort.
I am beginner in MATLAB. I would like to load 200 image files (size 192x192) in a specific folder by using a for loop.
The image names are '1.png', '2.png', '3.png' and so on.
My code is as below.
list = dir('C:/preds/*.png');
N = size(list,1);
sum_image = zeros(192,192,200);
for i = 1:N
sum_image(:,:,i) = imread('C:/preds/i.png');
end
Which part should I change ?
I would probably do it like the code below:
You are currently getting the list of filenames then not really doing much with it. Iterating over the list is safer otherwise if there is a missing number you could have issues. Also, the sort maybe unnecessary depending if you image numbering is zero-padded so they come out in the correct order ... but better safe than sorry. One other small change initializing the array to size N instead of hard-coding 200. This will make it more flexible.
searchDir = 'C:\preds\';
list = dir([searchDir '*.png']);
nameList = {list.name}; %Get array of names
imNum = str2double(strrep(nameList,'.png','')); %Get image number
[~,idx] = sort(imNum); %sort it
nameList = nameList(idx);
N = numel(nameList);
sum_image = zeros(192,192,N);
for i=1:N
sum_image(:,:,i) = imread(fullfile(searchDir,nameList{i}));
end
I would suggest changing the line within the loop to the following:
sum_image(:,:,i) = imread(['C:/preds/', num2str(i), '.png']);
MATLAB treats the i in your string as a character and not the variable i. The above line of code builds your string piece by piece.
If this isn't a homework problem, the right answer to this question is don't write this as a for loop. Use an imageDatastore:
https://www.mathworks.com/help/matlab/ref/imagedatastore.html
ds = imageDatastore('C:/preds/');
sumImageCellArray = readall(ds);
sumImage = cat(3,sumImageCellArray{:});
I'm new to MATLAB, and I can't manage to make my function work in order to save my data into a .mat file.
The input:
A structure, with 5 fields:
data: 3D matrix of 19x1000x143
labels: 1x143 matrix with 1 or -1 in it
subject_number: an integer
sampling_rate: an integer, 500 Hz
channel_names: 1x19 matrix with text in it
name: a string for the name of the file
clean: a matrix 1x143 with 1 or 0 in it.
The idea is to save only the clean data, marked as 1 in the clean matrix.
If clean(i) is equal to 1:
save data(:,:,i) and labels(:,i)
This is the code I've tried to implement in the saving.m file:
function saving(EEG_struct, clean, name)
subject_number = EEG_struct.subject_number;
fs = EEG_struct.sampling_rate;
chan_names = EEG_struct.channel_names;
nb_epoch = size(EEG_struct.data, 3);
for j=1:nb_epoch
if clean(j) == 1
% Keep the epoch and label
data = cat(3, data, EEG_struct.data(:,:,j));
labels = cat(2, labels, EEG_struct.labels(:,j));
end
end
save(name, data, labels, subject_number, fs, chan_names)
As you can see, I would like to save the data as a structure with the same shape as the EEG_struct input.
Moreover, I would like to use a parfor instead of a for, but it raised me an error I didn't quite get:
An UndefinedFunction error was thrown on the workers for 'data'. This might be because the file containing 'data' is not accessible on the workers. Use addAttachedFiles(pool, files) to specify the required files to be attached. See the documentation for 'parallel.Pool/addAttachedFiles' for more details. Caused by: Undefined function or variable 'data'.
Thanks for the help !
You can use your clean variable as a logical index and parse out your data and labels at once. So there is no need for a loop.
Also the save command needs the "names" of the vars to save not the variables themselves. So I just put ' ' around each one.
function saving(EEG_struct, clean, name)
subject_number = EEG_struct.subject_number;
fs = EEG_struct.sampling_rate;
chan_names = EEG_struct.channel_names;
nb_epoch = size(EEG_struct.data, 3);
%No need for a loop at all
data = EEG_struct.data(:,:,logical(clean));
labels = EEG_struct.labels(logical(clean)); %This is a 1xN so I removed the extra colon operator
save(name, 'data', 'labels', 'subject_number', 'fs', 'chan_names');
EDIT:
Per you comment if you want to just leave everything in the structure. I gave you 2 options for how to save it.
function saving(EEG_struct, clean, name)
%Crop out ~clead data
EEG_struct.data = EEG_struct.data(:,:,logical(clean));
EEG_struct.labels = EEG_struct.labels(logical(clean)); %This is a 1xN so I removed the extra colon operator
% Option 1
save(name, 'EEG_struct');
% Option2
save(name, '-struct', 'EEG_struct');
Option 1 will directly save the struct to the MAT file. So if you were to load the data back like this:
test = load(name);
test =
EEG_struct: [1x1 struct]
You would get your structure placed inside another structure ... which might not be ideal or require an extra line to de-nest it. On the other hand just loading the MAT file with no outputs load(name) would put EEG_struct into your current workspace. But if in a function then it sort of springs into existence without every being declared which makes code a bit harder to follow.
Option 2 uses the '-struct' option which breaks out each field automatically into separate vars in the MAT file. So loading like this:
EEG_struct = load(name);
Will put all the fields back together again. To me at least this looks cleaner when done within a function but is probably just my preference
So comment out which ever you prefer. Also, not I did not include clean in the save. You could either append it to the MAT or add it to your structure.
To get a structure the same as EEG_struct but with only the data/labels corresponding with the clean variable, you can simply make a copy of the existing structure and remove the rows where clean=0
function saving(EEG_struct, clean, name)
newstruct = EEG_struct;
newstruct.data(:,:,logical(~clean)) = '';
newstruct.labels(logical(~clean)) = '';
save(name,'newstruct');
I have a code for "for loop"
for i=1:4
statement...
y=sim(net, I);
end
now i need to export the value of y to excel sheet. for that i used..
xlswrite('output_data.xls', y, 'output_data', 'A1')
but my problem is that the ID of excel i.e. "A1" should change according to each iteration... in my case for iteration 1-> A1, iteration-> A2 and so on..
anybody please help me out ..
thanks in advance. for any assistance.. or suggestion..
You can store sim outputs in a vector (y(ii)) and save in the sheet with a single write. This is also more efficient since you perform a single bulk-write instead of many small writes.
Specify the first cell and y will be written starting from there.
last = someNumber;
for i=1:last statement... y(i)=sim(net, I); end
xlswrite('output_data.xls', y', 'output_data', 'A1');
If you prefer specify the range write ['A1:A',num2str(last)] instead of A1.
If you really want to write within the loop try:
for ii=1:last
...
y=sim(net, I);
xlswrite('output_data.xls', y, 'output_data', sprintf('A%d',ii));
end
You can also do for yourself what xlswrite does internally, which is interact using COM. I prefer to do this when I have a frequently used excel template or data file, because it allows for more control (albeit with more lines of code).
Excel = actxserver('Excel.Application');
Workbook = Excel.Workbooks.Open('myExcelFile.xlsx');
MySheet = Excel.ActiveWorkBook.Sheets.Item(1);
set( get(MySheet,'Range','A1:A10'), 'Value', yourValues);
...
invoke(Workbook, 'Save');
invoke(Excel, 'Quit');
delete(Excel);
This would allow you to save new data to new ranges without re-opening excel each time.
Even better would be to define an oncleanup function (as does xlswrite) to prevent lost file locks (especially when you're doing things like exiting out of debug mode):
...
myWorkbook = Excel.Workbooks.Open(filename,0,true);
cleanUp = onCleanup(#()xlsCleanup(Excel, filename));
function xlsCleanup(Excel,filepath)
try
Excel.DisplayAlerts = 0; %// Turn off dialog boxes
[~,n,e] = fileparts(filepath); %// Excel API expects just the filename
fileName = [n,e];
Excel.Workbooks.Item(fileName).Close(false);
end
Excel.Quit;
end
You can put xlswrite after for loop.You just want to do is save you result in a matrix.This function can write a matrix.
also,you can use [] to combine string to change the range.
>> for i=1:4
Range=['A' num2str(i)]
end
Range =
A1
Range =
A2
Range =
A3
Range =
A4
But,this is a bad way.You should open and write Excel file every time.
how to write a program in matlab that reads a certain number of images let's say 20 for example which are saved in a given directory (C:) such that later i can use them. suppose that the images are saved by numbers. later, i am gonna use them.
I'd have the code look something like this. Assuming cell array im holds your images.
Write out:
IMG_DIR = 'C:\';
filename_root = 'image';
IMG_EXT = '.jpg';
NUM_IMAGES = 20;
for i = 1:NUM_IMAGES
imwrite(im{i}, [IMG_DIR filename_root num2str(i) IMG_EXT]);
end
Read in:
for i = 1:NUM_IMAGES
im{i} = imread([IMG_DIR filename_root num2str(i) IMG_EXT]);
end
If you don't know how many there are, you can also use ls command (works differently in Windows vs. Linux).
If you don't know, in advance, which files will be in there, but you know that they have the string in them, 'rawImage' (like 'rawImage001.jpg' etc.) you can do something like
a = dir('c:\temp');
requiredBaseFileName = 'rawImage'; % you want them to contain the substring 'rawImage'
for i = 1:length(a),
fileName = a(i).name;
if(isempty(strfind(fileName,'.jpg')) & isempty(strfind(fileName,'.png')))
continue;
end
if(isempty(strfind(fileName,requiredBaseFileName)))
continue;
end
% do your processing here
end