I have two mat files with identical list of variables.
In file1.mat
*Variables* *Value* *Type*
Time [100X1] double
Force [100x1] double
In file2.mat
*Variables* *Value* *Type*
Time_1 [90X1] double
Force_1 [90x1] double
I would like to vertically concatenate these two files. The suffix _1 added to the file2 changes to _2 or _32 etc.
How can I refer the variables and concatenate them in a loop so I don't have to open the file every time and enter the variable names in vertcat?
You can use two nice properties of the load command for this task. Firstly, load with an output argument creates a structure with field names equal to the variable names, which means that you can load data without having to know ahead of time what the variables were named. Secondly, the fields are assigned in alphabetical order, which means that force will always be the first field, and time the second field.
Combining these properties, you can do the following:
%# get a listing of all save files
fileList = dir('file*');
nFiles = length(fileList);
loadedData = cell(nFiles,2); %# for storing {force,time}
%# loop through files and write results into loadedData
for iFile = 1:nFiles
tmp = load(fileList{iFile});
loadedData(iFile,:) = struct2cell(tmp)';
end
%# catenate
time = cat(1,loadedData(:,2));
force = cat(1,loadedData(:,1));
Note that if your files are called file1...file10, instead of file001...file010, the alphabetical ordering you get from using the dir command may not be ideal. In that case, you may have to extract the number at the end of the file name and re-order the list.
Does the following code snippet help to solve your problem?
Time_1 = [1; 2];
Time_2 = [2; 3];
Time_3 = 4;
All = [];
for i = 1:3
CurTime = eval(horzcat('Time_', num2str(i)));
All = [All; CurTime];
end
Essentially what is happening is I'm looping over the suffixes of Time_1, Time_2, and Time_3. For each iteration, I obtain the pertinent Time_x variable by manually building the name of the variable in a string, and then allocating it to CurTime using the eval function. Then simply perform the desired concatenation using CurTime
Of course, this code is not very efficient as All is growing within the loop. If you know the size of All beforehand, then you can pre-allocate it. If the size is unknown before the fact, you can implement the solution here (or else just pre-allocate it as being arbitrarily large and then cut it down to size once the loop is complete).
Let me know if I've misunderstood the problem you're having and I'll try and come up with something more helpful.
Also, some people think eval is evil. Certainly if your code contains 10 different calls to eval then you're probably doing it wrong.
Related
I have 200 time points. For each time point, there is an image, the size of which is 40*40 double, corresponds to this time point. For example, image 1 corresponds to time point 1; image k corresponds to time point k (k = 1,2,...,200).
The time points are T = 1:200, with the images named as Image_T, thus Image_1, Image_2 etc.
I want to put all these 200 images together. The final size is 40*40*200 double. The final image looks like fMRI image (fmri_szX = 40, fmri_szY = 40 and fmri_szT = 200). How to achieve that?
Thanks!
Dynamic variables
Note that whilst this is possible, it's considered to be bad programming (see for instance here, or this blog by Loren and even the Mathworks in their documentation tell you not to do this). It would be much better to load your images directly into either a 3D array or a cell structure, avoiding dynamic variable names. I just posted this for completeness; if you ever happen to have to use this solution, you should change to a (cell-) array immediately.
The gist of the linked articles as to why eval is such a bad idea, is that MATLAB can no longer predict what the outcome of the operation will be. For instance A=3*(2:4) is recognised by MATLAB to output a double-array. If you eval stuff, MATLAB can no longer do this. MATLAB is an interpreted language, i.e. each line of code is read then ran, without compiling the entire code beforehand. This means that each time MATLAB encounters eval, it has to stop, evaluate the expression, then check the output, store that, and continue. Most of the speed-engines employed by MATLAB (JIT/MAGMA etc) can't work without predicting the outcome of statements, and will therefore shut down during the eval evaluation, rendering your code very slow.
Also there's a security aspect to the usage of eval. Consider the following:
var1 = 1;
var2 = 2;
var3 = 3;
varnames = {'var1', 'var2; disp(''GOTCHA''); %', 'var3'};
accumvar = [];
for k = 1:numel(varnames)
vname = varnames{k};
disp(['Reading from variable named ' vname]); eval(['accumvar(end+1) = ' vname ';']);
end
Now accumvar will contain the desired variable names. But if you don't set accumvar as output, you might as well not use a disp, but e.g. eval('rm -rf ~/*') which would format your entire disk without even telling you it's doing so.
Loop approach
for ii = 200:-1:1
str = sprintf('Image_%d',ii);
A(:, :, :, ii) = eval(str);
end
This creates your matrix. Note that I let the for loop run backwards, so as to initialise A in its largest size.
Semi-vectorised approach
str = strsplit(sprintf('image_%d ',1:200),' '); % Create all your names
str(end) = []; % Delete the last entry (empty)
%Problem: eval cannot handle cells, loop anyway:
for ii = 200:-1:1
A(:, :, :, ii) = eval(str{ii});
end
eval does not support arrays, so you cannot directly plug the cellarray strin.
Dynamic file names
Despite having a similar title as above, this implies having your file names structured, so in the file browser, and not MATLAB. I'm assuming .jpg files here, but you can add every supported image extension. Also, be sure to have all images in a single folder and no additional images with that extension, or you have to modify the dir() call to include only the desired images.
filenames = dir('*.jpg');
for ii = length(filenames):-1:1
A(:,:,:,ii) = imread(filenames{ii});
end
Images are usually read as m*n*3 files, where m*n is your image size in pixels and the 3 stems from the fact that they're read as RGB by imread. Therefore A is now a 4D matrix, structured as m*n*3*T, where the last index corresponds to the time of the image, and the first three are your image in RGB format.
Since you do not specify how you obtain your 40*40 double, I have left the 4D matrix. Could be you read them and then switch to using a uint16 interpretation of RGB, which is a single number, which would result in a m*n*1*T variable, which you can reduce to a 3D variable by calling A = squeeze(A(:,:,1,:));
I am trying to loop through multiple structures at once, extract variables of interest, and combine them into a single cell array. The problem: all of the variables have the same names. I have a working pseudocode--here it is:
Let's say I load i structures in my workspace. Now I want to loop through each structure, and extract time and position data from each structure.
First, I load my structures. Something like...
data_1
data_2
data_3
Then, I create appropriately sized cell arrays.
time{i,:} = zeros(size(structures));
position{i,:} = zeros(size(structures));
Finally, I loop through my structures to extract cell arrays and create a single array.
for i = 1:size(structures)
time_i= data_i.numbers.time;
position_i= data_i.numbers.position;
time {i,:} = time_i;
position{i,:} = position_i;
end
I want to end with a cell array containing a concatenation of all the variables in a single cell structure.
Could you please help convert my pseudo code/ideas into a script, or point me to resources that might help?
Thanks!
You're likely going to be better off loading your data internal to the loop and storing it into a cell or structure rather than trying to deal with iteratively named variables in your workspace. eval is, in nearly all cases, a significant code smell, not least of which because MATLAB's JIT compiler ignores eval statements so you get none of the engine's optimizations. eval statements are also difficult to parse, debug, and maintain.
An example of a stronger approach:
for ii = 1:nfiles
tmp = load(thefilenames{ii}); % Or use output of dir
trialstr = sprintf('trial_%u', ii); % Generate trial string
data.(trialstr).time = tmp.numbers.time;
data.(trialstr).position = tmp.numbers.position;
end
Which leaves you with a final data structure of:
data
trial_n
time
position
Which is far easier to iterate through later.
My final script for anyone interested:
for i = 1:4 %for 4 structures that I am looping through
eval(['time_',num2str(i),'= data_',num2str(i),'.numbers.time;']);
eval(['position_',num2str(i),'= data_',num2str(i),'.numbers.position;']);
%concatenate data into a single cell array here
time{i} = {eval(['time_',num2str(i)])};
position{i} = {eval(['position_',num2str(i)])};
end
...
eval(['time_',num2str(i),'= data_',num2str(i),'.numbers.time;'])
eval(['position_',num2str(i),'= data_',num2str(i),'.numbers.position;'])
...
I have 200 time points. For each time point, there is an image, the size of which is 40*40 double, corresponds to this time point. For example, image 1 corresponds to time point 1; image k corresponds to time point k (k = 1,2,...,200).
The time points are T = 1:200, with the images named as Image_T, thus Image_1, Image_2 etc.
I want to put all these 200 images together. The final size is 40*40*200 double. The final image looks like fMRI image (fmri_szX = 40, fmri_szY = 40 and fmri_szT = 200). How to achieve that?
Thanks!
Dynamic variables
Note that whilst this is possible, it's considered to be bad programming (see for instance here, or this blog by Loren and even the Mathworks in their documentation tell you not to do this). It would be much better to load your images directly into either a 3D array or a cell structure, avoiding dynamic variable names. I just posted this for completeness; if you ever happen to have to use this solution, you should change to a (cell-) array immediately.
The gist of the linked articles as to why eval is such a bad idea, is that MATLAB can no longer predict what the outcome of the operation will be. For instance A=3*(2:4) is recognised by MATLAB to output a double-array. If you eval stuff, MATLAB can no longer do this. MATLAB is an interpreted language, i.e. each line of code is read then ran, without compiling the entire code beforehand. This means that each time MATLAB encounters eval, it has to stop, evaluate the expression, then check the output, store that, and continue. Most of the speed-engines employed by MATLAB (JIT/MAGMA etc) can't work without predicting the outcome of statements, and will therefore shut down during the eval evaluation, rendering your code very slow.
Also there's a security aspect to the usage of eval. Consider the following:
var1 = 1;
var2 = 2;
var3 = 3;
varnames = {'var1', 'var2; disp(''GOTCHA''); %', 'var3'};
accumvar = [];
for k = 1:numel(varnames)
vname = varnames{k};
disp(['Reading from variable named ' vname]); eval(['accumvar(end+1) = ' vname ';']);
end
Now accumvar will contain the desired variable names. But if you don't set accumvar as output, you might as well not use a disp, but e.g. eval('rm -rf ~/*') which would format your entire disk without even telling you it's doing so.
Loop approach
for ii = 200:-1:1
str = sprintf('Image_%d',ii);
A(:, :, :, ii) = eval(str);
end
This creates your matrix. Note that I let the for loop run backwards, so as to initialise A in its largest size.
Semi-vectorised approach
str = strsplit(sprintf('image_%d ',1:200),' '); % Create all your names
str(end) = []; % Delete the last entry (empty)
%Problem: eval cannot handle cells, loop anyway:
for ii = 200:-1:1
A(:, :, :, ii) = eval(str{ii});
end
eval does not support arrays, so you cannot directly plug the cellarray strin.
Dynamic file names
Despite having a similar title as above, this implies having your file names structured, so in the file browser, and not MATLAB. I'm assuming .jpg files here, but you can add every supported image extension. Also, be sure to have all images in a single folder and no additional images with that extension, or you have to modify the dir() call to include only the desired images.
filenames = dir('*.jpg');
for ii = length(filenames):-1:1
A(:,:,:,ii) = imread(filenames{ii});
end
Images are usually read as m*n*3 files, where m*n is your image size in pixels and the 3 stems from the fact that they're read as RGB by imread. Therefore A is now a 4D matrix, structured as m*n*3*T, where the last index corresponds to the time of the image, and the first three are your image in RGB format.
Since you do not specify how you obtain your 40*40 double, I have left the 4D matrix. Could be you read them and then switch to using a uint16 interpretation of RGB, which is a single number, which would result in a m*n*1*T variable, which you can reduce to a 3D variable by calling A = squeeze(A(:,:,1,:));
I have three m files using the same variables and carrying out calculations on these variables. I have made an index m file in which i have declared all the variables and I can share the variables to the remaining m files using the variable names. My problem is that the variable names change too often and then I have to change the variable names in all these files manually. How can I make a Matlab script which can automatically get the variable names and value from the index m file and put these to the remaining m files.
I feel like you just need a little example from where you could go on so here we go:
First calling each value with a different variable name. if you have a lot of values of the same type a array is easier like:
A0=0; A1=6; A2=12 %each one with its own name
B=zeros(16,1); %create an array of 16 numbers
B(1)= 0; %1 is the first element of an array so care for A0
B(2)= 6;
B(8)= 12;
disp(B); % a lot of numbers and you can each address individually
disp(B(8)); %-> 12
you can put all that in your script and try it. Now to the function part. Your function can have input, output, neither or both. If you just want to create data you wont need an input but an output.
save this as myfile1.m:
function output = myfile1()
number=[3;5;6]; %same as number(1)=3;number(2)=5;number(3)=6
%all the names just stay in this function and the variable name is chosen in the script
output = number; %output is what the file will be
end
and this as myfile2.m
function output = myfile2(input)
input=input*2;%double all the numbers
%This will make an error if "input" is not an array with at least 3
%elements
input(3)=input(3)+2; %only input(3) + 2;
output = input;
end
and now try
B=myfile1() %B will become the output of myfile1
C=myfile2(B) %B is the input of myfile2 and C will become the output
save('exp.mat','C')
I hope this will get you started.
I am quite new to data analysis, so if this is a rookie question, I'm sorry, I am learning as I go.
I have just started doing some work in variable star astronomy. I have about 100 files for every night of observation that all contain the same basic information (star coordinates, magnitude, etc.). I am loading all of the files into my workspace as arrays using a for-loop
files = dir('*.out');
for i=1:length(files)
eval(['load ' files(i).name ' -ascii']);
end
I'm only really interested in two columns in each file. Is there a way to extract a column and set it to a vector while this for-loop is running? I'm sure that it's possible, but the actual syntax for it is escaping me.
try using load as a function and save it's output to a variable
files = dir('*.out');
twoCols = {};
for ii=1:length(files)
data = load( files(ii).name, '-ascii' ); % load file into "data"
twoCols{ii} = data(:,1:2); % take only two columns
end
Now variable twoCols holds the two columns of each file in a different cell.
You have to assign the load result to a new variable. Then if lets say your variable is starsInfo you can use
onlyTwoFirst = starsInfo(:,1:2)
That means take all the rows, but only columns 1 and 2.