I have a number of .mat files. I have written a short script where i use file1.mat and analyse the data. The next step in the process would be doing the same thing for file2.mat. One way to do this could be to simply copy my previous code and replace all the "file1.mat" with "file2.mat" and do the same for 3, 4.... However, I feel that there has to be a more elegant solution. The optimal situation would be if i could write a function that takes the filename (preferably not the whole path) as an argument. Is this possible?
I have scurried the net and the closest I get is the "feval" function which works fine if i have .m files but not at all with .mat files.
Is there a way to pass .mat files to a matlab-function?
Let's say that you have the following script, which just loads some .mat file and processes two variables -
load('C:\data\input1.mat'); %// loads x, y into the workspace
z = x + y;
save('C:\data\output1.mat', 'z');
and you want to also process input2.mat, input3.mat etc. The best way is to write a function that wraps up all this work into a neat blob -
function processData(fnameIn, fnameOut)
pathIn = fullfile('C:\data', fnameIn);
pathOut = fullfile('C:\data', fnameOut);
load(pathIn); %// loads x, y into the workspace
z = x + y;
save(pathOut, 'z');
end
Now you can call it like this
processData('input1.mat', 'output1.mat')
processData('input2.mat', 'output2.mat')
etc, or even better
inputNames = {'input1.mat', 'input2.mat' };
outputNames = {'output1.mat', 'output2.mat'};
for i = 1:length(inputNames)
processData(inputNames{i}, outputNames{i});
end
or, if your filenames happen to be structured, you can just do
for i = 1:2
infile = sprintf('input%d.mat', i);
outfile = sprintf('output%d.mat', i);
processData(infile, outfile);
end
Another possible solution would be to write your function so that it doesn't do any file loading or saving at all, but instead it receives some data as input and returns it as output. This is more flexible, because now you can control how you access your data (e.g. maybe you want to load it all into the workspace before processing any of it - you can do that now, whereas before the data loading, processing and saving were all tied together in one function). The processData function would look like this
function dataOut = processData(dataIn)
x = dataIn.x;
y = dataIn.y;
dataOut.z = x + y;
end
and you use it like this if you want to load the files and then process them one at a time,
for i = 1:length(inputNames)
dataIn = load(fullfile('C:\data', inputNames{i}));
dataOut = processData(dataIn);
save(fullfile('C:\data', outputNames{i}), '-struct', 'dataOut');
end
or like this if you wanted to do all the loading, then all the processing, then all the saving -
for i = 1:length(inputNames)
dataIn(i) = load(fullfile('C:\data', inputNames{i}));
end
for i = 1:length(inputNames)
dataOut(i) = processData(dataIn);
end
for i = 1:length(inputNames)
tmp = dataOut(i);
save(fullfile('C:\data', outputNames{i}), '-struct', 'tmp');
end
One big advantage of writing processData in this way is that if you want to test or debug it, it is suddenly much easier. If the file loading/saving is inside the function, and you want to test it, you have to
Create the test data
Save the test data to a file
Run processData with the filename as input
Load the file containing the output data
Check that the output data is correct
Remember to clean up the files containing the test data from wherever you put them
If you separate the data loading/saving from the processing, then your testing procedure becomes
Create the test data
Run processData with the test data as input
Check that the output is correct
Much simpler, and you didn't have to mess around with saving/loading files at any point, and you didn't create any messy files on your hard drive. If you need to do any debugging, the process is also much easier - just create an example that your function will fail on, and then step through the code to see where it actually fails.
Related
I have a ".mat" file supposedly containing a [30720000x4 double] matrix (values from accelerometers). When I try to open this file with "Import data" in Matlab I get the following error:
Error using load
Can't read file F:\vibration_exp_2\GR_UB50n\bearing1\GR_UB50n_1_2.mat.
Error using load
Unknown text on line number 1 of ASCII file
F:\vibration_exp_2\GR_UB50n\bearing1\GR_UB50n_1_2.mat
"MATLAB".
Error in uiimport/runImportdata (line 456)
datastruct = load('-ascii', fileAbsolutePath);
Error in uiimport/gatherFilePreviewData (line 424)
[datastruct, textDelimiter, headerLines]= runImportdata(fileAbsolutePath,
type);
Error in uiimport (line 240)
[ctorPreviewText, ctorHeaderLines, ctorDelim] = ...
The filesize is 921MB which is the same as my other files that do open. I also tried opening the file using python, but no success. Any suggestions? I use MATLAB R2013b .
More info:
How the file was create:
%% acquisition of vibration data
% input:
% sample rate in Hz (max. 51200 Hz, should be used as bearing
% faults are high-frequent)
% time in seconds, stating the duration of the measurement
% (e.g. 600 seconds = 10 minutes)
% filename for the file to be saved
%
% examples:
% data = DAQ(51200, 600, 'NF1_1.mat');
% data = DAQ(51200, 600, 'NF1_2.mat');
function data = DAQ(samplerate,time,filename)
s = daq.createSession('ni'); % Creates the DAQ session
%%% Add the channels as accelerometer channels (meaning IEPE is turned on)
s.addAnalogInputChannel('cDAQ1Mod1','ai0','Accelerometer');
s.addAnalogInputChannel('cDAQ1Mod1','ai1','Accelerometer');
s.addAnalogInputChannel('cDAQ1Mod1','ai2','Accelerometer');
s.addAnalogInputChannel('cDAQ1Mod1','ai3','Accelerometer');
%s.addAnalogInputChannel('cDAQ1Mod2','ai0','Accelerometer');
s.Rate = samplerate;
s.NumberOfScans = samplerate*time;
%%% Defining the Sensitivities in V/g
s.Channels(1).Sensitivity = 0.09478; %31965, top outer
s.Channels(2).Sensitivity = 0.09531; %31966, back outer
s.Channels(3).Sensitivity = 0.09275; %31964, top inner
s.Channels(4).Sensitivity = 0.09363; %31963, back inner
data = s.startForeground(); %Acquiring the data
save(filename, 'data');
More info:
When I open the file using a simple text editor I can see a lot of characters that do not make sense but also the first line:
MATLAB 5.0 MAT-FILE, Platform: PCWIN64, Created on: Thu Apr 30
16:29:07 2015
More info:
The file itself: https://www.dropbox.com/s/r7mavil79j47xa2/GR_UB50n_1_2.mat?dl=0
It is 921MB.
EDIT:
How can I recover my data?
I've tried this, but got memory errors.
I've also tried this, but it did not work.
I fear I can't add many good news to what you know already, but it hasn't been mentioned yet.
The reason the .mat-file can't be load is due to the data beeing corrupted. What makes it 'unrecoverable' is the way it is stored internally. The exact format is specified in the MAT-File Format Documentation. So I decided to manually construct a simple reader to specifically read your .mat file.
It makes sense, that the splitmat.m can't recover anything, as it will basicly split the data into chunks, one stored variable per chunk, however in this case there is only 1 variable stored and thus only one chunk, which happens to be the corrupted one.
In this case, the data is stored as a miCOMPRESSED, which is a normal matlab array compressed using gzip. (Which, as a side note, doesn't seem like a good fit for 'random' vibration data.) This might explain previous comments about the smaller file size then the full data, as the filesize matches exatly with the internally stored value.
I extracted the compressed archive and tried to uncompress it in a variety of ways. Basicly it is a '.gz' without the header, that can be appended manually. Unfortunatly there seems to be a corrupted block near the start of the dataset. I am by no means an expert on gzip, but as far as I know the dictionary (or decryption key) is stored dynamicly which makes all data useless from the point the block is corrupted. If you are really eager, there seems to be a way to recover data even behind the point where data is corrupted, but that method is massively timeconsuming. Also the only way to validate data of those sections is manual inspection, which in your case might proof very difficult.
Below is the code, that I used to extract the .gz-file, so if you want to give it a try, this might get you started. If you manage to decrypt the data, you can read it as described in the MAT-File Format, 13f.
corrupted_file_id = fopen('corrupt.mat','r');
%% some header data
% can be skipped replacing this block with
% fread(id,132);
%header of .mat file
header_text = char(fread(corrupted_file_id,116,'char')');
subsystem_data_offset = fread(corrupted_file_id,8,'uint8');
version = fread(corrupted_file_id,1,'int16');
endian_indicator = char(fread(corrupted_file_id,2,'int8')');
data_type = fread(corrupted_file_id,4,'uint8');
%data_type is 15, so it is a compressed matlab array
%% save te content
data_size = fread(corrupted_file_id,1,'uint32');
gz_file_id = fopen('compressed_array.gz','w');
% first write a valid gzip head
fwrite(gz_file_id,hex2dec('1f8b080000000000'),'uint64',0,'b');
% then write the data sequentialy
step = 1:1e3:data_size;% 1MB steps
for idx = step
fwrite(gz_file_id,fread(corrupted_file_id,1e3,'uint8'));
end
step = step(end):data_size;% 1B steps
for idx = step
fwrite(gz_file_id,fread(corrupted_file_id,1,'uint8'));
end
fclose(gz_file_id);
fclose(corrupted_file_id);
To answer literally to the question, my suggestion would be to make sure first that the file is okay. This tool on File Exchange apparently knows how to diagnose corrupted .MAT files starting with version V5 (R8):
http://www.mathworks.com/matlabcentral/fileexchange/6893-matcat-mat-file-corruption-analysis-tool
The file's size (indices going out of range) seems to be a problem. Octave, which should read .mat files, gives the error
memory exhausted or requested size too large for range of Octave's index type
To find out what is wrong you may need to write a test program outside MatLab, where you have more control over memory management. Examples are here, including instructions on how to build them on your own platform. These stand-alone programs may not have the same memory issues. The program matdgns.c is specifically made to check .mat files for errors.
I am looking for a way to save 'everything' in the matlab session when it is stopped for debugging.
Example
function funmain
a=1;
if a>1
funsub(1)
end
funsub(2)
end
function funsub(c)
b = c + 1;
funsubsub(c)
end
function funsubsub(c)
c = c + 2; %Line with breakpoint
end
When I finally reach the line with the breakpoint, I can easily navigate all workspaces and see where all function calls are made.
The question
How can I preserve this situation?
When debugging nested programs that take a long time to run, I often find myself waiting for a long time to reach a breakpoint. And sometimes I just have to close matlab, or want to try some stuff and later return to this point, so therefore finding a way to store this state would be quite desirable. I work in Windows Server 2008, but would prefer a platform independant solution that does not require installation of any software.
What have I tried
1. Saving all variables in the workspace: This works sometimes, but often I will also need to navigate other workspaces
2. Saving all variables in the calling workspace: This is already better as I can run the lowest function again, but may still be insufficient. Doing this for all nested workspaces is not very convenient, and navigating the saved workspaces may be even worse.
Besides the mentioned inconveniences, this also doesn't allow me to see the exact route via which the breakpoint is reached. Therefore I hope there is a better solution!
Code structure example
The code looks a bit like this
function fmain
fsub1()
fsub2()
fsub3()
end
function fsub1
fsubsub11
fsubsub12
...
fsubsub19
end
function fsub2
fsubsub21
fsubsub22
...
fsubsub29
end
function fsub3
fsubsub31
fsubsub32
...
fsubsub39
end
function fsubsub29
fsubsubsub291
fsubsubsub292% The break may occur in here
...
fsubsubsub299
The break can of course occur anywhere, and normally I would be able to navigate the workspace and all those above it.
Checkpointing
What you're looking to implement is known as checkpointing code. This can be very useful on pieces of code that run for a very long time. Let's take a very simple example:
f=zeros(1e6,1);
for i=1:1e6
f(i) = g(i) + i*2+5; % //do some stuff with f, not important for this example
end
This would obviously take a while on most machines so it would be a pain if it ran half way, and then you had to restart. So let's add a checkpoint!
f=zeros(1e6,1);
i=1; % //start at 1
% //unless there is a previous checkpoint, in which case skip all those iterations
if exist('checkpoint.mat')==2
load('checkpoint.mat'); % //this will load f and i
end
while i<1e6+1
f(i) = g(i) + i*2+5;
i=i+1;
if mod(i,1000)==0 % //let's save our state every 1000 iterations
save('checkpoint.mat','f','i');
end
end
delete('checkpoint.mat') % //make sure to remove it when we're done!
This allows you to quit your code midway through processing without losing all of that computation time. Deciding when and how often to checkpoint is the balance between performance and lost time!
Sample Code Implementation
Your Sample code would need to be updated as follows:
function fmain
sub1done=false; % //These really wouldn't be necessary if each function returns
sub2done=false; % //something, you could just check if the return exists
sub3done=false;
if exist('checkpoint_main.mat')==2, load('checkpoint_main.mat');end
if ~sub1done
fprintf('Entering fsub1\n');
fsub1()
fprintf('Finished with fsub1\n');
sub1done=true;
save('checkpoint_main.mat');
end
if ~sub2done
fprintf('Entering fsub2\n');
fsub2()
fprintf('Finished with fsub2\n');
sub2done=true;
save('checkpoint_main.mat');
end
if ~sub3done
fprintf('Entering fsub3\n');
fsub3()
fprintf('Finished with fsub3\n');
sub3done=true;
save('checkpoint_main.mat');
end
delete('checkpoint_main.mat');
end
function fsub2
subsub21_done=false;subsub22_done=false;...subsub29_done=false;
if exist('checkpoint_fsub2')==2, load('checkpoint_fsub2');end
if ~subsub21_done
fprintf('\tEntering fsubsub21\n');
fsubsub21
fprintf('\tFinished with fsubsub21\n');
subsub21_done=true;
save('checkpoint_fsub2.mat');
end
...
if ~subsub29_done
fprintf('\tEntering fsubsub29\n');
fsubsub29
fprintf('\tFinished with fsubsub29\n');
subsub29_done=true;
save('checkpoint_fsub2.mat');
end
delete('checkpoint_fsub2.mat');
end
function fsubsub29
subsubsub291_done=false;...subsubsub299_done=false;
if exist('checkpoint_fsubsub29.mat')==2,load('checkpoint_fsubsub29.mat');end
if ~subsubsub291_done
fprintf('\t\tEntering fsubsubsub291\n');
fsubsubsub291
fprintf('\t\tFinished with fsubsubsub291\n');
subsubsub291_done=true;
save('checkpoint_fsubsub29.mat');
end
if ~subsubsub292_done
fprintf('\t\tEntering fsubsubsub292\n');
fsubsubsub292% The break may occur in here
fprintf('\t\tFinished with fsubsubsub292\n')
subsubsub292_done=true;
save(checkpoint_fsubsub29.mat');
end
delete('checkpoint_fsubsub29.mat');
end
So in this structure if you restarted the program after it was killed it would resume back to the last saved checkpoint. So for example if the program died in subsubsub291, the program would skip fsub1 altogether, just loading the result. And then it would skip subsub21 all the way down to subsub29 where it would enter subsub29. Then it would skip subsubsub291 and enter 292 where it left off, having loaded all of the variables in that workspace and in previous workspaces. So if you backed out of 292 into 29 you would have the same workspace as if the code just ran. Note that this will also print a nice tree structure as it enters and exits functions to help debug execution order.
Reference:
https://wiki.hpcc.msu.edu/pages/viewpage.action?pageId=14781653
After a bit of googling, I found that using putvar (custom function from here: http://au.mathworks.com/matlabcentral/fileexchange/27106-putvar--uigetvar ) solved this.
I'm doing a project modelling the flow of sauce around pasta, using a BEMLIB code for stokes flow called prtcl_3d_ss_visualize.m which is available from "http://dehesa.freeshell.org/BEMLIB/". Here is the code:
%---
file2 = fopen('prtcl_3d_ss_visualize.m')
Npnt = fscanf(file2,'%f',[1,1])
Nvert = fscanf(file2,'%f',[1,1])
Nface = fscanf(file2,'%f',[1,1])
vert = fscanf(file2,'%f',[3,Nvert]);
wall = fscanf(file2,'%f',[1,1])
fclose(file2)
%---
for i=1:Nvert
save = vert(2,i);
vert(2,i) = vert(3,i);
vert(3,i) = save;
end
Ic=0;
for i=1:Nface
for j=1:Npnt
Ic=Ic+1;
fac(j,i) = Ic;
end
end
patch('faces',Nfac’,'vertices',vert’,...
'FaceColor','y',...
'FaceLighting','phong',...
'BackFaceLighting','lit')
%light('Position',[1 3 2]);
%light('Position',[-3 -1 3]);
%material dull
%material shiny
%axis vis3d off
axis([-1.5 1.5 -1.5 1.5 -0.5 2.5 ])
%view(45,34)
xlabel('x')
ylabel('z')
zlabel('y')
xw(1)=-1.5; zw(1)=-1.5; yw(1)=wall;
xw(2)= 1.5; zw(2)=-1.5; yw(2)=wall;
xw(3)= 1.5; zw(3)= 1.5; yw(3)=wall;
xw(4)=-1.5; zw(4)= 1.5; yw(4)=wall;
xw(5)=-1.5; zw(5)=-1.5; yw(5)=wall;
patch(xw,zw,yw,yw);
hold off
I have encountered several errors when inputing the code in matlab, the first of which i solved changing fopen('prtcl_3d') to fopen('partcl_3d_ss_visualize.m') which is the name of he file.
The second error I came up against was where I input
patch('faces',Nfac’,'vertices',vert’,...
'FaceColor','y',...
'FaceLighting','phong',...
'BackFaceLighting','lit')
Nfac' was orighinally fac'but i got error "??? Undefined function or variable fac" so I changed it to an already defined variable Nfac.
The problems I am now faced with the last part of the code. When I enter the first line
xw(1)=-1.5; zw(1)=-1.5; yw(1)=wall;
I get the error message: "??? In an assignment A(I) = B, the number of elements in B and I must be the same."
I get this for the rest of the xw,zw,yw inputs too, what am I doing wrong?
I solved changing fopen('prtcl_3d') to
fopen('partcl_3d_ss_visualize.m') which is the name of he file.
No, you didn't solve it, you broke it. In the original script, partcl_3d_ss_visualize.m (or the version I found on the internets, at least), we have these lines:
file2 = fopen('prtcl_3d.net')
Npnt = fscanf(file2,'%f',[1,1])
Nvert = fscanf(file2,'%f',[1,1])
Nface = fscanf(file2,'%f',[1,1])
vert = fscanf(file2,'%f',[3,Nvert]);
wall = fscanf(file2,'%f',[1,1])
fclose(file2)
This opens some sort of data file prtcl_3d.net, and loads in some required values (this file is presumably included somewhere in the code you downloaded). What you are making it do is make the script load itself and try and find some non-existent floating point numbers at the start of the file.
Having done this, it is likely that the output of wall is empty. Therefore yw(1) (1 element) and wall (zero elements) have different sizes and when you try to assign one to the other you get an error.
Important Note: It won't error earlier because it was able to open the file and try to read something - it is your job to check if what was read in is correct. In this case, just manually looking at your workspace would have been enough. In other cases, error checking can be done using functions such as isempty, isnumeric, size, and so on depending on what sort of output you're expecting.
Regarding fac/Nfac, I imagine that this script does not stand alone and you are supposed to run another script or function before this one that creates the correct variables.
It says on that website:
BEMLIB is a boundary-element software library of Fortran 77 (compatible with Fortran 90) and Matlab codes accompanying the book by C. Pozrikidis, A Practical Guide to Boundary Element Methods with the software library BEMLIB,'' Champan & Hall/CRC, (2002). Chapters 8-12 of the book contain the BEMLIB User Guide
I suggest you need the BEMLIB User Guide to help you understand what the code does and how to use it, and until you do understand that you shouldn't be making any changes to it.
I'm a Matlab novice and have been struggling with this particular task for weeks now.
I'm trying to create a nested for loop for uploading all of my data into Matlab. I need the code to go into the file for subject 1, go into the file for the first exercise, upload 3 files (EMG, Kinetic, and kinematic data), then go back and enter the file for the second exercise and upload the data in that file, repeat this for all 5 exercises, and then repeat this whole process for all 12 subjects. I have created code for uploading data from just one file using information I've read off the internet, but to create this programme to get all of the data from all these files has proven to be very difficult.
Below is the code I have currently written:
clear all;
Subjects = dir('C:\Users\pricep\Desktop\JuggaData');
Exercise = dir('C:\Users\pricep\Desktop\JuggaData\Subject1');
Trialdata = dir('C:\Users\pricep\Desktop\JuggaData\Subject1\*.xlsx');
subjectnum = numel(Subjects);
exercisenum = numel(Exercise);
datanum = numel(Trialdata);
myData = cell(datanum,1);
for k = 1:subjectnum
for j = 1:exercisenum
for i = 1:datanum
filename = sprintf(Trialdata(i).name);
myData{k} = importdata(filename);
end
end
end
No error message appears, but no data appears either.
As you can tell I'm a complete novice so any help would be greatly appreciated.
Verify that subjectnum, exercisenum and datanum are above zero, probably one of the files is zero causing nothing to happen. Besides this, there are other issues:
exercisenum is constant, which assumes that every subject has the same amount of exercises. Might be wrong.
dir returns the file names .. (parent directory) and . (current directory) as well. You don't filter these.
importdata is called with a relative file path, which is not possible.
myData = cell(datanum,1); allocates not enough space, assuming there is more than one subject and exercise.
How can i do the same math function(eg:average) for a folder containing large number of data files using matlab(2010) and store them in one output?
This is the code I have, but it was unsuccessful.
function A = RepeatForAll()
A = 0;
% audio is the folder name where audio files are saved
path = fullfile(pwd,'audio');
files = dir(path);
for fileIndex=3:length(files)
if (files(fileIndex).isdir == 0)
if (~isempty(strfind(files(fileIndex).name,'wav')))
fullfile(path,files(fileIndex).name)
result = wavread(fullfile(path,files(fileIndex).name));
% Any thing you have to for each audio file
inputz=wavread(result);
outx = mfccx(result);
A(count,:) = mean(outx,2);
count=count+1;
end;
end;
end;
You need to set
count=0;
before going into the for loop, now count is undefined.
Also the line inputz=wavread(result); is both unused and wrong. you don't use inputz anywhere, and you are calling wavread on result, which is not a filename (see the line above). This would generate an error.
Please be more clear next time, at least give us error messages. Describing what you think should happen is also useful.
Last, correctly format your code. In matlab, select al your code (control+a) then press control+i to format it.