I'm a Matlab novice and have been struggling with this particular task for weeks now.
I'm trying to create a nested for loop for uploading all of my data into Matlab. I need the code to go into the file for subject 1, go into the file for the first exercise, upload 3 files (EMG, Kinetic, and kinematic data), then go back and enter the file for the second exercise and upload the data in that file, repeat this for all 5 exercises, and then repeat this whole process for all 12 subjects. I have created code for uploading data from just one file using information I've read off the internet, but to create this programme to get all of the data from all these files has proven to be very difficult.
Below is the code I have currently written:
clear all;
Subjects = dir('C:\Users\pricep\Desktop\JuggaData');
Exercise = dir('C:\Users\pricep\Desktop\JuggaData\Subject1');
Trialdata = dir('C:\Users\pricep\Desktop\JuggaData\Subject1\*.xlsx');
subjectnum = numel(Subjects);
exercisenum = numel(Exercise);
datanum = numel(Trialdata);
myData = cell(datanum,1);
for k = 1:subjectnum
for j = 1:exercisenum
for i = 1:datanum
filename = sprintf(Trialdata(i).name);
myData{k} = importdata(filename);
end
end
end
No error message appears, but no data appears either.
As you can tell I'm a complete novice so any help would be greatly appreciated.
Verify that subjectnum, exercisenum and datanum are above zero, probably one of the files is zero causing nothing to happen. Besides this, there are other issues:
exercisenum is constant, which assumes that every subject has the same amount of exercises. Might be wrong.
dir returns the file names .. (parent directory) and . (current directory) as well. You don't filter these.
importdata is called with a relative file path, which is not possible.
myData = cell(datanum,1); allocates not enough space, assuming there is more than one subject and exercise.
Related
I have a ".mat" file supposedly containing a [30720000x4 double] matrix (values from accelerometers). When I try to open this file with "Import data" in Matlab I get the following error:
Error using load
Can't read file F:\vibration_exp_2\GR_UB50n\bearing1\GR_UB50n_1_2.mat.
Error using load
Unknown text on line number 1 of ASCII file
F:\vibration_exp_2\GR_UB50n\bearing1\GR_UB50n_1_2.mat
"MATLAB".
Error in uiimport/runImportdata (line 456)
datastruct = load('-ascii', fileAbsolutePath);
Error in uiimport/gatherFilePreviewData (line 424)
[datastruct, textDelimiter, headerLines]= runImportdata(fileAbsolutePath,
type);
Error in uiimport (line 240)
[ctorPreviewText, ctorHeaderLines, ctorDelim] = ...
The filesize is 921MB which is the same as my other files that do open. I also tried opening the file using python, but no success. Any suggestions? I use MATLAB R2013b .
More info:
How the file was create:
%% acquisition of vibration data
% input:
% sample rate in Hz (max. 51200 Hz, should be used as bearing
% faults are high-frequent)
% time in seconds, stating the duration of the measurement
% (e.g. 600 seconds = 10 minutes)
% filename for the file to be saved
%
% examples:
% data = DAQ(51200, 600, 'NF1_1.mat');
% data = DAQ(51200, 600, 'NF1_2.mat');
function data = DAQ(samplerate,time,filename)
s = daq.createSession('ni'); % Creates the DAQ session
%%% Add the channels as accelerometer channels (meaning IEPE is turned on)
s.addAnalogInputChannel('cDAQ1Mod1','ai0','Accelerometer');
s.addAnalogInputChannel('cDAQ1Mod1','ai1','Accelerometer');
s.addAnalogInputChannel('cDAQ1Mod1','ai2','Accelerometer');
s.addAnalogInputChannel('cDAQ1Mod1','ai3','Accelerometer');
%s.addAnalogInputChannel('cDAQ1Mod2','ai0','Accelerometer');
s.Rate = samplerate;
s.NumberOfScans = samplerate*time;
%%% Defining the Sensitivities in V/g
s.Channels(1).Sensitivity = 0.09478; %31965, top outer
s.Channels(2).Sensitivity = 0.09531; %31966, back outer
s.Channels(3).Sensitivity = 0.09275; %31964, top inner
s.Channels(4).Sensitivity = 0.09363; %31963, back inner
data = s.startForeground(); %Acquiring the data
save(filename, 'data');
More info:
When I open the file using a simple text editor I can see a lot of characters that do not make sense but also the first line:
MATLAB 5.0 MAT-FILE, Platform: PCWIN64, Created on: Thu Apr 30
16:29:07 2015
More info:
The file itself: https://www.dropbox.com/s/r7mavil79j47xa2/GR_UB50n_1_2.mat?dl=0
It is 921MB.
EDIT:
How can I recover my data?
I've tried this, but got memory errors.
I've also tried this, but it did not work.
I fear I can't add many good news to what you know already, but it hasn't been mentioned yet.
The reason the .mat-file can't be load is due to the data beeing corrupted. What makes it 'unrecoverable' is the way it is stored internally. The exact format is specified in the MAT-File Format Documentation. So I decided to manually construct a simple reader to specifically read your .mat file.
It makes sense, that the splitmat.m can't recover anything, as it will basicly split the data into chunks, one stored variable per chunk, however in this case there is only 1 variable stored and thus only one chunk, which happens to be the corrupted one.
In this case, the data is stored as a miCOMPRESSED, which is a normal matlab array compressed using gzip. (Which, as a side note, doesn't seem like a good fit for 'random' vibration data.) This might explain previous comments about the smaller file size then the full data, as the filesize matches exatly with the internally stored value.
I extracted the compressed archive and tried to uncompress it in a variety of ways. Basicly it is a '.gz' without the header, that can be appended manually. Unfortunatly there seems to be a corrupted block near the start of the dataset. I am by no means an expert on gzip, but as far as I know the dictionary (or decryption key) is stored dynamicly which makes all data useless from the point the block is corrupted. If you are really eager, there seems to be a way to recover data even behind the point where data is corrupted, but that method is massively timeconsuming. Also the only way to validate data of those sections is manual inspection, which in your case might proof very difficult.
Below is the code, that I used to extract the .gz-file, so if you want to give it a try, this might get you started. If you manage to decrypt the data, you can read it as described in the MAT-File Format, 13f.
corrupted_file_id = fopen('corrupt.mat','r');
%% some header data
% can be skipped replacing this block with
% fread(id,132);
%header of .mat file
header_text = char(fread(corrupted_file_id,116,'char')');
subsystem_data_offset = fread(corrupted_file_id,8,'uint8');
version = fread(corrupted_file_id,1,'int16');
endian_indicator = char(fread(corrupted_file_id,2,'int8')');
data_type = fread(corrupted_file_id,4,'uint8');
%data_type is 15, so it is a compressed matlab array
%% save te content
data_size = fread(corrupted_file_id,1,'uint32');
gz_file_id = fopen('compressed_array.gz','w');
% first write a valid gzip head
fwrite(gz_file_id,hex2dec('1f8b080000000000'),'uint64',0,'b');
% then write the data sequentialy
step = 1:1e3:data_size;% 1MB steps
for idx = step
fwrite(gz_file_id,fread(corrupted_file_id,1e3,'uint8'));
end
step = step(end):data_size;% 1B steps
for idx = step
fwrite(gz_file_id,fread(corrupted_file_id,1,'uint8'));
end
fclose(gz_file_id);
fclose(corrupted_file_id);
To answer literally to the question, my suggestion would be to make sure first that the file is okay. This tool on File Exchange apparently knows how to diagnose corrupted .MAT files starting with version V5 (R8):
http://www.mathworks.com/matlabcentral/fileexchange/6893-matcat-mat-file-corruption-analysis-tool
The file's size (indices going out of range) seems to be a problem. Octave, which should read .mat files, gives the error
memory exhausted or requested size too large for range of Octave's index type
To find out what is wrong you may need to write a test program outside MatLab, where you have more control over memory management. Examples are here, including instructions on how to build them on your own platform. These stand-alone programs may not have the same memory issues. The program matdgns.c is specifically made to check .mat files for errors.
I have a number of .mat files. I have written a short script where i use file1.mat and analyse the data. The next step in the process would be doing the same thing for file2.mat. One way to do this could be to simply copy my previous code and replace all the "file1.mat" with "file2.mat" and do the same for 3, 4.... However, I feel that there has to be a more elegant solution. The optimal situation would be if i could write a function that takes the filename (preferably not the whole path) as an argument. Is this possible?
I have scurried the net and the closest I get is the "feval" function which works fine if i have .m files but not at all with .mat files.
Is there a way to pass .mat files to a matlab-function?
Let's say that you have the following script, which just loads some .mat file and processes two variables -
load('C:\data\input1.mat'); %// loads x, y into the workspace
z = x + y;
save('C:\data\output1.mat', 'z');
and you want to also process input2.mat, input3.mat etc. The best way is to write a function that wraps up all this work into a neat blob -
function processData(fnameIn, fnameOut)
pathIn = fullfile('C:\data', fnameIn);
pathOut = fullfile('C:\data', fnameOut);
load(pathIn); %// loads x, y into the workspace
z = x + y;
save(pathOut, 'z');
end
Now you can call it like this
processData('input1.mat', 'output1.mat')
processData('input2.mat', 'output2.mat')
etc, or even better
inputNames = {'input1.mat', 'input2.mat' };
outputNames = {'output1.mat', 'output2.mat'};
for i = 1:length(inputNames)
processData(inputNames{i}, outputNames{i});
end
or, if your filenames happen to be structured, you can just do
for i = 1:2
infile = sprintf('input%d.mat', i);
outfile = sprintf('output%d.mat', i);
processData(infile, outfile);
end
Another possible solution would be to write your function so that it doesn't do any file loading or saving at all, but instead it receives some data as input and returns it as output. This is more flexible, because now you can control how you access your data (e.g. maybe you want to load it all into the workspace before processing any of it - you can do that now, whereas before the data loading, processing and saving were all tied together in one function). The processData function would look like this
function dataOut = processData(dataIn)
x = dataIn.x;
y = dataIn.y;
dataOut.z = x + y;
end
and you use it like this if you want to load the files and then process them one at a time,
for i = 1:length(inputNames)
dataIn = load(fullfile('C:\data', inputNames{i}));
dataOut = processData(dataIn);
save(fullfile('C:\data', outputNames{i}), '-struct', 'dataOut');
end
or like this if you wanted to do all the loading, then all the processing, then all the saving -
for i = 1:length(inputNames)
dataIn(i) = load(fullfile('C:\data', inputNames{i}));
end
for i = 1:length(inputNames)
dataOut(i) = processData(dataIn);
end
for i = 1:length(inputNames)
tmp = dataOut(i);
save(fullfile('C:\data', outputNames{i}), '-struct', 'tmp');
end
One big advantage of writing processData in this way is that if you want to test or debug it, it is suddenly much easier. If the file loading/saving is inside the function, and you want to test it, you have to
Create the test data
Save the test data to a file
Run processData with the filename as input
Load the file containing the output data
Check that the output data is correct
Remember to clean up the files containing the test data from wherever you put them
If you separate the data loading/saving from the processing, then your testing procedure becomes
Create the test data
Run processData with the test data as input
Check that the output is correct
Much simpler, and you didn't have to mess around with saving/loading files at any point, and you didn't create any messy files on your hard drive. If you need to do any debugging, the process is also much easier - just create an example that your function will fail on, and then step through the code to see where it actually fails.
How can i do the same math function(eg:average) for a folder containing large number of data files using matlab(2010) and store them in one output?
This is the code I have, but it was unsuccessful.
function A = RepeatForAll()
A = 0;
% audio is the folder name where audio files are saved
path = fullfile(pwd,'audio');
files = dir(path);
for fileIndex=3:length(files)
if (files(fileIndex).isdir == 0)
if (~isempty(strfind(files(fileIndex).name,'wav')))
fullfile(path,files(fileIndex).name)
result = wavread(fullfile(path,files(fileIndex).name));
% Any thing you have to for each audio file
inputz=wavread(result);
outx = mfccx(result);
A(count,:) = mean(outx,2);
count=count+1;
end;
end;
end;
You need to set
count=0;
before going into the for loop, now count is undefined.
Also the line inputz=wavread(result); is both unused and wrong. you don't use inputz anywhere, and you are calling wavread on result, which is not a filename (see the line above). This would generate an error.
Please be more clear next time, at least give us error messages. Describing what you think should happen is also useful.
Last, correctly format your code. In matlab, select al your code (control+a) then press control+i to format it.
Up until now, this block of code has been using to build documents with text for several months with no snags. I am now trying to dynamically add images. I've spent about two days staring at code and researching and am at an end. I suspect the issue is that relationships are not being created (more details below.) Maybe not?
//set stuff up...
WordprocessingDocument doc = WordprocessingDocument.Open(fsPat, true, new OpenSettings(){
AutoSave = true,
MarkupCompatibilityProcessSettings = new MarkupCompatibilityProcessSettings(MarkupCompatibilityProcessMode.ProcessAllParts,
DocumentFormat.OpenXml.FileFormatVersions.Office2007),
MaxCharactersInPart = long.MaxValue
});
MainDocumentPart mainPart = doc.MainDocumentPart;
.
.Other stuff goes here
.
//now the fun...
Run r2 = new Run();
// Add an ImagePart.
ImagePart ip = mainPart.AddImagePart(ImagePartType.Png);
string imageRelationshipID = mainPart.CreateRelationshipToPart(ip); //
using (Stream imgStream = ip.GetStream())
{
System.Drawing.Bitmap b = new System.Drawing.Bitmap("myfile.png");
b.Save(imgStream, System.Drawing.Imaging.ImageFormat.Png);
}
Drawing drawing = BuildImage(imageRelationshipID, "name"+imageRelationshipID.ToString(), 17, 17);
r2.Append(drawing);
p.Append(r2);
The image part is essentially copied from http://blog.stuartwhiteford.com/?p=33) and is running in a loop presently. I also copied his BuildImage() function and use it as-is.
When I open the resulting docx, I see red Xs where the images are saying "This image cannot currently be displayed."
When I open the zip, the images will appear in root/media, but not root/word/media as I'd expect. I also cannot find the images referenced in any of the relationship files. Ideally they'd be in root/word/_rels/document.xml.rels. You'll notice I changed how imageRelationshipID is set hoping to fix this. It didn't.
Please help. Thank you.
So... It seems like OpenXML just hates me. I copied AddImagePart code from like 3-4 places among trying other things--none of which lasted long--and just could not get relationships to form. The implication I see is that they happen automatically with the AddImagePart function.
I ended up doing a complete workaround where I add all the pictures I might want to put and remove the Drawing nodes' parents of the ones I didn't want (Run nodes, generally.) Since these are very small pictures, it's feasible and in ways more elegant than trying to add them as necessary since I don't have to keep track of where images are stored on disk.
I am having a problem when creating an avi file using Matlab. My aim is to use an edge filter on an entire video and save the file as an avi. The filter works fine, my problem is the writing of the avi file.
My code:
vidFile = VideoReader('video.avi');
edgeMov = avifile('edges','fps',30);
for i = 1:vidFile.numberofframes
frameI = read(vidFile,i);
frameIgray = rgb2gray(frameI);
edgeI = edge(frameIgray,'canny',0.6);
edgeIuint8 = im2uint8(edgeI);
edgeIuint8(:,:,2) = edgeIuint8(:,:,1);
edgeIuint8(:,:,3) = edgeIuint8(:,:,1);
edgeMov = addframe(edgeMov,edgeIuint8);
end
edgeMov = close(edgeMov)
When the loop finishes and the avifile is closed, I go to play the video and it says "Windows Media Player encountered a problem while playing this file". I've also tried, without success, Media Player Classic and VLC which lead me to believe that the problem must be the file itself. Using GSpot I checked the file and it said that the AVI header is corrupt.
Retrying the loop again returned exactly the same problem. What's confusing me is when I run the loop for a smaller number of frames, 30 for example, the video writes fine and I can watch it. The size of the video I am trying to convert is in excess of 1000 frames so I don't know if size is a problem?
Any help would be greatly appreciated, thank you.
I've used the following to create AVI
edgeMov = avifile('video.avi','compression','Indeo5','fps',15,'quality',95);
Give it a try.