I have a ".mat" file supposedly containing a [30720000x4 double] matrix (values from accelerometers). When I try to open this file with "Import data" in Matlab I get the following error:
Error using load
Can't read file F:\vibration_exp_2\GR_UB50n\bearing1\GR_UB50n_1_2.mat.
Error using load
Unknown text on line number 1 of ASCII file
F:\vibration_exp_2\GR_UB50n\bearing1\GR_UB50n_1_2.mat
"MATLAB".
Error in uiimport/runImportdata (line 456)
datastruct = load('-ascii', fileAbsolutePath);
Error in uiimport/gatherFilePreviewData (line 424)
[datastruct, textDelimiter, headerLines]= runImportdata(fileAbsolutePath,
type);
Error in uiimport (line 240)
[ctorPreviewText, ctorHeaderLines, ctorDelim] = ...
The filesize is 921MB which is the same as my other files that do open. I also tried opening the file using python, but no success. Any suggestions? I use MATLAB R2013b .
More info:
How the file was create:
%% acquisition of vibration data
% input:
% sample rate in Hz (max. 51200 Hz, should be used as bearing
% faults are high-frequent)
% time in seconds, stating the duration of the measurement
% (e.g. 600 seconds = 10 minutes)
% filename for the file to be saved
%
% examples:
% data = DAQ(51200, 600, 'NF1_1.mat');
% data = DAQ(51200, 600, 'NF1_2.mat');
function data = DAQ(samplerate,time,filename)
s = daq.createSession('ni'); % Creates the DAQ session
%%% Add the channels as accelerometer channels (meaning IEPE is turned on)
s.addAnalogInputChannel('cDAQ1Mod1','ai0','Accelerometer');
s.addAnalogInputChannel('cDAQ1Mod1','ai1','Accelerometer');
s.addAnalogInputChannel('cDAQ1Mod1','ai2','Accelerometer');
s.addAnalogInputChannel('cDAQ1Mod1','ai3','Accelerometer');
%s.addAnalogInputChannel('cDAQ1Mod2','ai0','Accelerometer');
s.Rate = samplerate;
s.NumberOfScans = samplerate*time;
%%% Defining the Sensitivities in V/g
s.Channels(1).Sensitivity = 0.09478; %31965, top outer
s.Channels(2).Sensitivity = 0.09531; %31966, back outer
s.Channels(3).Sensitivity = 0.09275; %31964, top inner
s.Channels(4).Sensitivity = 0.09363; %31963, back inner
data = s.startForeground(); %Acquiring the data
save(filename, 'data');
More info:
When I open the file using a simple text editor I can see a lot of characters that do not make sense but also the first line:
MATLAB 5.0 MAT-FILE, Platform: PCWIN64, Created on: Thu Apr 30
16:29:07 2015
More info:
The file itself: https://www.dropbox.com/s/r7mavil79j47xa2/GR_UB50n_1_2.mat?dl=0
It is 921MB.
EDIT:
How can I recover my data?
I've tried this, but got memory errors.
I've also tried this, but it did not work.
I fear I can't add many good news to what you know already, but it hasn't been mentioned yet.
The reason the .mat-file can't be load is due to the data beeing corrupted. What makes it 'unrecoverable' is the way it is stored internally. The exact format is specified in the MAT-File Format Documentation. So I decided to manually construct a simple reader to specifically read your .mat file.
It makes sense, that the splitmat.m can't recover anything, as it will basicly split the data into chunks, one stored variable per chunk, however in this case there is only 1 variable stored and thus only one chunk, which happens to be the corrupted one.
In this case, the data is stored as a miCOMPRESSED, which is a normal matlab array compressed using gzip. (Which, as a side note, doesn't seem like a good fit for 'random' vibration data.) This might explain previous comments about the smaller file size then the full data, as the filesize matches exatly with the internally stored value.
I extracted the compressed archive and tried to uncompress it in a variety of ways. Basicly it is a '.gz' without the header, that can be appended manually. Unfortunatly there seems to be a corrupted block near the start of the dataset. I am by no means an expert on gzip, but as far as I know the dictionary (or decryption key) is stored dynamicly which makes all data useless from the point the block is corrupted. If you are really eager, there seems to be a way to recover data even behind the point where data is corrupted, but that method is massively timeconsuming. Also the only way to validate data of those sections is manual inspection, which in your case might proof very difficult.
Below is the code, that I used to extract the .gz-file, so if you want to give it a try, this might get you started. If you manage to decrypt the data, you can read it as described in the MAT-File Format, 13f.
corrupted_file_id = fopen('corrupt.mat','r');
%% some header data
% can be skipped replacing this block with
% fread(id,132);
%header of .mat file
header_text = char(fread(corrupted_file_id,116,'char')');
subsystem_data_offset = fread(corrupted_file_id,8,'uint8');
version = fread(corrupted_file_id,1,'int16');
endian_indicator = char(fread(corrupted_file_id,2,'int8')');
data_type = fread(corrupted_file_id,4,'uint8');
%data_type is 15, so it is a compressed matlab array
%% save te content
data_size = fread(corrupted_file_id,1,'uint32');
gz_file_id = fopen('compressed_array.gz','w');
% first write a valid gzip head
fwrite(gz_file_id,hex2dec('1f8b080000000000'),'uint64',0,'b');
% then write the data sequentialy
step = 1:1e3:data_size;% 1MB steps
for idx = step
fwrite(gz_file_id,fread(corrupted_file_id,1e3,'uint8'));
end
step = step(end):data_size;% 1B steps
for idx = step
fwrite(gz_file_id,fread(corrupted_file_id,1,'uint8'));
end
fclose(gz_file_id);
fclose(corrupted_file_id);
To answer literally to the question, my suggestion would be to make sure first that the file is okay. This tool on File Exchange apparently knows how to diagnose corrupted .MAT files starting with version V5 (R8):
http://www.mathworks.com/matlabcentral/fileexchange/6893-matcat-mat-file-corruption-analysis-tool
The file's size (indices going out of range) seems to be a problem. Octave, which should read .mat files, gives the error
memory exhausted or requested size too large for range of Octave's index type
To find out what is wrong you may need to write a test program outside MatLab, where you have more control over memory management. Examples are here, including instructions on how to build them on your own platform. These stand-alone programs may not have the same memory issues. The program matdgns.c is specifically made to check .mat files for errors.
Related
this is my first topic if anything wrong please advise me.
I am not a great matlab user but I need to use matlab with sensoray 826 to capture the data.
However, once I install the driver and try to run 826 Matlab demo I got the error code below.
Warning: Warnings messages were produced while parsing. Check the functions you intend to use for correctness. Warning text can be viewed using:
[notfound,warnings]=loadlibrary(...)
I try to find the root cause of this problem and get new error.
[notfound, warnings] = loadlibrary('C:\Program Files (x86)\Sensoray\826\API\x64\s826.dll')
Error using loadlibrary
There was an error loading the library "C:\Program Files (x86)\Sensoray\826\API\x64\s826.dll"
Unrecognized function or variable 's826_proto'.
Caused by:
Error using feval
Unrecognized function or variable 's826_proto'.
At this point I believe that the error is cause from incorrect DLL file from the sensoray.
I try to contact the Sensoray but they rarely answer back to me.
If anyone know how to solve this issue please kindly reply back to me.
The code I use to run in Matlab:
% Load the 826 API and open a connection to the 826 board
dllPath = 'C:\Program Files (x86)\Sensoray\826\API\x64\s826.dll'; % Replace with the correct path to the DLL file
hdrPath = 'C:\Program Files (x86)\Sensoray\826\API\826api.h'; % Replace with the correct path to the header file
[errcode, boardflags] = s826.SystemOpen(dllPath, hdrPath);
if errcode ~= 0
error('Error opening connection to 826 board: %d', errcode);
end
% Configure the rotary sensor
counter = 0; % Choose the counter that the rotary sensor is connected to (0-3)
s826.CounterModeWrite(counter, s826.CNTR_QUADX4); % Configure the counter for quadrature input
s826.CounterPreloadWrite(counter, 0, 0); % Reset the counter
% Read the counter value
count = s826.CounterRead(counter);
% Close the connection to the 826 board
s826.SystemClose();
I would like to go step by step since I never use this 826 cards before.
So first just correctly call the DLL and H file is enough for me.
I have a weird problem... I want to create a video with the extension .tif with a lot of frames. My script is running well 2 times over 3 but it crashes randomly sometimes...
I have a loop which has the length of the total of frames in the video and at each turn I add a tif to my multipage tif.
There is my code to create the new video :
% --- Create the new frame
newVid.cData = iL(y0:y0end, x0:x0end);
% --- Create the new video
if nbrFrames == 1
imwrite(newVid.cData,dataOutVid);
else
imwrite(newVid.cData,dataOutVid,'WriteMode','append');
end
Each turn I change the value of "newVid.cData". in fact the new video is a portion of an original video in which I focused on a specific object (a mouse for me). "dataOutVid" is the path where I store the new video and the extension of the path is .tif.
How I obtain the path:
disp('Where do you want to save the new video and under which name ?');
[name, path] = uiputfile({'.tif'}, 'Save Video');
dataOutVid = strcat(path,name);
There is the mistake I can possibly have randomly:
Error using imwrite (line 454)
Unable to open file "D:\Matlab\Traitement Vidéo\test.tif" for writing. You might not have write permission.
Error in mouseExtraction(line 164)
imwrite(newVid.cData,dataOutVid,'WriteMode','append');
Well I don't understand why this error appears randomly (one time at the frame 270, another time at the frame 1250, etc...). How is it possible I suddendly loose the right to overwrite my file...
Edit : I already checked if I didn't have a RAM problem but I only use 20% of it during the execution of the script...
I have a number of .mat files. I have written a short script where i use file1.mat and analyse the data. The next step in the process would be doing the same thing for file2.mat. One way to do this could be to simply copy my previous code and replace all the "file1.mat" with "file2.mat" and do the same for 3, 4.... However, I feel that there has to be a more elegant solution. The optimal situation would be if i could write a function that takes the filename (preferably not the whole path) as an argument. Is this possible?
I have scurried the net and the closest I get is the "feval" function which works fine if i have .m files but not at all with .mat files.
Is there a way to pass .mat files to a matlab-function?
Let's say that you have the following script, which just loads some .mat file and processes two variables -
load('C:\data\input1.mat'); %// loads x, y into the workspace
z = x + y;
save('C:\data\output1.mat', 'z');
and you want to also process input2.mat, input3.mat etc. The best way is to write a function that wraps up all this work into a neat blob -
function processData(fnameIn, fnameOut)
pathIn = fullfile('C:\data', fnameIn);
pathOut = fullfile('C:\data', fnameOut);
load(pathIn); %// loads x, y into the workspace
z = x + y;
save(pathOut, 'z');
end
Now you can call it like this
processData('input1.mat', 'output1.mat')
processData('input2.mat', 'output2.mat')
etc, or even better
inputNames = {'input1.mat', 'input2.mat' };
outputNames = {'output1.mat', 'output2.mat'};
for i = 1:length(inputNames)
processData(inputNames{i}, outputNames{i});
end
or, if your filenames happen to be structured, you can just do
for i = 1:2
infile = sprintf('input%d.mat', i);
outfile = sprintf('output%d.mat', i);
processData(infile, outfile);
end
Another possible solution would be to write your function so that it doesn't do any file loading or saving at all, but instead it receives some data as input and returns it as output. This is more flexible, because now you can control how you access your data (e.g. maybe you want to load it all into the workspace before processing any of it - you can do that now, whereas before the data loading, processing and saving were all tied together in one function). The processData function would look like this
function dataOut = processData(dataIn)
x = dataIn.x;
y = dataIn.y;
dataOut.z = x + y;
end
and you use it like this if you want to load the files and then process them one at a time,
for i = 1:length(inputNames)
dataIn = load(fullfile('C:\data', inputNames{i}));
dataOut = processData(dataIn);
save(fullfile('C:\data', outputNames{i}), '-struct', 'dataOut');
end
or like this if you wanted to do all the loading, then all the processing, then all the saving -
for i = 1:length(inputNames)
dataIn(i) = load(fullfile('C:\data', inputNames{i}));
end
for i = 1:length(inputNames)
dataOut(i) = processData(dataIn);
end
for i = 1:length(inputNames)
tmp = dataOut(i);
save(fullfile('C:\data', outputNames{i}), '-struct', 'tmp');
end
One big advantage of writing processData in this way is that if you want to test or debug it, it is suddenly much easier. If the file loading/saving is inside the function, and you want to test it, you have to
Create the test data
Save the test data to a file
Run processData with the filename as input
Load the file containing the output data
Check that the output data is correct
Remember to clean up the files containing the test data from wherever you put them
If you separate the data loading/saving from the processing, then your testing procedure becomes
Create the test data
Run processData with the test data as input
Check that the output is correct
Much simpler, and you didn't have to mess around with saving/loading files at any point, and you didn't create any messy files on your hard drive. If you need to do any debugging, the process is also much easier - just create an example that your function will fail on, and then step through the code to see where it actually fails.
I'm a Matlab novice and have been struggling with this particular task for weeks now.
I'm trying to create a nested for loop for uploading all of my data into Matlab. I need the code to go into the file for subject 1, go into the file for the first exercise, upload 3 files (EMG, Kinetic, and kinematic data), then go back and enter the file for the second exercise and upload the data in that file, repeat this for all 5 exercises, and then repeat this whole process for all 12 subjects. I have created code for uploading data from just one file using information I've read off the internet, but to create this programme to get all of the data from all these files has proven to be very difficult.
Below is the code I have currently written:
clear all;
Subjects = dir('C:\Users\pricep\Desktop\JuggaData');
Exercise = dir('C:\Users\pricep\Desktop\JuggaData\Subject1');
Trialdata = dir('C:\Users\pricep\Desktop\JuggaData\Subject1\*.xlsx');
subjectnum = numel(Subjects);
exercisenum = numel(Exercise);
datanum = numel(Trialdata);
myData = cell(datanum,1);
for k = 1:subjectnum
for j = 1:exercisenum
for i = 1:datanum
filename = sprintf(Trialdata(i).name);
myData{k} = importdata(filename);
end
end
end
No error message appears, but no data appears either.
As you can tell I'm a complete novice so any help would be greatly appreciated.
Verify that subjectnum, exercisenum and datanum are above zero, probably one of the files is zero causing nothing to happen. Besides this, there are other issues:
exercisenum is constant, which assumes that every subject has the same amount of exercises. Might be wrong.
dir returns the file names .. (parent directory) and . (current directory) as well. You don't filter these.
importdata is called with a relative file path, which is not possible.
myData = cell(datanum,1); allocates not enough space, assuming there is more than one subject and exercise.
How can i do the same math function(eg:average) for a folder containing large number of data files using matlab(2010) and store them in one output?
This is the code I have, but it was unsuccessful.
function A = RepeatForAll()
A = 0;
% audio is the folder name where audio files are saved
path = fullfile(pwd,'audio');
files = dir(path);
for fileIndex=3:length(files)
if (files(fileIndex).isdir == 0)
if (~isempty(strfind(files(fileIndex).name,'wav')))
fullfile(path,files(fileIndex).name)
result = wavread(fullfile(path,files(fileIndex).name));
% Any thing you have to for each audio file
inputz=wavread(result);
outx = mfccx(result);
A(count,:) = mean(outx,2);
count=count+1;
end;
end;
end;
You need to set
count=0;
before going into the for loop, now count is undefined.
Also the line inputz=wavread(result); is both unused and wrong. you don't use inputz anywhere, and you are calling wavread on result, which is not a filename (see the line above). This would generate an error.
Please be more clear next time, at least give us error messages. Describing what you think should happen is also useful.
Last, correctly format your code. In matlab, select al your code (control+a) then press control+i to format it.