issues of saving a large scale matrix to mat file - matlab

I was trying to save a matrix into a mat file, but the Matlab returns the following messages:
Warning: Variable 'listmatrix' cannot be saved to a MAT-file whose version is older than 7.3.
To save this variable, use the -v7.3 switch.
Skipping...
What does it mean for "use the -v7.3 switch"?
Should I use
save testresult.mat -v7.3 listmatrix
or sth else?

Hi i thought I’d reply to this thread as I’ve been trying to figure out how to save a large (>2 GB) .mat file in matlab v7 (v7.1.0.183) (R14) and finally found a solution.
If you try to use the save command you will get the following error:
save('test.mat', 'data');
Warning: Variable 'data' cannot be saved to a MAT-file because its
storage requirements exceed 2^31 bytes. This limitation will be
addressed in a future release. Consider storing this variable in HDF5
file format (see HDF5WRITE). Skipping...
The solution is to write a HDF5 file instead:
hdf5write('test.hdf5', '/dataset1', data);
You can then read the data back into matlab using:
hdf5read('test.hdf5', '/dataset1');

A quick google search says yes. Try
save -v7.3 testresult.mat listmatrix
How big is your object? (Do whos listmatrix)
You could potentially save memory by using different data type such as uint8.
http://www.mathworks.ch/matlabcentral/newsreader/view_thread/243327
http://www.mathworks.de/matlabcentral/newsreader/view_thread/307845

Related

unable to save a workspace variable in matlab

while trying to save a workspace variable it throws an error in matlab saying I need to switch to -v7.3. I am not sure how to resolve this error since the earlier versions dont support readtable function or xlsread function for an excel file.
You simply need to use one of these:
save('file.m','var1','var2',-v7.3);
save file.m var1 var2 -v7.3
Matlab will save data by default to an older format of .mat files. This format does not support variables larger than 2^31 bytes but can be read by old releases of Matlab (pre-R2006b). If you have large variables, you need to switch to format 7.3.
All of this is in the documentation from the save function. Type doc save and read.

how to open large .mat file that wasn't saved with -v7.3 flag?

I am trying to open a .mat file with one huge array in it.
When I use matfile function like below:
file=matfile('input_file');
Following errors show up:
Warning: The file 'C:...\input_file.mat' was saved in a
format that does not support partial loading. Temporarily loading variable 'var_name' into memory. To use partial loading
efficiently, save MAT-files with the -v7.3 flag.
Error using my_function (line 11)
Cannot read file C:...\input_file.mat.
How can I "resave" it properly? Or is there any other way to load data from the file?
Thank you
To load a file, you can:
Double click on that file in the workspace directory.
Use the command:
load('file.mat');
To save a file after processing data, you can use:
save('filename.mat',nameobj,'-v7.3');
or
save('filename.mat',nameobj);

check if MAT file is corrupt without load

I have a data set consisting of large number of .mat files. Each .mat file is of considerable size i.e. loading them is time-consuming. Unfortunately, some of them are corrupt and load('<name>') returns error on those files. I have implemented a try-catch routine to determine which files are corrupt. However, given the situation that only handful of them are corrupt, loading each file and checking if it is corrupt is time taking. Is there any way I can check the health of a .mat file without using load('<name>')?
I have been unsuccessful in finding such solution anywhere.
The matfile function is used to access variables in MAT-files, without loading them into memory. By changing your try-catch routine to use matfile instead of load, you reduce the overhead of loading the large files into the memory.
As matfile appears to only issue a warning when reading a corrupt file, you'll have to check if this warning was issued. This can be done using lastwarn: clear lastwarn before calling matfile, and check if the warning was issued afterwards:
lastwarn('');
matfile(...);
[~, warnId] = lastwarn;
if strcmp(warnId, 'relevantWarningId')
% File is corrupt
end
You will have to find out the relevant warning id first, by running the above code on a corrupt file, and saving the warnId.
A more robust solution would be to calculate a checksum or hash (e.g. MD5) of the file upon creation, and comparing this checksum before reading the file.

error in loading a file in MATLAB

I want to load a video in yuv extension using the function load.
load foreman_qcif.yuv
But I get this error
??? Error using ==> load
Unknown text on line number 1 of ASCII file C:\Users\Chaine\Downloads\foreman_qcif.yuv
"Øÿ".
Also, I would like to place the values or parameters of that video into a cell array. I found a function "struct2cell" but I think I used incorrectly. How must I manipulate the digital video in frames using matrices for each frame?
load is for loading .mat files (MATLAB data files) or ASCII files, see documentation for more details.
You probably want to use the VideoReader class, but you'll need to convert your file into a supported file format first.
I think there don't exist way to read (or, load) YUV files. Function load exists only for MAT files (.mat).
Check these links:
http://www.mathworks.com/matlabcentral/fileexchange/6318
https://www.mathworks.com/matlabcentral/fileexchange/36417-yuv-files-reading-and-converting

load .mat file - run out of memory

I have a matrix cube which I load in my program to read data from. The size of this .mat file is 2.8 GB. I am not being able to load it with the error of 'running out of memory'. Is there a way to fix this?
You can use the matfile class to work on ranges within variables inside MatLab files. See
Load and save parts of variables in MAT-files
Here's some additional discussion that discloses that this feature is new with R2011b.
If the size of the data exceeds the available memory on your machine, then you are in trouble - this is unavoidable. However, if you only want certain variables inside the .mat file you can try to load just those variables using the
load(filename, variables)
version of the load function. It really depends on the contents of your .mat file. If the file is 2.8GB and you need ALL of the variables in the file and your machine does not have enough memory to cope, your only option is to buy more RAM.
EDIT Apparently this answer is incorrect if you are running R2011b and above as explained in the answer of Ben Voight