I am trying to extract a specific layer from multiple ( > 4000 ) HDF5 files. I used the code below. It worked, but when I tried to load the new saved files, they were not recognized as HDF5. Help would be highly appreciated.
files=dir('C:\OLD_GPM\*.HDF5') % Open dataset
for j = 1:numel(files)
r = h5read(files(j).name,'/Grid/precipitationCal');% Read the correct layer "Precipitation calibrated mm/hr"
save([''C:\New_GPM\' files(j).name],'r'); % save this layer
end
Only mat-file version 7.3 are HDF5 files, all older versions use another format. Use save(['C:\New_GPM\' files(j).name],'r','-v7.3'); to enforce writing an HDF5 file.
Related
I have multiple small *.mat files, each containing 4 input images (template{1:4} and a second channel template2{1:4}) and 4 output images (region_of_interests{1:4}), a binarized ('mask') image to train a deep neural network.
I basically followed an example on Mathworks and it suggests to use a function (in this example #matreader) to read in custom file formats.
However ...
It seems impossible to load multiple images from one *.mat file using any load function as it only allows one output, and imageDatastore doen't seem to allow loading data from workspace. How could this be achieved?
Similarly, it seems impossible to load a pixelLabelDatastore from a workspace variable. As a workaround I ended up saving the contents of my *.mat file to an image (using imwrite, saving to save_dir), and re-loading it from there (in this case, the function doesn't even allow to load *.mat files.). (How) can this be achieved without re-saving the file as image?
Here my failed attempt to do so:
%main script
image_dir = pwd; %location of *.mat files
save_dir = [pwd '/a/']; %location of saved output masks
imds = imageDatastore(image_dir,'FileExtensions','.mat','ReadFcn',#matreader); %load template (input) images
pxds = pixelLabelDatastore(save_dir,{'nothing','something'},[0 255]);%load region_of_interests (output) image
%etc, etc, go on to train network
%matreader function, save as separate file
function data=matreader(filename)
in=1; %give up the 3 other images stored in template{1:4}
load(filename); %loads template and template2, containing 4x input images each
data=cat(3,template{in},template2{in}); %concatinate 2 template input images in 3rd dimension
end
%generate example data for this question, will save into a file 'example.mat' in workspace
for ind=1:4
template{ind}=rand([200,400]);
template2{ind}=rand([200,400]);
region_of_interests{ind}=rand([200,400])>.5;
end
save('example','template','template2','output')
You should be able to achieve this using the standard load and save function. Have a look at this code:
image_dir = pwd;
save_dir = pwd;
imds = imageDatastore(image_dir,'FileExtensions',{'.jpg','.tif'});
pxds = pixelLabelDatastore(save_dir,{'nothing','something'},[0 255]);
save('images.mat','imds', 'pxds')
clear
load('images.mat') % gives you the variable "imds" and "pxds" directly -> might override previous variables
tmp = load('images.mat'); % saves all variables in a struct, access it via tmp.imds and tmp.pxds
If you only want to select the variables you want to load use:
load('images.mat','imds') % loads "imds" variable
load('images.mat','pxds') % loads "pxds" variable
load('images.mat','imds','pxds') % loads both variables
EDIT
Now I get the problem, but I fear this is not how it is going to work. The Idea behind the Datastore objects is, that it is used if the data is too big to fit in memory as a whole, but every little piece is small enough to fit in memory. You can use the Datastore object than to easily process and read multiple files on a disk.
This means for you: Simply save your images not as one big *mat file but as multiple small *.mat files that only contain one image.
EDIT 2
Is it strictly necessary to use an imageDatastore for this task? If not you can use something like the following:
image_dir = pwd;
matFiles = dir([image_dir '*.mat']);
for i=1:length(matFiles)
data = load(matFiles(i).name);
img = convertMatToImage(data); % write custom function which converts the mat input to your image
% or something like this:
% for j=1:4
% img(:,:,j) = cat(3,template{j},template2{j});
% end
% process image
end
another alternative would be to create a "image" in your 'matreader' which does not only have 2 bands but to simply put all bands (all templates) on top of each other providing a "datacube" and then in an second step after iterating over all small mat files and reading them splitting the single images out of the one bigger datacube.
would look something like this:
function data=matreader(filename)
load(filename);
for in=1:4
data=cat(3,template{in},template2{in});
end
end
and in your main file, you have to simply split the data into 4 pieces.
I have never tested it but maybe it is possible to return a cell instead of a matrix?
function data=matreader(filename)
load(filename);
data = cell(1,4)
for in=1:4
data{in}=cat(3,template{in},template2{in});
end
end
Not sure if this would work.
However, the right way to go forward from here really depends on how you plan to use the images from imds and if it is really necessary to use a imageDatastore.
I wanted to create a feature vector for training dataset and wanted to store all feature as rows in the.mat file. The .mat file must be in the form Feature Vector. I am able to extract Feature of 1 image and store it in excel file or .mat file but not able to extract all image feature and store it in .mat file. Can anyone knows this?
Its something like appending the same variables in the same .mat file. I have tried
save('feat.mat','feature','-append');
Where 'feature' is an array
feature = [mydata, stats{k}];
I have a folder which contains images who's feature I wanted to extract and store as training dataset. Any help will be appreciated.
the code was rectified.
stats = graycoprops(GLCM_values{'contrast','homogeneity','Correlation','Energy'});
stats was a structure so converted into the array using
features=struct2array(stats);
and was able to save the features
I generated and saved large number of data files using Octave, and now I need to open them in MATLAB as part of analyzing them. MATLAB spits out this error.
Error using load
Unable to read MAT-file
[file path]/PhPar_40.mat: not a binary MAT-file.
Try LOAD -ASCII to read as text.
Trying its suggestion of load -ASCII then gives this error.
Error using load
Number of columns on line 2 of ASCII file
[filepath]/PhPar_40.mat must be the same as previous lines.
I (now) understand that Octave is capable of saving in a MATLAB readable format, but re-creating these data files would take an inordinate amount of time and really isn't an option. Is there a way to get MATLAB to read these files?
MATLAB can't open these files because these are not saved by octave properly. Try saving them in octave by following command:
save -mat7-binary '[filepath]/PhPar_40.mat' 'm'
If you have large number of files, you can place all files in folder and then run an iterator to read all load and save in correct format automatically. This iterator will look like:
file = dir('[filepath_read]/*.mat');
index = 1;
while (index==length(file)+1)
m = load('file(index).name')
save -mat7-binary strcat("[filepath_write]/", file(index).name, ".mat") 'm';
index = index+1;
pause(1);
endwhile
Once you have all the files converted to right format, load them in MATLAB. I hope it will solve your problem
I'm trying to save models in oldest MATLAB versions as below
I look for each folder and subfolder to find any .mdl or .slx to save it as 2007b version
The problem I have is :
it works if I just look for one extension whereas I'm wondering
to do that on each .mdl and.slx .
the save_system takes too much
time
Do you know how could I get all .mdl and .slx and is there an optimized way to save ?
Thanks
rootPath = fullfile('M:\script\ytop','tables');
files = dir(rootPath );
for ii = 3:numel(files)
x = fullfile(rootPath ,files(ii).name);
cd(x);
mdl = { dir('*.mdl'),dir('*.slx')}; % here it works if only I set dir('*.mdl')
for jj = 1:numel(mdl)
load_system(mdl(jj).name);
save_system(mdl(jj).name,mdl(jj).name, 'SaveAsVersion','R2007b');
end
end
%here you used {} which created a cell array of two structs. cat creates a single struct which.
mdl=cat(1,dir('*.mdl'),dir('*.slx'));
for jj = 1:numel(mdl)
[~,sysname,~]=fileparts(mdl(jj).name);
load_system(mdl(jj).name);
%use only sysname without extension. R2007b is mdl only. You can't store files for R2007b in slx format
save_system(sysname,sysname, 'SaveAsVersion','R2007b');
%close system to free memory.
close_system(sysname);
end
Applying only the required fixes your code has one odd behaviour. For mdls the file is replaced with the original one, for slx a mdl is created next to the original one. You may want to add a delete(mdl(jj).name) after loading.
I have a data file having 50 2-D data points written in Notepad. I want to use it in clustering algorithm to cluster these 50 points. How can I import this file? Is there any other way to use it in program?
You can save the data as a .csv file or you can save it to an excel spreadsheet and use xlsread(). See here for more info: http://www.mathworks.com/help/techdoc/ref/xlsread.html
For the .csv case, this post should prove helpful: Fastest way to import CSV files in MATLAB
Imagine you had the following data:
X = [randn(100,2)-1 ; randn(100,2)];
save data.mat X
Then its as simple as doing:
%# load data from MAT-file
load data.mat
%# cluster into K=2 clusters
C = kmeans(X,2);
%# show cluster assignment
gscatter(X(:,1), X(:,2), C)
It depends how you have formatted the data file. You say it is saved on notepad but that is not too helpful. Depending on what you have used as the data delimiter you can import the datafile into an array using the dlmread function. For example if your file is called filename.dat and have used a ; character to separate each data item within this file you could read the data into a matrix A using
A = dlmread("filename.dat",';');
I would suggest reading the help documentation on the dlmread function in matlab.