OCTAVE: Loading and processing multiples files (32 files) - matlab

I need to load and do some maths with 32 files (extension .mat) at the same time. So, after running the code, I expect to have 32 results of the maths.
The problem is that all the codes I'm trying just load the first or the last file.
The name of my files are: 21 pcb 11_01.mat; 21 pcb 11_02 ....21 pcb 11_32. I've tried this:
for i=1:32
filename=strcat("21 pcb 11_",sprintf("%02d",i),".mat")
load(filename)
endfor
As a result, the code only shows the last file in the workspace.
I expected the code to load the 32 files.
Can you help me?

If your Picoscope files are all the same length, say Lpico, then this ought to work:
Pico=NaN*ones(32,Lpico);
for k=1:32
filename=strcat("21 pcb 11_",sprintf("%02d",i),".mat")
load(filename)
Lthisrun=length(A);
Pico(k,1:Lthisrun)=A;
endfor
If they have different length, then make Lpico as long as the longest A. Shorter scope ouputs will be padded with NaN's

Related

how to read multiple text files in matlab

Could someone please help me, I have 40 txt files which I want to read into matlab so that each one will be displayed as a matrix (each file has 5 columns and 2025 rows). If I run this code, then txtfiles and j is displayed correctly, but T shows me just one matrix.... Thanks in advance!
txtfiles=dir('*.txt');
for j=1:length(txtfiles)
T=readmatrix(txtfiles(j).name)
end
you can instead write in the loop T as a cell or as an array, for example:
...
T{j}=...
...
or
T(:,:,j)=...

Save not successful for big cell array in matlab

I am working with Matlab 2009b in Windows 7 64 bit environment.
I am not able to save cell array of size 2.5GB using v7.3 switch but save is successful for struct of arrays of size 6 GB, containing only double values. Please advise me about the alternatives I can try.
What is working
I am able to save following variable in workspace successfully.
>> whos
Name Size Bytes Class Attributes
master_data 1x159 6296489360 struct
>> save('arrayOfStruct.mat','-v7.3')
Here master data is an array of 159 structure. Each of these 159 structures have five array of 1 million double values. Mat file of 594 MB saved in the filesystem.
What is not working
I am not able to save a cell array, which contains strings, doubles and array of doubles.
>> whos
Name Size Bytes Class Attributes
result_combined 57888x100 2544467328 cell
>> save('cellArray.mat','-v7.3');
When I execute the save command, a cellArray.mat file of size 530 MB is generated in the filesystem but the prompt never returns to matlab.(I have waited for more than 4 hours and run this after restarting the computer). If I terminate the matlab program while it is waiting for the prompt to return, the generated cellArray.mat is not usable as matlab shows the file cannot be loaded as it is corrupt.
Please suggest what I can try to save this variable result_combined.
NOTE
The save command works successfully in Matlab 2015a. Any suggestions as to how I can make it work in Matlab 2009b. I am using Matlab 2009b as default and do not want to migrate to 2015a as it might break existing setup.

Determine number of bytes in a line of a binary file Matlab

I am working on importing some data interpretation of binary files from Fortran to MATLAB and have come across a bit of an issue.
In the Fortran file I am working with the following check is performed
CHARACTER*72 PICTFILE
CHARACTER*8192 INLINE
INTEGER NPX
INTEGER NLN
INTEGER BYTES
c This is read from another file but I'll just hard code it for now
NPX = 1024
NLN = 1024
bytes=2
open(unit=10, file=pictfile, access='direct', recl=2*npx, status='old')
read(10,rec=nln, err=20) inline(1:2*npx)
go to 21
20 bytes=1
21 continue
close(unit=10)
where nln is the number of lines in the file being read, and npx is the number of integers contained in each line. This check basically determines whether each of those integers is 1 byte or 2 bytes. I understand the Fortran code well enough to figure that out, but now I need to figure out how to perform this check in MATLAB. I have tried using the fgetl command on the file and then reading the length of the characters contained but the length never seems to be more than 4 or 5 characters, when even if each integer is 1 byte the length should be somewhere around 1000.
Does someone know a way that I can automatically perform this check in MATLAB?
So what we figured out was that the check is simply to see if the file is the correct size. In Matlab this can be done as
fullpath=which(file); %extracting the full file path
s=dir(fullpath); %extracting information about hte file
fid=fopen(file_name,'r'); %opening image file
if s.bytes/NLN==2*NPX %if the file is NLN*NPX*2 bytes
for n=1:NLN %for each line
dn(n,:) = (fread(fid, NPX, '*uint16','b'))'; %reading in lines into DN
end
elseif s.bytes/NLN==NPX %Else if the file is NLN*NPX bytes
for n=1:NLN %for each line
dn(n,:) = (fread(fid, NPX, '*uint8','b'))'; %reading in lines into DN
end
else %If the file is neither something went wrong
error('Invalid file. The file is not the correct size specified by the SUM file')
end
where file contains the filename, nln contains the number of lines, and npx contains the number of columns. Hope this helps anyone who may have a similar answer, but be warned because this will only work if your file only contains data that has the same number of bytes for each entry, and if you know the total number of entries there should be!
Generally speaking, binary files don't have line lengths; only text files have line lengths. MATLAB's getl will read until it finds the binary equivalent of newline characters. It then removes them and returns the result. The binary file, on the other hand, should read a block of length 2*npx and return the result. It looks like you want to use fread to get a block of data like this:
inline = fread(fileID,2*npx)
Your fortran code is requesting to read record nln. If the code you have shared reads all the records starting at the first one and working up, then can just put the above code in a loop for nln=1:maxValue. However, if you really do want to yank out record nln you need to fseek to that position first:
fseek(fileID, nln*2*npx, -1);
inline = fread(fileID,2*npx)
so you get something like the following:
Either reading them all in a loop:
fileID = fopen(pictfile);
nln = 0;
while ~feof(fileID)
nln = nln+1;
inline = fread(fileID,2*npx);
end
fclose(fileID);
or picking out only the number `nln record:
fileID = fopen(pictfile);
nln = 7;
fseek(fileID, nln*2*npx, -1);
inline = fread(fileID,2*npx);
fclose(fileID);

Loading huge binary file partially into Matlab

I have a huge binary file with double precision numbers and I would like to load parts of it into Matlab. Is there a way to do this?
One way would be if I could convert it to a .mat file (without loading it in Matlab first), but I haven't been able to figure out how (or if it's actually possible).
Any ideas?
PS: I was thinking of using c++ to do the conversion but it turns out this is really problematic because I'm using a linux version of c++ (through cygwin) and a windows version of Matlab.
If you know what parts of the file you want to load, you can use fseek followed by fread (both preceded by fopen, of course).
For example, jump a few thousand bytes into a file and read a certain number of bytes, as doubles:
fid = fopen('binary.dat','r');
fseek(fid, 3000, 'bof');
A = fread(fid, N, 'double');
fclose(A); % don't forget to close the file
See the section of documentation called Reading Portions of a File for more information.

Why is MATLAB failing to successfully read in binary files?

Matlab is failing to read in the specified number of elements from a file. I have a simple program that needs to read in two files, perform a linear operation on the data and write a combined result to a third file.
My questions are: 1) Why does Matlab fail to read the specified number of elements and 2) is there a workaround for this? Any of your thoughts will be helpful.
Some details on the input files:
they are large (~18GB)
they are both the same size (exactly)
Details on the procedure (2-4 are conditioned on an feof check of both files:
Open the input and output files for reading and writing (resp.)
Read in N floats (N*4 bytes) from each of the input files
Perform an operation on the data (say 0.5*(datin1+datin2))
Write the result to the output file.
Granted, this is all very simple and in most cases in the past this has worked well. Unfortunately, at some point in the cycle, MATLAB doesn't get all N floats from one of the files and gives a matrix dimension error on step 3.
CODE SNIP:
N = 2048;
fidin1 = fopen('file1.dat','r','l');
fidin2 = fopen('file2.dat','r','l');
fidout = fopen('outfile.dat','w','l');
%# I could do some assertions on the file sizes,
%# but I know they are the same size (w/o question).
while(~feof(fidin1) && ~feof(fidin2))
datin1 = fread(fidin1,N,'float=>single',0,'l');
datin2 = fread(fidin2,N,'float=>single',0,'l');
%# the following line produces an error after 100
%# or more iterations in to the procedure
datout = 0.5*(datin1+datin2);
fwrite(fidout,datout,'float',0,'l');
end
UPDATE 1
The error message I'm receiving is:
???Error using ==> plus
Matrix dimension must agree.
UPDATE 2
I followed a suggestion and included ferrorchecks after each read and magically the problem went away. So now a modification to my questions: What could be the root of the problem here? Is this simply a timing issue or bug?
Here is a snip of the updated code (showing only a portion of the code). I'm sure there are better ways to do this. Regardless, the addition of these checks allowed Matlab to complete all the reads from each of the files successfully.
[datin1 count1]= fread(fidin1,N,'float=>single',0,'l');
[msg errn1]=ferror(fidin1);
if errn1
pos1 = ftell(fidin1);
error('Error at Position %d in file. %d bytes were read.',...
pos1,count1);
end
[datin2 count2]= fread(fidin2,N,'float=>single',0,'l');
[msg errn2]=ferror(fidin2);
if errn2
pos2 = ftell(fidin2);
error('Error at Position %d in file. %d bytes were read.',...
pos2,count2);
end
%# the following line produces an error after 100
%# or more iterations in to the procedure
datout = 0.5*(datin1+datin2);
fwrite(fidout,datout,'float',0,'l');
Have you specifically looked at the datin1 and datin2 variables at the time the error occurs? Try going to 'Debug-->Stop if Errors/Warnings...' then select 'Always stop if error (dstop if error)'. Run your program and then once it crashes, look at datin1 and datin2. Hopefully that will explain why adding them together is now working.