I've read the post "Read and write from/to a binary file in Matlab" but I still have doubts. I have a binary file of long double values created with fwrite in C and in Matlab I'm using
fid = fopen('vz3.dat', 'r')
mydata = fread(fid, 'double')
where vz3.dat is my file. But I'm getting garbage values in Matlab. According to
[cinfo, maxsize, ordering] = computer
in Matlab, my computer is a little-endian system (byte ordering system). Any suggestions?
By the way, does a binary file necessarily have to end in .bin .I'm using the .dat extension. Is it ok to do so?
Thanks a lot
To open a file with little endian, use
fid = fopen('vz3.dat','r','l');
It doesn't matter what the file is called, by the way.
In case you have to use a file handle opened elsewhere, you can also use the machineformat parameter to fread (which is optional).
The documentation is available on the MathWorks site.
Related
I have been struggling with this bug. When using MATLAB to read a binary file that contains three columns of numbers in float formats.
I am reading one number at a time using this line.
pt(j) = fread(fid,1,'float','a');
I have found that sometimes (rarely) MATLAB instead of reading four bytes for a float, it uses 5 bytes. And it misses up the rest of the reading. I am not sure if the file is corrupted or MATLAB has a bug there. When I printed the file as a txt and read it in txt everything works well.
As a work around here is what I did:
cur = ftell(fid);
if (cur - prev)~= 4
pt(j) = 0; % I m throwing this reading away for the sake of saving the rest of the data. This is not ideal
cur = prev +4;
fseek(fid, cur,'bof');
end
prev = cur;
I tried different combinations of different formats float32 float64 etc... nothing works MATLAB always read 5 bytes instead of 4 at this particular location.
EDIT:
To solve it based on Chris's answer. I was using this command to open the file.
fid = fopen(fname,'rt');
I replaced it with this
fid = fopen(fname,'r');
Sometimes, rarely, skipping a byte. It sounds to me like you are on Windows, and have opened the file in text mode. See the permissions parameter to the fopen function.
When opening a file in text mode on Windows, the sequence \r\n (13,10) is replaced with \n (10). This happens before fread gets to it.
So, when opening the file, don't do:
fid = fopen('name', 'rt');
The t here indicates "text". Instead, do:
fid = fopen('name', 'r');
To make this explicit, you can add b to the permissions. This is not documented, but is supposed to mean "binary", and makes the call similar to what you'd do in C or in the POSIX fopen():
fid = fopen('name', 'rb');
I have a quite big binary file that contains the information in a volume of about one GB of data.
I am trying to split one GB of .bin file into 4 files of 256 MB each, I am getting no help.
I tried using textscan and memmapfile functions but of no use.
Any help on this?
MATLAB can both read and write binary files.
Try using the fopen(), fread()/fwrite(), and fclose() commands.
To read a binary file into MATLAB, first open the file for reading.
Then the magic comes -- you need to know the context/structure of the binary data inside the .bin file so as to be able to scan the data thereafter into variables using the correct format.
fHANDLE = fopen( 'aHugeBinFileNAME.bin','r'); % Open the binary file for reading with a fileHandle fHANDLE
[dVECTOR,count] = fread( fHANDLE, 'int16' ); % Scan the data into a vector of int16 values, in this case called dVECTOR
fclose( fHANDLE); % Close the file
Anyway, get more details about fileIO services via help fread() et al on more options to read "first/next" 256MB of the huge input-file for your specific context of use.
dVECTOR_256M = fread( fHANDLE, 256*1024*1024, 'uint8', 0, 'b' ); % Big-Endian convention
I have a huge binary file with double precision numbers and I would like to load parts of it into Matlab. Is there a way to do this?
One way would be if I could convert it to a .mat file (without loading it in Matlab first), but I haven't been able to figure out how (or if it's actually possible).
Any ideas?
PS: I was thinking of using c++ to do the conversion but it turns out this is really problematic because I'm using a linux version of c++ (through cygwin) and a windows version of Matlab.
If you know what parts of the file you want to load, you can use fseek followed by fread (both preceded by fopen, of course).
For example, jump a few thousand bytes into a file and read a certain number of bytes, as doubles:
fid = fopen('binary.dat','r');
fseek(fid, 3000, 'bof');
A = fread(fid, N, 'double');
fclose(A); % don't forget to close the file
See the section of documentation called Reading Portions of a File for more information.
I have large .bin files (10GB 60GB) that I want to import to MATLAB; each binary file represents the output of two sensors, thus there are too columns of data. Here is a more manageable sized example of my data.
You will notice that there is a .txt version of the data; I need to upload the .bin files directly to MATLAB, I can't use the .txt version because it takes hours to convert with larger files.
The problem I have is that the .bin file has header information that I can't seem to interpret properly, and thus I cannot extract the data in MATLAB every time I try I seem to get gibberish values.
This is all the information I have about the binary header:
Loading Labview Binary Data into Matlab
LabVIEW Data Logger: Binary Header File Format
Any help/advice would be much appreciated I have been trying to solve this problem for days now.
P.S. Someone has already written a function to solve this problem but it does not seem to work with my binary data (could be something to do with the dimensions/size of my data): http://www.mathworks.co.uk/matlabcentral/fileexchange/27195-load-labview-binary-data
Below is the code that I am using to import my data, I believe that that d1 and d2 are the dimensions of my binary data. D2 is probably incorrect for the example file in the dropbox because it has been truncated.
The problem I have is that the code extracts my data and I know it is correct because I can check it with the .txt file (also in the drop box) however there are seaming random bad values between the good data points. These bad values result from the following strings following strings: "NI_ChannelName", "Sensor A", "Sensor B", "NI_UnitDescription", and "Volts" scatted throughout the binary file.
clear all
clc
fname = 'RTL5_57.bin';
fid = fopen(fname,'r','ieee-be');
d1 = fread(fid,4);
trash=fread(fid,2,'double');
d2 = fread(fid,4);
trash=fread(fid,1,'double');
data=fread(fid,'double');
I suppose you will need to change the data-format. See Matlab help.
https://decibel.ni.com/content/docs/DOC-39038
Scope:
1) Write a binary file in matlab and read into labview. 2) Write a binary file in labview and read into matlab.
Background:
IMPORTANT:
You must know (3) things about the binary data in the file before you can read the data:
1) what binary format (precision) was used to store the data
2) the exact number of values in the file to read.
3) Endianness
There is no row or column in binary files. Think of a long row/or a long column that needs to be mapped to a 2D array.
Resources on data in binary format.
http://cse.unl.edu/~sincovec/Matlab/Lesson%2024/Binary/CS211%20Lesson%2024%20-%20Binary%20File%20Input-Output.htm
My knowledge of matlab is merely on a need to know basis, so this is probably an elementary question. Nevertheless here it comes:
I have got a file containing data (16-bit integers) stored in binary format. How do I read it into a vector /an array in matlab? How do I write this data to a file in matlab? Is there any smart tweak to increase the performance speed when reading/writing a huge amount of data (gigabytes)?
As Bill the Lizard wrote you can use fread to load the data into a vector. I just want to expand a little on his answer.
Reading Data
>> fid=fopen('data.bin','rb') % opens the file for reading
>> A = fread(fid, count, 'int16') % reads _count_ elements and stores them in A.
The commands fopen and fread default to Little-endian[1] encoding for the integers. If your file is Big-endian encoded you will need to change the fread to
>> A = fread(fid, count, 'int16', 'ieee-be');
Also, if you want to read the whole file set
>> count=inf;
and if you want to read the data into matrix with n columns use
>> count=[n inf];
Writing Data
As for witting the data to a file. The command, fwrite, in Bill's answer will write to a binary file. If you want to write the data to a text file you can use dlmwrite
>> dlmwrite('data.csv',A,',');
References
[1] http://en.wikipedia.org/wiki/Endianness
Update
The machine format (IE, ieee-be,
ieee-le, vaxd etc.) of the binary data can be specified in either the
fopen or the fread commands in Matlab. Details of the supported
machine format can be found in
Matlab's documentation of fopen.
Scott French's comment to Bill's
answer
suggests reading the data into an
int16 variable. To do this use
>> A = int16(fread(fid,count,precision,machineFormat));
where count is the size/shape of
the data to be read, precision is
the data format, and machineformat
is the encoding of each byte.
See commands fseek to move around the file. For example,
>> fseek(fid,0,'bof');
will rewind the file to the beginning where bof stands for beginning of file.
Assuming you know how many values you have stored in the file, you can do something like this to read the data into an array.
fid = fopen('data.bin','rb')
A = fread(fid, count, 'int16')
To write data to a file do this:
fid = fopen('data.bin','w')
count = fwrite(fid, A, 'int16')
The fwrite function returns the number of elements (not bytes) written to the file.
As far as performance tuning goes, you can read data in chunks to only use as much as you need to process. This is the same in any language, and there's no way to speed it up that's specific to Matlab.
I usually hate seeing links in a response, but this looks pretty close:
http://www.mathworks.com/support/tech-notes/1400/1403.html
As to the second part of performance tuning, it's been 6 years since I've used Matlab, so I don't know.
HTH