I have the following bytes stored in a vector:
data = [189 33 136 147]
These 4 bytes represent a single float in Big-endian order. How can I get this number in MATLAB?
I will need to concatenate and convert. I tried:
x = typecast(str2num(sprintf('%d%d%d%d',data(1),data(2),data(3),data(4))), 'single')
To no avail (I got x = []).
great example here:
>> dataL = typecast(uint8([189, 33, 136, 147]), 'uint32')
dataL =
2475172285
>> dataF = double(dataL)
dataF =
2.4752e+09
big to little, try swapbytes
>> dataLbig = swapbytes(dataL)
dataLbig =
3173091475
>> dataFbig = double(dataLbig)
dataFbig =
3.1731e+09
Is this what you were expecting?
I'll leave this here in case it's useful for anyone. As #MarkMikofski showed, using typecast and swapbytes is the standard method for this. However, if you your data is already floating-point, these functions can be inefficient in some cases. I use the following utility function in my video-encoding/decoding tools:
function x = Bit32toDouble(y)
n = numel(y);
if n >= 65536
x = double(swapbytes(typecast(uint8(y(:)),'uint32'))).';
elseif n > 4
x = sum(bsxfun(#times,[16777216;65536;256;1],reshape(y(:),4,n/4)));
else
x = sum([16777216;65536;256;1].*y(:));
end
There are separate cases depending on the number of bytes passed in. Only when a large amount of data is processed at once is the typecast/swapbytes most efficient. If the function is called repeatedly with smaller inputs, as is common in my application, the other cases are much faster because they do every thing in Matlab's native floating-point.
Related
I've declared a function that will be used to calculate the convolution of an image using an arbitrary 3x3 kernel. I also created a script that will prompt the user to select both an image as well as enter the convolution kernel of their choice. However, I do not know how to go about dealing with negative pixel values that will arise for various kernels. How would I implement a condition into my script that will deal with these negative values?
This is my function:
function y = convul(x,m,H,W)
y=zeros(H,W);
for i=2:(H-1)
for j=2:(W-1)
Z1=(x(i-1,j-1))*(m(1,1));
Z2=(x(i-1,j))*(m(1,2));
Z3=(x(i-1,j+1))*(m(1,3));
Z4=(x(i,j-1))*(m(2,1));
Z5=(x(i,j))*(m(2,2));
Z6=(x(i,j+1))*(m(2,3));
Z7=(x(i+1,j-1))*(m(3,1));
Z8=(x(i+1,j))*(m(3,2));
Z9=(x(i+1,j+1))*(m(3,3));
y(i,j)=Z1+Z2+Z3+Z4+Z5+Z6+Z7+Z8+Z9;
end
end
And this is the script that I've written that prompts the user to enter an image and select a kernel of their choice:
[file,path]=uigetfile('*.bmp');
x = imread(fullfile(path,file));
x_info=imfinfo(fullfile(path,file));
W=x_info.Width;
H=x_info.Height;
L=x_info.NumColormapEntries;
prompt='Enter a convulation kernel m: ';
m=input(prompt)/9;
y=convul(x,m,H,W);
imshow(y,[0,(L-1)]);
I've tried to use the absolute value of the convolution, as well as attempting to locate negatives in the output image, but nothing worked.
This is the original image:
This is the image I get when I use the kernel [-1 -1 -1;-1 9 -1; -1 -1 -1]:
I don't know what I'm doing wrong.
MATLAB is rather unique in how it handles operations between different data types. If x is uint8 (as it likely is in this case), and m is double (as it likely is in this case), then this operation:
Z1=(x(i-1,j-1))*(m(1,1));
returns a uint8 value, not a double. Arithmetic in MATLAB always takes the type of the non-double argument. (And you cannot do arithmetic between two different types unless one of them is double.)
MATLAB does integer arithmetic with saturation. That means that uint8(5) * -1 gives 0, not -5, because uint8 cannot represent a negative value.
So all your Z1..Z9 are uint8 values, negative results have been set to 0. Now you add all of these, again with saturation, leading to a value of at most 255. This value is assigned to the output (a double). So it looks like you are doing your computations correctly and outputting a double array, but you are still clamping your result in an odd way.
A Correct implementation would cast each of the values of x to double before multiplying by a potentially negative number. For example:
for i = 2:H-1
for j = 2:W-1
s = 0;
s = s + double(x(i-1,j-1))*m(1,1);
s = s + double(x(i-1,j))*m(1,2);
s = s + double(x(i-1,j+1))*m(1,3);
s = s + double(x(i,j-1))*m(2,1);
s = s + double(x(i,j))*m(2,2);
s = s + double(x(i,j+1))*m(2,3);
s = s + double(x(i+1,j-1))*m(3,1);
s = s + double(x(i+1,j))*m(3,2);
s = s + double(x(i+1,j+1))*m(3,3);
y(i,j) = s;
end
end
(Note that I removed your use of 9 different variables, I think this is cleaner, and I also removed a lot of your unnecessary brackets!)
A simpler implementation would be:
for i = 2:H-1
for j = 2:W-1
s = double(x(i-1:i+1,j-1:j+1)) .* m;
y(i,j) = sum(s(:));
end
end
I am writing a simple code to output some large matrices to the disk that subsquently will be read in matlab.
I have written the following code, which exemplifies the writing for one such matrix. I am concerned with two things:
Efficiency in writing to disk (looking for something that is not too slow)
Easily being able to read it in matlab
a
PROGRAM WriteDisk
character(80) :: filename = ' '
INTEGER :: indt
INTEGER :: ind1, n1 = 161
INTEGER :: ind2, n2=20
INTEGER :: ind3, n3=2
INTEGER :: ind4, n4=2
INTEGER :: ind5, n5=21
INTEGER :: ind6, n6=20
INTEGER :: ind7, n7=2
INTEGER :: ind8, n8=2
INTEGER :: dummy
REAL, ALLOCATABLE :: m1(:,:,:,:,:,:,:,:,:)
ALLOCATE(m1(2,n1,n2,n3,n4,n5,n6,n7,n8))
dummy = 1
do ind8 = 1,n8
do ind7 = 1,n7
do ind6 = 1,n6
do ind5 = 1,n5
do ind4 = 1,n4
do ind3 = 1,n3
do ind2 = 1,n2
do ind1=1,n1
m1(2,ind1,ind2,ind3,ind4,ind5,ind6,ind7,ind8) = dummy
dummy = dummy + 1
end do
end do
end do
end do
end do
end do
end do
end do
indt = 1
write(filename,'(a,i0,a)')'PF_m1_',indt,'.txt'
OPEN(UNIT=25,FILE=filename,STATUS='replace',ACTION='write')
WRITE(25, *) m1(2,:,:,:,:,:,:,:,:)
CLOSE(UNIT=25)
END PROGRAM
The program above writes the matrix m1 as a 4327680 x 5 . This makes it cumbersome to reshape it in matlab (although totally possible), as in Matlab I need to do the following:
Maybe I was not clear enough in my question. When Fortran writes that matrix it writes is with 4327680 rows and 5 columns. I.e. when I open it in matlab I have to do something like to get the matrix in the original format:
n1 = 161;
n2 = 20;
n3 = 2;
n4 = 2;
n5 = 21;
n6 = 20;
n7 = 2;
n8 = 2;
m1 = load('PF_m1_1.txt'); %This is a two dimensional matrix that needs to be transposed and reshaped TWICE to get the original matrix
m1 = m1';
m1 = m1(:);
m1 = reshape(m1, n1,n2,n3,n4,n5,n6,n7,n8)
Is there anyway to write it as a single vector with element with element m1(2,1,1,1,1,1,1,1,1) as first element, m1(2,2,1,1,1,1,1,1,1) as second element, ... , m1(2,end,end,end,end,end,end,end,end) as last element, etc?
Or anyway that I am not aware of, to quickly save it directly as .mat file?
"Is there anyway to write it as a single vector with element with element m1(2,1,1,1,1,1,1,1,1) as first element, m1(2,2,1,1,1,1,1,1,1) as second element, ... , m1(2,end,end,end,end,end,end,end,end) as last element, etc?"
Yes, this the default Fortran column major order. This is the order your file is already written. There is nothing you have to do.
"This makes it cumbersome to reshape it in Matlab (although totally possible), as in Matlab I need to do the following:
m1 = reshape(m1, n1,n2,n3,n4,n5,n6,n7,n8)"
Reshape just updates the internal descriptor. It should be a very fast operation. Completely negligible. Even if it needed to shuffle the data, it would still be much quicker than reading from the hard-drive.
"I am concerned with two things: 1. Efficiency in writing to disk (looking for something that is not too slow)"
Use unformatted (also known as binary) I/O:
OPEN(UNIT=25,FILE=filename,ACCESS='stream',STATUS='replace',ACTION='write')
WRITE(25) m1(2,:,:,:,:,:,:,:,:)
CLOSE(UNIT=25)
"2. Easily being able to read it in Matlab"
To read it from Matlab, learn how to read binary data from Read and write from/to a binary file in Matlab from the Matlab documentation https://www.mathworks.com/help/matlab/ref/fread.html and from loads of other resources.
Don't forget to tell Matlab the right dimensions. Or store the dimensions in the first bytes of the data file (a header).
I know it is not possible to create a true binary variable in Matlab. Even 'true(8,1)' has a size of 8 bytes, instead of 8 bits. Very memory inefficient.
In Jon Bentley's Programming Pearls he poses this problem:
input: a file containing at most n positive integers, each less then n, where n = 1E7. No integer exists twice.
output: a sorted list in increasing order of input integers
constraints: At most roughly 1MB of storage available in main memory. Ample disk storage.
In the answer he claims the program should be able to solve it lightning fast using a binary map, or bit-vector. I tried to implement this bit-vector in matlab:
classdef binar
properties (Access=protected)
byteslist uint8
end
methods
function obj = binar(N)
N=ceil(N/8);
obj.byteslist=zeros([N,1],'uint8');
end
function obj = setbit(obj,N,bit) %N from 0 to N-1
v=rem(N,8)+1; %byte position [1 8]
B = (N-v+1)/8+1; %byte nr [1 ..]
obj.byteslist(B)=bitset(obj.byteslist(B),v,bit);
end
function bit = getbit(obj,N)
v=rem(N,8)+1;
B = (N-v)/8+1;
bit = bitget(obj.byteslist(B),v);
end
end
end
To my surprise setting bits is very slow. I timed this code:
N=1E7;
B = binar(N);
itr=1E5;
tic
for ct = 1:itr
pos = uint32(randi([0,N-1],1));
B = setbit(B,pos,1);
end
toc
From the size perspective this works fine. 'S=whos('B');S.bytes/2^20' returns 1.19MB. However it does not seem to be very fast. Running bitset 1E5 times takes 0.367s, where I would think this is literally the simplest operation you can ask of a computer.
Is there a way to make this faster in Matlab?
update: I tried with bitand, and a predeclared vector
obj.byte=uint8([1 2 4 8 16 32 64 128]); %declared on creation of obj
obj.byteslist(B)=bitand(obj.byteslist(B),obj.byte(v));
Then i tried:
obj.byteslist(B)=obj.byteslist(B)+obj.byte(v);
There is hardly a speed difference with bitset.
Let's say I have a random variable a=1.2400, and I want to print it with four significant figures, i.e., 1.240. How would I go about that?
fprintf('%0.4g',a) % drops rightmost zero
fprintf('%0.3f',a) % give too many sig figs if a >= 10
Using '%g' drops the important zeros, and with '%f' I can only specify the number of digits after the decimal, which results in too many significant figures if, say, a=10.04. I'm not too familiar with formatting ,but there has to be a simple method. I haven't found it in my searches.
If the values to be printed are all less than 10000, you can do the following. (Sorry, only tested in octave.)
octave:62> a = 1.24
a = 1.2400
octave:63> sprintf('%.*f\n', 3-floor(log10(abs(a))), a)
ans = 1.240
octave:64> a = 234.56
a = 234.56
octave:65> sprintf('%.*f\n', 3-floor(log10(abs(a))), a)
ans = 234.6
For more about the expression floor(log10(abs(a))), see How can I get the exponent of each number in a np.array?
If you don't mind exponential notation, another alternative is to use '%.3e' to always get the same number of signficant digits:
octave:70> a = 1.24
a = 1.2400
octave:71> sprintf('%.3e\n', a)
ans = 1.240e+00
octave:72> a = 234.56
a = 234.56
octave:73> sprintf('%.3e\n', a)
ans = 2.346e+02
I decided to build on the answer by Warren, and I wrote a function that should work for both small and large numbers alike. Perhaps someone will improve on this, but I am pleased with it.
function str=sigfigstr(a,sigfigs)
numdecimal = floor(log10(abs(a)));
if sigfigs - numdecimal < 0
str=sprintf('%.0f',round(a,sigfigs,'significant'));
else
str=strip(sprintf('%.*f\n', sigfigs-floor(log10(abs(a))), a));
end
Here are a few examples if it in action in Matlab
>> sigfigstr(.000012431634,3)
ans = '0.0000124'
>> sigfigstr(26666,3)
ans = '26700'
Please suggest how to sort out this issue:
nNodes = 50400;
adj = sparse(nNodes,nNodes);
adj(sub2ind([nNodes nNodes], ind, ind + 1)) = 1; %ind is a vector of indices
??? Maximum variable size allowed by the program is exceeded.
I think the problem is 32/64-bit related. If you have a 32 bit processor, you can address at most
2^32 = 4.294967296e+09
elements. If you have a 64-bit processor, this number increases to
2^64 = 9.223372036854776e+18
Unfortunately, for reasons that are at best vague to me, Matlab does not use this full range. To find out the actual range used by Matlab, issue the following command:
[~,maxSize] = computer
On a 32-bit system, this gives
>> [~,maxSize] = computer
maxSize =
2.147483647000000e+09
>> log2(maxSize)
ans =
3.099999999932819e+01
and on a 64-bit system, it gives
>> [~,maxSize] = computer
maxSize =
2.814749767106550e+14
>> log2(maxSize)
ans =
47.999999999999993
So apparently, on a 32-bit system, Matlab only uses 31 bits to address elements, which gives you the upper limit.
If anyone can clarify why Matlab only uses 31 bits on a 32-bit system, and only 48 bits on a 64-bit system, that'd be awesome :)
Internally, Matlab always uses linear indices to access elements in an array (it probably just uses a C-style array or so), which implies for your adj matrix that its final element is
finEl = nNodes*nNodes = 2.54016e+09
This, unfortunately, is larger than the maximum addressable with 31 bits. Therefore, on the 32-bit system,
>> adj(end) = 1;
??? Maximum variable size allowed by the program is exceeded.
while this command poses no problem at all on the 64-bit system.
You'll have to use a workaround on a 32-bit system:
nNodes = 50400;
% split sparse array up into 4 pieces
adj{1,1} = sparse(nNodes/2,nNodes/2); adj{1,2} = sparse(nNodes/2,nNodes/2);
adj{2,1} = sparse(nNodes/2,nNodes/2); adj{2,2} = sparse(nNodes/2,nNodes/2);
% assign or index values to HUGE sparse arrays
function ret = indHuge(mat, inds, vals)
% get size of cell
sz = size(mat);
% return current values when not given new values
if nargin < 3
% I have to leave this up to you...
% otherwise, assign new values
else
% I have to leave this up to you...
end
end
% now initialize desired elements to 1
adj = indHuge(adj, sub2ind([nNodes nNodes], ind, ind + 1), 1);
I just had the idea to cast all this into a proper class, so that you can use much more intuitive syntax...but that's a whole lot more than I have time for now :)
adj = sparse(ind, ind + 1, ones(size(ind)), nNodes, nNodes, length(ind));
This worked fine...
And, if we have to access the last element of the sparse matrix, we can access by adj(nNodes, nNodes), but adj(nNodes * nNodes) throws error.