I am using GPU for computation in matlab. And I keep on getting Out of memory problem.
So I think I could convert some of my variables from double, which is the default type of matlab, to single. Then I did the following experiment
A = gpuArray([1,2,3])
A =
1 2 3
whos A
Name Size Bytes Class
A 1*3 4 gpuArray
B = gpuArray(single([1,2,3]))
B =
1*3 gpuArray single row vector
1 2 3
whos B
Name Size Bytes Class
B 1*3 4 gpuArray
Now I am a little bit confusing. On one hand, it does show me that B is a 1*3 gpuArray single row vector. However, on the other hand, the whos command shows no difference between A and B.
I am wondering if this double to single conversion will indeed help me reduce the memory usage of my GPU in matlab. Basically, my question is: when I move 2 variables on cpu, one is double and the other is single, to gpu, do they consume same amount of memory of GPU in matlab? whos command shows no difference.
Note the following:
A = gpuArray([1:1000])
whos A
Name Size Bytes Class Attributes
A 1x1000 4 gpuArray
Interesting! Only 4 bytes!
But this has an easy explanation: whos is only giving you the size of the variable on CPU RAM. Its 4 bytes because its just a memory address, not the data itself. The data is on the GPU, and it can not "easily" be accessed by the CPU.
Answering your question: Yes, single will take half of the memory of double on the GPU.
Related
I am trying to store a matrix of size 4 x 10^6, but the Matlab can't do it when running it, it's like it can't store a matrix with that size or I should use another way to store. The code is as below:
matrix = [];
for j = 1 : 10^6
x = randn(4,1);
matrix = [matrix x];
end
The problem it still running for long time and can't finish it, however when I remove the line matrix = [matrix x]; , it finishes the loop very quickly. So what I need is to have the matrix in file so that I can use it wherever I need.
It is determined by your amount of available RAM. If you store double values, like here, you require 64 bits per number. Thus, storing 4M values requires 4*10^6*64 = 256M bits, which in turn is 32MB RAM.
A = rand(4,1e6);
whos A
Name Size Bytes Class Attributes
A 4x1000000 32000000 double
Thus you only cannot store this if you have less than 32MB RAM free.
The reason your code takes so long, is because you grow your matrix in place. The orange wiggles on the line matrix = [matrix x]; are not because the festive season is almost here, but because it is very bad practise to do this. As the warning tells you: preallocate your matrix. You know how large it will be, so just initialise it as matrix = zeros(4,1e6); instead of growing it.
Of course in this case you can simply do matrix = rand(4,1e6), which is even faster than looping.
For more information about preallocation see the official MATLAB documentation, this question (which I answered), or this one.
Assume a simple example where I have indices
index_pos = [3,4,5];
index_neg = [1,2];
I would like to have a matrix:
result =
1 3
2 3
1 4
2 4
1 5
2 5
For this purpose I write the following code:
[X,Y] = meshgrid(index_pos,index_neg);
result = [Y(:) X(:)];
I think this is not a very efficient way. Also, this uses too much of my memory when I use big instances. I get the following error:
Error using repmat
Out of memory. Type "help memory" for your options.
Error in meshgrid (line 58)
xx = repmat(xrow,size(ycol));
Error in FME_funct (line 36)
[X,Y] = meshgrid(index_pos,index_neg);
Is there any 'clever' way to generate this matrix using less memory?
PS: I noticed that what I do is also given here. Most probably I have found this idea from there.
This depends entirely on how big your two variables are in relation to the amount of memory in your computer (plus the types of numbers you're using).
Try this:
res = zeros(numel(index_neg)*numel(index_pos), 2)
If that gives you an out-of-memory error then you don't have enough memory in your computer to store the result, regardless of the efficiency of the generator, so if the above errors, then you're stuck. If it does not error, then you could well write a looping algorithm that uses less temporary memory.
That said, by default MATLAB represents numbers with double precision, 8 bytes per number. If your index_ variables happen to contain, say, only positive integers (all less than 65,536) then you could use 16-bit unsigned integers. These are just 2 bytes per number and so take up 4 times less space than doubles. You can test this with:
res = zeros(numel(index_neg)*numel(index_pos), 2, 'uint16')
Finally you can find out how much memory is available to MATLAB with the memory command.
Here is a faster way to generate such a matrix. It avoids explicit temporary arrays by building the matrix directly in place,
res2 = [ reshape( bsxfun( #times , index_neg.' , ones(size(index_pos)) ) , [] , 1 ) , ...
reshape( bsxfun( #times , index_pos , ones(size(index_neg)).' ) , [] , 1 ) ] ;
Note that this require the same amount of memory to hold the main array, so it will not be possible to generate arrays larger than with your method (which fails at the meshgrid stage). This maximum size is ultimately dictated by the amount of RAM available to your system.
I am having Out of Memory error while trying to solve a certain linear equation (I will put the code below). Since I am used to coding in C where you have every control over the objects you create I am wondering if I am using matlab inefficiently. Here is the relevant part of the code
myData(n).AMatrix = sparse(fscanf(fid2, '%f', [2*M, 2*M]));
myData(n).AMatrix = transpose(myData(n).AMatrix);
%Read the covariance^2 matrix
myData(n).CovMatrix = sparse(fscanf(fid2, '%f', [2*M,2*M]));
myData(n).CovMatrix = reshape(myData(n).CovMatrix, [4*M*M,1]);
%Kronecker sum of A with itself
I=sparse(eye(2*M));
myData(n).AA=kron( I, myData(n).AMatrix)+kron( myData(n).AMatrix,I);
myData(n).AMatrix=[];
I=[];
%Solve (A+A)x = Vec(CovMatrix)
x=myData(n).CovMatrix\myData(n).AA;
Trying to use this code I get the error
Error using \
Out of memory. Type HELP MEMORY for your options.
Error in COV (line 62)
x=myData(n).CovMatrix\myData(n).AA;
Before this piece of code I only open some files (which contain two 100x100 array of floats) so I dont think they contribute to this error. The element AMatrix is a 100 x 100 array. So the linear equation in question has dimensions 10000 x 10000. Also AA has one dimensional kernel, I dont know if this affects the numerical computations. Later I project the obtained solution to the orthogonal complement of the kernel to get the "good" solution but it comes after the error. For people who are familiar with it this is just a solution to the Lyapunov equation AX + XA = Cov. The matrix A is sparse, it has 4 50x50 sublocks one of which is all zeros, the other is identity, the other is diagonal and the other has less than 1000 non-zero elements. The matrix CovMatrix is diagonal with 50 non-zero elements in the diagonal.
The problem is at the moment I can only do the calculations on a small personal computer with 2GB RAM with 2.5-6GB of virtual memmory. When I run memmory on matlab it gives
>> memory
Maximum possible array: 311 MB (3.256e+08 bytes) *
Memory available for all arrays: 930 MB (9.749e+08 bytes) **
Memory used by MATLAB: 677 MB (7.102e+08 bytes)
Physical Memory (RAM): 1931 MB (2.025e+09 bytes)
I am not very knowledgable when it comes to memory so I am open to even simple advices. Thanks.
Complex functions usually allocate temp memory during computation. 10000x10000 looks quite large if a temp dense matrix of such size is allocated during the computation. You could try a few smaller problem sizes and find out the upper limit of your current computer.
Today I have the need to allocate a vector with size 100000 in Matlab. I try to do it simply using:
a=ones(100000);
which my Matlab angrily answered with:
Out of memory. Type HELP MEMORY for your options.
Which is strange since I have Matlab 64 bit running on a 64 bit machine with 8 GB RAM. I tried many of the "resolving out of memory errors in Matlab" recipe in SO or other places but no luck so far.
Now I'm more confused when something like:
a=ones(10000,10000);
Runs without problem in my machine.
Does this mean that Matlab have some mechanism to limit the number of elements of a vector in a single-dimensional space?
Today I have the need to allocate a vector with size 100000 in Matlab.
Now, as noted in the comments and such, the method you tried (a=ones(100000);) creates a 100000x100000 matrix, which is not what you want.
I would suggest you try:
a = ones(1, 100000);
Since that creates a vector rather than a matrix.
Arguments Matter
Calling Matlab's ones() or zeros() or magic() with a single argument n, creates a square matrix with size n-by-n:
>> a = ones(5)
a = 1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
Calling the same functions with 2 arguments (r, c) instead creates a matrix of size r-by-c:
>> a = ones(2, 5)
a = 1 1 1 1 1
1 1 1 1 1
This is all well documented in Matlab's documentation.
Size Matters Too
Doubles
Having said this, when you do a = zeros(1e6) you are creating a square matrix of size 1e6 * 1e6 = 1e12. Since these are doubles the total allocated size would be 8 * 1e12 Bytes which is circa (8 * 1e12) / 1024^3 = 7450.6GB. Do you have this much RAM on your machine?
Compare this with a = zeros(1, 1e6) which creates a column-vector of size 1 * 1e6 = 1e6, for a total allocated size of (8 * 1e6) / 1024^3 = 7.63MB.
Logicals
Logical values, on the other hand are boolean values, which can be set to either 0 or 1 representing False or True. With this in mind, you can allocate matrices of logicals using either false() or true(). Here the same single-argument rule applies, hence a = false(1e6) creates a square matrix of size 1e6 * 1e6 = 1e12. Matlab today, as many other programming languages, stores bit values, such as booleans, into single Bytes. Even though there is a clear cost in terms of memory usage, such a mechanism provides significant performance improvements. This is because it is accessing single bits is a slow operation.
The total allocated size of our a = false(1e6) matrix would therefore be 1e12 Bytes which is circa 1e12 / 1024^3 = 931.32GB.
Well the first declaration tries to build a matrix of 1000000x1000000 ones. That would be ~931 GB.
The second tries to declare a matrix of 10000 x 10000. That would be ~95MB.
I assumed each one is stored on a byte. If they use floats, than the requested memory size will be 4 times larger.
I am developing a program to read a time series of NIfTY format images to a 4D matrix in MATLAB. There are about 60 images in the stack and the program runs without problems until the 28th image. (All the images are approximately same size, same details) But after that the reading get slower and slower.
In fact, the delay is accumulating.
I checked the program again and there are no open files. Everything looks fine.
Can someone give me an advice?
Size of current array (double)
Unless you are running on a machine with more than ~20GB RAM memory your matrix simply becomes too large to handle.
To check the size of the first three dimensions of your matrix:
A = rand(512,512,160);
whos('A')
Output:
Name Size Bytes Class Attributes
A 512x512x160 335544320 double
Now multiply by 60 to obtain the size of your 4D matrix and divide by 1024^3 to obtain GB's:
335544320*60/1024^3 = 18.7500 GB
So yes, your matrix is most likely too large to handle efficiently/effectively.
A matrix exceeding your RAM memory forces MatLab to use the swap file (HDD/SSD) which is orders of magnitude slower than your random access memory (even if you have a SSD).
Switch to different data types
I you do not require double precision, i.e. 16 digits of accuracy, you can always switch to less digits, i.e. single precision floating point numbers. By doing this you can reduce size. You can even reduce size further is the numbers are for example unsigned integers in the range of 0-255. See code below:
% Create doubles
A_double = rand(512,512,160);
S1=whos('A_double');
% Create floats
A_float = single(A_double);
S2=whos('A_float');
% Create unsigned int range 0-255
A_uint=uint8(randi(256,[512,512,160])-1);
S3=whos('A_uint');
fprintf('Size A_double is %4.2f GB\n',(S1.bytes*60)/1024^3)
fprintf('Size A_float is %4.2f GB\n',(S2.bytes*60)/1024^3)
fprintf('Size A_uint is %4.2f GB\n',(S3.bytes*60)/1024^3)
Output:
Size A_double is 18.75 GB
Size A_float is 9.38 GB
Size A_uint is 2.34 GB
Which may just fit inside your RAM. Make sure you indeed pre-allocate memory first, i.e. create an empty matrix using the zeros() function.