I have two matrices, a1 and a2. a1 is 3x12000 and a2 is 3x4000. I want to create another array that is 3x4000 which is the left matrix division (mldivide, \) of the 3x3 sub-matrices of a1 and the 3x1 sub-matrices of a2. You can do this easily with a for loop:
for ii = 1:3:12000
a = a1(:,ii:ii+2)\a2(:, ceil(ii/3));
end
However, I was wondering if there was a faster way to do this.
Edit: I am aware preallocation increases speed, I was just showing that for visual purposes.
Edit2: Removed iterative increase of array. It seems my questions has been misinterpreted a bit. I was mainly wondering if there were some matrix operations I could do to achieve my goal as that would likely be quicker than a for loop i.e. reshape a1 to a 3x3x4000 matrix and a2 to a 3x1x4000 matrix and left matrix divide each level in one go, however, you can't left matrix divide with 3D matrices.
You can create one system of equations containing many independent 'sub-systems' of equations by putting the sub-matrices of a1 in a the diagonal of a 12000x12000 matrix like this:
a1(1,1) a1(1,2) a1(1,3) 0 0 0 0 0 0
a1(2,1) a1(2,2) a1(2,3) 0 0 0 0 0 0
a1(3,1) a1(3,2) a1(3,3) 0 0 0 0 0 0
0 0 0 a1(1,4) a1(1,5) a1(1,6) 0 0 0
0 0 0 a1(2,4) a1(2,5) a1(2,6) 0 0 0
0 0 0 a1(3,4) a1(3,5) a1(3,6) 0 0 0
0 0 0 0 0 0 a1(1,7) a1(1,8) a1(1,9)
0 0 0 0 0 0 a1(2,7) a1(2,8) a1(2,9)
0 0 0 0 0 0 a1(3,7) a1(3,8) a1(3,9)
and then left divide it by a2(:).
This can be done using kron and sparse matrix like this (source):
a1_kron = kron(speye(12000/3),ones(3));
a1_kron(logical(a1_kron)) = a1(:);
a = a1_kron\a2(:);
a = reshape(a, [3 12000/3]);
Advantage - Speed: This is about 3-4 times faster than a for loop with preallocation on my PC.
Disadvantage: There is one disadvantage you must consider with this approach: when using left division, Matlab looks for the best way to solve the systems of linear equations, so if you solve each sub-system independently, the best way will be chosen for each sub-system, but if you solve theme as one system, Matlab will find the best way for all the sub-systems together - not the best for each sub-system.
Note: As shown in Stefano M's answer, using one big system of equations (using kron and sparse matrix) is faster than using a for loop (with preallocation) only for very small size of sub-systems of equations (on my PC, for number of equation <= 7) for bigger sizes of sub-systems of equations, using a for loop is faster.
Comparing different methods
I wrote and ran a code to compare 4 different methods for solving this problems:
for loop, no preallocation
for loop, with preallocation
kron
cellfun
Test:
n = 1200000;
a1 = rand(3,n);
a2 = rand(3,n/3);
disp('Method 1: for loop, no preallocation')
tic
a_method1 = [];
for ii = 1:3:n
a_method1 = [a_method1 a1(:,ii:ii+2)\a2(:, ceil(ii/3))];
end
toc
disp(' ')
disp('Method 2: for loop, with preallocation')
tic
a1_reshape = reshape(a1, 3, 3, []);
a_method2 = zeros(size(a2));
for i = 1:size(a1_reshape,3)
a_method2(:,i) = a1_reshape(:,:,i) \ a2(:,i);
end
toc
disp(' ')
disp('Method 3: kron')
tic
a1_kron = kron(speye(n/3),ones(3));
a1_kron(logical(a1_kron)) = a1(:);
a_method3 = a1_kron\a2(:);
a_method3 = reshape(a_method3, [3 n/3]);
toc
disp(' ')
disp('Method 4: cellfun')
tic
a1_cells = mat2cell(a1, size(a1, 1), repmat(3 ,1,size(a1, 2)/3));
a2_cells = mat2cell(a2, size(a2, 1), ones(1,size(a2, 2)));
a_cells = cellfun(#(x, y) x\y, a1_cells, a2_cells, 'UniformOutput', 0);
a_method4 = cell2mat(a_cells);
toc
disp(' ')
Results:
Method 1: for loop, no preallocation
Elapsed time is 747.635280 seconds.
Method 2: for loop, with preallocation
Elapsed time is 1.426560 seconds.
Method 3: kron
Elapsed time is 0.357458 seconds.
Method 4: cellfun
Elapsed time is 3.390576 seconds.
Comparing the results of the four methods, you can see that using method 3 - kron gives slightly different results:
disp(['sumabs(a_method1(:) - a_method2(:)): ' num2str(sumabs(a_method1(:)-a_method2(:)))])
disp(['sumabs(a_method1(:) - a_method3(:)): ' num2str(sumabs(a_method1(:)-a_method3(:)))])
disp(['sumabs(a_method1(:) - a_method4(:)): ' num2str(sumabs(a_method1(:)-a_method4(:)))])
Result:
sumabs(a_method1(:) - a_method2(:)): 0
sumabs(a_method1(:) - a_method3(:)): 8.9793e-05
sumabs(a_method1(:) - a_method4(:)): 0
You are solving a series of N systems with m linear equations each, the N systems are of the form
Ax = b
You can convert these to a single system of Nm linear equations:
|A1 0 0 ... 0 | |x1| |b1|
|0 A2 0 ... 0 | |x2| |b2|
|0 0 A3 ... 0 | |x3| = |b3|
|. . . ... . | |. | |. |
|0 0 0 ... AN| |xN| |bN|
However, solving this one system of equations is a lot more expensive than solving all the little ones. Typically, the cost is O(n^3), so you go from O(N m^3) to O((Nm)^3). A huge pessimization. (Eliahu proved me wrong here, apparently the sparsity of the matrix can be exploited.)
Reducing the computational cost can be done, but you need to provide guarantees about the data. For example, if the matrices A are positive definite, the systems can be solved more cheaply. Nonetheless, given that you are dealing with 3x3 matrices, the winnings there will be slim, since those are pretty simple systems to solve.
If you are asking this because you think that loops are inherently slow in MATLAB, you should know that this is no longer the case, and hasn’t been the case since MATLAB gained a JIT 15 years or so ago. Today, many vectorization attempts lead to equally fast code, and oftentimes to slower code, especially for large data. (I could fish up some timings I’ve posted here on SO to prove this if necessary.)
I would think that solving all systems in one go could reduce the number of checks that MATLAB does every time the operator \ is called. That is, hard-coding the problem size and type might improve throughout. But the only way to do so is to write a MEX-file.
MarginalBiggest improvement would be to preallocate the output matrix, instead of growing it:
A1 = reshape(A1, 3, 3, []);
a = zeros(size(A2));
for i = 1:size(A1,3)
a(:,i) = A1(:,:,i) \ A2(:,i);
end
With the preallocate array, if the Parallel Toolbox is available, you can try parfor
Edit
This answer is no more relevant, since the OP rephrased the question to avoid growing the result array, which was the original major bottleneck.
The problem here is that one has to solve 4000 independent 3x3 linear systems. The matrix is so small that an ad hoc solution could be of interest, especially if one has some information on the matrix properties (symmetric, or not, condition number, etc.). However sticking to the \ matlab operator, the best way to speed up computations is by explicitly leverage of parallelism, e.g. by the parfor command.
The sparse matrix solution of the other answer by Eliahu Aaron is indeed very clever, but its speed advantage is not general but depends on the specific problem size.
With this function you can explore different problem sizes:
function [t2, t3] = sotest(n, n2)
a1 = rand(n,n*n2);
a2 = rand(n,n2);
tic
a1_reshape = reshape(a1, n, n, []);
a_method2 = zeros(size(a2));
for i = 1:size(a1_reshape,3)
a_method2(:,i) = a1_reshape(:,:,i) \ a2(:,i);
end
t2 = toc;
tic
a1_kron = kron(speye(n2),ones(n));
a1_kron(logical(a1_kron)) = a1(:);
a_method3 = a1_kron\a2(:);
a_method3 = reshape(a_method3, [n n2]);
t3 = toc;
assert ( norm(a_method2 - a_method3, 1) / norm(a_method2, 1) < 1e-8)
Indeed for n=3 the sparse matrix method is clearly superior, but for increasing n it becomes less competitive
The above figure was obtained with
>> for i=1:20; [t(i,1), t(i,2)] = sotest(i, 50000); end
>> loglog(1:20, t, '*-')
My final comment is that an explicit loop with the dense \ operator is indeed fast; the sparse matrix formulation is slightly less accurate and could become problematic in edge cases; and for sure the sparse matrix solution is not very readable. If the number n2 of systems to solve is very very big (>1e6) then maybe ad hoc solutions should be explored.
I want to change the rounding mode for floating point operations in MATLAB. According to IEEE 754-2008, there are 5 strategies for rounding:
round to nearest, ties to even
round to nearest, ties away from zero
round toward zero
round up (toward positive infinity)
round down (toward negative infinity)
Does MATLAB supports these 5 strategies? How to change the rounding mode for floating point operations in MATLAB?
Answer
Kind of. There is an undocumented feature('setround') function call that you can use to get or set the rounding mode used by Matlab.
So, it can be done, but you shouldn’t do it. :)
WARNING: This is an undocumented, unsupported feature! Use at your own peril!
This feature('setround') supports 4 of the 5 IEEE-754 rounding modes: there’s only one “nearest” mode, and I don't know if it’s “ties to even” or “ties away from zero”.
Supported modes:
feature('setround') – Get current rounding mode
feature('setround', 0.5) – Round toward nearest (don’t know if it’s ties to even or away from zero)
feature('setround', Inf) – Round up (towards +Inf)
feature('setround', 0) – Round toward zero
feature('setround', -Inf) – Round down (towards -Inf)
Note on testing: The IEEE-754 rounding mode does not affect round() and its relatives. Rather, it governs how arithmetic operations behave around the limits of floating-point precision.
Demonstration
%ROUNDINGEXAMPLE Demonstrates IEEE-754 Rounding Mode control
%
% This uses a completely undocumented and unsupported feature!
% Not for production use!
%% Setup
clear; clc
n = 2000;
X = ones(n)*1E-30; % matrix with n^2 elements
defaultRoundingMode = feature('setround'); % store default rounding mode
%%
feature('setround',0.5);
r1 = prettyPrint('Nearest', sum(X(:)));
%{
sign exponent mantissa
0 01110110001 0011010101111100001010011001101001110101010000011110
| \_________/ \__________________________________________________/
| | ______________________|___________________________
| | / \
(-1)^0 2^( 945 - 1023) 1.0011010101111100001010011001101001110101010000011110 = 4e-24
%}
%%
feature('setround',-Inf);
r2 = prettyPrint('To -Infinity', sum(X(:)));
%{
sign exponent mantissa
0 01110110001 0011010101111100001010011001101001011100000111000110
| \_________/ \__________________________________________________/
| | ______________________|___________________________
| | / \
(-1)^0 2^( 945 - 1023) 1.0011010101111100001010011001101001011100000111000110 = 4e-24
%}
%%
feature('setround',Inf);
r3 = prettyPrint('To Infinity', sum(X(:)));
%{
sign exponent mantissa
0 01110110001 0011010101111100001010011001101010100011101100100001
| \_________/ \__________________________________________________/
| | ______________________|___________________________
| | / \
(-1)^0 2^( 945 - 1023) 1.0011010101111100001010011001101010100011101100100001 = 4e-24
%}
%%
feature('setround',0);
r4 = prettyPrint('To zero', sum(X(:)));
%{
sign exponent mantissa
0 01110110001 0011010101111100001010011001101001011100000111000110
| \_________/ \__________________________________________________/
| | ______________________|___________________________
| | / \
(-1)^0 2^( 945 - 1023) 1.0011010101111100001010011001101001011100000111000110 = 4e-24
%}
%%
feature('setround',defaultRoundingMode);
r5 = prettyPrint('No accumulated roundoff error', 4e-24);
%{
sign exponent mantissa
0 01110110001 0011010101111100001010011001101010001000111010100111
| \_________/ \__________________________________________________/
| | ______________________|___________________________
| | / \
(-1)^0 2^( 945 - 1023) 1.0011010101111100001010011001101010001000111010100111 = 4e-24
%}
%% Helper function
function r = prettyPrint(s, r)
fprintf('%s:\n%65.60f\n\n', s, r);
end
I get:
Nearest:
0.000000000000000000000003999999999966490758963870373537264729
To -Infinity:
0.000000000000000000000003999999999789077070014108839608005726
To Infinity:
0.000000000000000000000004000000000118618095059505975310731249
To zero:
0.000000000000000000000003999999999789077070014108839608005726
No accumulated roundoff error:
0.000000000000000000000003999999999999999694801998206811298525
Acknowledgments
Thanks to Ryan Klots at MathWorks Technical Support for setting me straight on this and providing the nice demo code!
#HYF: I found that feature('setround', 0.5) leads to round to even.
I checked the following:
a=1+2^-52
This means that mantissa looks like so: 1.0...01 where the last 1 is still in the mantissa. The leading 1 is not stored in IIIE754 format.
( I checked that 1+2^-52 == 1 but not 1+2^-53 == 1)
Then I computed b = a + 2^-53.
Without rounding it is 1.0...01|1 where the last digit is to be rounded away.
I found the following true: b==1+2^-51
We have b == 1.0...010.
We have several rounding modes submodes of round to nearest:
This can be round to inf, round away from 0 or round to even.
Next we check -b==-1-2^-51 to be true which excludes round to inf
still allowing round away from 0 or round to even.
Then I checked 1.5==1.5+2^-53.
Of course 1.5 = 1.10...0 binary and
and 1.5+2^-53 = 1.10...0|1 without rounding, the last digit to be rounded away.
Rounding away from 0 would be 1.10...01
and rounding to even is 1.10...0.
So the latter is the case.
I'm using the following code to get specgram2D from np array:
specgram2D, freq, time = mlab.specgram(samples, Fs=11025, NFFT=1024, window=mlab.window_hanning, noverlap=int(1024 * 0.5))
Then I print out specgram2D like
print len(specgram2D) # returns 513
I got 513 instead of expected 512 which is half the window size.
What am I doing wrong?
Can I just ignore specgram2D[512]?
I got 513 instead of expected 512 which is half the window size.
What am I doing wrong?
For a real-valued signal, the frequency spectrum obtained from the Discrete Fourier Transform (DFT) is symmetric and hence only half of the spectrum is necessary to describe the entire spectrum (since the other half can be obtained from symmetry). That is probably why you are expecting the size to be exactly half the input window size of 1024.
The problem is that with even sized inputs, the midpoint of the spectrum falls exactly on a frequency bin. As a result, that frequency bin is its own symmetry. To illustrate this, the symmetry can be seen from the following graph:
frequency: 0 fs/N ... fs/2 ... fs
bin number: 0 1 ... 511 512 513 ... 1023 1024
^ ^ ^ ^ ^ ^ ^ ^
| | | |-| | | |
| | | | | |
| | |--------| | |
| | | |
| |----------------------------| |
| |
|--------------------------------------|
Where N is the size of the FFT (as determined by the NFFT=1024 parameter) and fs is the sampling frequency. As you can see the spectrum is fully specified by taking bins 0 to 512, inclusive. Correspondingly you should be expecting the size to be floor(N/2)+1 (simply N/2 + 1 with integer division, but I included the floor to emphasis the round down operation), or 513 in your case.
Can I just ignore specgram2D[512]?
As previously shown it is an integral part of the spectrum, but many applications do not specifically require every single frequency bins (i.e. ignoring that bin depends on whether your application is mostly interested in other frequency components).
According to my previous question, I want to optimize an objective function using binary integer linear programming (all of variables are binary) as follows:
Minimize f = (c1*x1) + (c2*x2) + MAX((c3*x3),(c4*x4)) + (c5*x5)
Subject to: some equality and inequality constraints
For MAX operator, I used auxiliary variable x6 and added x6>=(c3*x3) and x6>=(c4*x4) constraints to the problem so the problem turn into:
Minimize f = (c1*x1) + (c2*x2) + x6 + (c5*x5), with added constraints.
I used CPLEX API for MATLAB to optimize the objective function.
Because all of variables are binary except one ((x6) which is continuous) and coefficients have double values, the problem turn into mixed integer linear programming so I used cplexmilp function with this configuration:
Variable types: ctype='BBBBBC' ( B:Binary, C:Continuous)
Lower Bounds: lb=[0 0 0 0 0 0]
Upper Bounds: ub=[0 0 0 0 0 inf]
Function Call:
[fval] = cplexmilp(f, Aineq, bineq, Aeq, beq,[],[],[],lb,ub,ctype)
but sometimes in the results I see x3 and x4 have continuous values(between 0 and 1) and x3+x4=1.
So my questions are:
Can any one tell me what's wrong with x3 and x4?
Is there a solution for not using auxiliary variable and solving this optimization problem with cplexbilp ?
Thanks in advance
[UPDATE]:
One parts of my code had logical errors which I fixed them, now in all cases which x3 and x4 are not binary we have x3*(1-x3)< 10^(-5), x4*(1-x4)< 10^(-5) and x3+x4=1, so #David Nehme were right (according to his useful comment), but my second question still remains here!
David's solution shows you why your formulation has become linearized but non-binary. You could also try printing out the problem in LP or MPS format to see all the resulting constraints.
You asked about a formulation that continues to be purely binary. Here's one way to do that:
Transforming this to a purely binary formulation
Here's a way to keep the problem with Max() also binary. It involves additional aux. variables, but it is relatively straight-forward once you apply the standard if-then IP tricks.
First, let's list out the four possible cases in a simple table, and see what values the max() term can take. These cases are mutually exclusive.
x3 | x4 | max (c3.x4, c4.x3)
-------------------------------
0 | 0 | 0
1 | 0 | c3
0 | 1 | c4
1 | 1 | max(c3, c4) - a constant
Now, let C34 be the max(c3, c4). Note that C34 is a number, not a variable in the problem. We need this for the new Objective function.
Introducing new binary variables
For each of the four cases above, let's introduce an auxiliary BINARY variable. For clarity, call them y0, y3, y4, y34.
Only one of the cases in the table above can hold, so we add:
y0 + y3 + y4 + y34 = 1
yi are BINARY
Now, all that remains is to add linking constraints that ensure:
If x3=0 AND x4=0 then y0=1
If x3=1 AND x4=0 then y3=1
If x3=0 AND x4=1 then y4=1
If x3=1 AND x4=1 then y34=1
We can ensure that by adding a pair of linear constraints for each of the conditions above.
2 y0 <= (1- x3) + (1 -x4)
(1-x3) + (1-x4) <= y0 + 1
2 y3 <= x3 + (1-x4)
x3+(1-x4) <= y3 + 1
2 y4 <= x4 + (1-x3)
x4+(1-x3) <= y4 + 1
2 y34 <= x3 + x4
x3+x4 <= y34 + 1
The new objective function now becomes:
Minimize f = (c1*x1) + (c2*x2) + (c5*x5) + 0*Y0 + C3*Y3 + C4*Y4 + C34*Y34
Notice that we don't have the Max() term anymore in the objective function. And all x and y variables are binary. All your original constraints plus the new ones above (8+1 = 9 of them) should be included. Once you do that, you can use cplexbilp because it is a pure BILP problem.
Hope that helps.
In your case, the auxiliary variable x6 is needed because the "MAX" function is not linear (it has a discontinuous gradient where c3*x3 == c4*x4). By adding the additional variable and constraints you are creating a linear version of the problem with a solution that is equivalent to your original nonlinear problem. The trade-off is that if c3 or c4 have a value other than 0 or 1, then your problem is not a pure binary problem. That is a very good trade-off, especially if you are using a good MIP solver.
I'm trying to simulate the following distribution:
a | 0 | 1 | 7 | 11 | 13
-----------------------------------------
p(a) | 0.34 | 0.02 | 0.24 | 0.29 | 0.11
I already simulated a similar problem: four type of balls with chances of 0.3, 0.1, 0.4 and 0.2. I created a vector F = [0 0.3 0.4 0.8 1] and used repmat to grow it by 1000 rows. Then I compared it with a columnvector of 1000 random numbers grown with 5 columns using the same repmat approach. I compared those two, calculated the sumvector of the matrix, and calculated the difference to get the frequences (e.g. [301 117 386 196]). .
But with the current distribution I don't know how to create the initial matrix F and whether I can use the same approach I used before at all.
I need the answer to be "vectorised", so no (for, while or if) loops.
This question on math.stackexchange
What if you create the following arrays:
largeNumber = 1000000;
a=repmat( [0], 1, largeNumber*0.34 );
b=repmat( [1], 1, largeNumber*0.02 );
% ...
e=repmat( [13], 1, largeNumber*0.11 );
Then you concatenate all of these arrays (to get a single array where your entries are represented with their corresponding probabilities), shuffle them, and extract the first N elements to get an N-dimensional vector drawn from your distribution.
EDIT: of course this answer is the way to go.