I stumbled across an unexpected behavior of the addition operator.
a=[1 2 3];
b=[5; 7; 11];
a+b
Produces
ans =
6 7 8 % a(1)+b(1) a(1)+b(2) a(1)+b(3)
8 9 10 % a(2)+b(1) a(2)+b(2) a(2)+b(3)
12 13 14 % a(3)+b(1) a(3)+b(2) a(3)+b(3)
This behavior probably falls under the Add a Vector to a Matrix example in the documentation.
Create an array, A, and add a column vector to it. The vector is
treated as though it is a matrix of the same size as A, so that each
element in the vector is added to a row in A.
Although, b in my example cannot be treated as the same size as a. Both vectors must be duplicated.
What can I call this behavior when I am describing it to others? None of the definitions of Matrix Addition fit. Vector addition also has a different definition. The best I could come up with was the "sum of the elements in the cartesian product".
That's called binary singleton expansion. Starting with R2016b, MATLAB does it automatically with standard operators. See bsxfun for more details.
Today I was quite surprised by this:
>> M = [0, 0, 0;6, 1, 3;1, 7, 0];
>> rank(M)
ans =
3
>> rank(M')
ans =
2
I'm aware of the fact that the rank function is not necessarily numerically stable since it thresholds the singular values. I was however expecting problems to happen for matrices that are either large in size or large in elements and not a 3 by 3 matrix of small integers.
I checked what happens and in fact svd(M) gives singular values 7.82, 5.93, 2.91e-15, while the default tolerance is only max(size(A))*eps(max(s)) = 2.665e-15. On the other hand, svd(M') gives 0 as third singular values (probably due to a whole column being zero).
Of course I can manually increase the tolerance in calling rank, but how would I know how far to increase it?
Is there another numerically stable method to compute the rank (say that we know that the matrix is integer)?
edit: I just found that this behavior is version-dependent. The above test was carried out with Matlab 2014a. On Matlab 2016b, svd(M) returns the third singular value as 4.15e-16 and rank works properly. So maybe there was indeed an issue with svd that was fixed between version. Still, I'm not sure anymore how far I can trust rank, so I believe my question remains valid.
Matlab 2015a/2015b seem to work (see below)
>> M = [0, 0, 0;6, 1, 3;1, 7, 0];
>> rank(M)
ans =
2
>> rank(M')
ans =
2
>>
H = [1 2; 3 4; 5 6; 7 8; 9 10; 11 12; 13 14; 15 16];
X = [7; 9];
Y = H*X;
H1 = Y/X;
This is my code. As you can see, I was trying to get back the H values. However, it gave me something else. I have tried to use inv() but this is not possible because X is not a square matrix.
You can't get a value of rank 2 back by dividing a value of rank 1. The system is underconstrained.
Both mrdivide and pinv (for pseudo-inverse) can be used to get a solution to the system. Because there are multiple solutions, it wouldn't necessary be the one you started with. Instead you'll get a "simplest" solution, either in the sense of lowest cardinality or lowest 2-norm, depending on whether you use mrdivide or pinv.
Here, the pinv documentation page probably explains it more precisely than I can. Just note it is discussing X\Y instead of Y/X:
If A has more rows than columns and is not of full rank, then the overdetermined least squares problem
minimize norm(A*x-b)
does not have a unique solution. Two of the infinitely many solutions are
x = pinv(A)*b
and
y = A\b
These two are distinguished by the facts that norm(x) is smaller than the norm of any other solution and that y has the fewest possible nonzero components.
In short, my question is:
Is a double in Matlab really a double, or is it a class with the additional property to act as an integer?
And here is the context and motivation for the question :)
>> 1:4
ans =
1 2 3 4
>> class(ans)
ans =
double
Just doing this creates a double...
>> 1.00:4.00
ans =
1 2 3 4
>> class(ans)
ans =
double
...as does this, even though it's printed as integers.
The floating point nature of the numbers only shows when greater numerical uncertainty is introduced.
>> acosd(cosd(1:4))
ans =
0.999999999999900 1.999999999999947 3.000000000000045 4.000000000000041
Is a double in Matlab really a double, or is it a class with the additional property to act as an integer?
A vector defined with "integers" (which of course really is doubles), it can be used to index another vector, which is usually a property of integers.
>> A = [9 8 7 6]
A =
9 8 7 6
>> idx = [4 3 2 1]
idx =
4 3 2 1
>> class(idx)
ans =
double
>> A(idx)
ans =
6 7 8 9
I also tried A(acosd(cosd(1:4))) which does not work.
It's just a double, but your command prompt format gives you the most compact view. Specifically,
format short
But you can change it to always display decimals, and lots of them, with
format longEng
There are many other options on the format help page.
Interestingly, you can use non-integer numbers as indexes with the colon operator, but it will warn. I would take this warning seriously as this indexing behavior is odd.
As I mentioned in my comments, the reason it is OK for MATLAB to use doubles for indexing has to do with the largest value of an integer that can be specified without losing precision in MATLAB. Double precision (64-bit) floating point numbers can exactly represent integers up to 2^53 (9,007,199,254,740,992) without losing any accuracy. The maximum array size allowed by MATLAB is far below this number, so there is no risk of indexing errors as a result of floating point precision.
In MATLAB, all numeric literals (i.e. numbers in the text of your program) are interpreted as double-precision. You must cast them explicitly to get any other type. It's worth remembering that IEEE floating point can exactly represent a wide range of integer values, up to FLINTMAX.
I have 12 sets of vectors (about 10-20 vectors each) and i want to pick one vector of each set so that a function f that takes the sum of these vectors as argument is maximized. In addition i have constraints for some components of that sum.
Example:
a_1 = [3 2 0 5], a_2 = [3 0 0 2], a_3 = [6 0 1 1], ... , a_20 = [2 12 4 3]
b_1 = [4 0 4 -2], b_2 = [0 0 1 0], b_3 = [2 0 0 4], ... , b_16 = [0 9 2 3]
...
l_1 = [4 0 2 0], l_2 = [0 1 -2 0], l_3 = [4 4 0 1], ... , l_19 = [3 0 9 0]
s = [s_1 s_2 s_3 s_4] = a_x + b_y + ... + l_z
Constraints:
s_1 > 40
s_2 < 100
s_4 > -20
Target: Chose x, y, ... , z to maximize f(s):
f(s) -> max
Where f is a nonlinear function that takes the vector s and returns a scalar.
Bruteforcing takes too long because there are about 5.9 trillion combinations, and since i need the maximum (or even better the top 10 combinations) i can not use any of the greedy algorithms that came to my mind.
The vectors are quite sparse, about 70-90% are zeros. If that is helping somehow ...?
The Matlab Optimization toolbox didnt help either since it doesnt much support for discrete optimization.
Basically this is a lock-picking problem, where the lock's pins have 20 distinct positions, and there are 12 pins. Also:
some of the pin's positions will be blocked, depending on the positions of all the other pins.
Depending on the specifics of the lock, there may be multiple keys that fit
...interesting!
Based on Rasman's approach and Phpdna's comment, and the assumption that you are using int8 as data type, under the given constraints there are
>> d = double(intmax('int8'));
>> (d-40) * (d+100) * (d+20) * 2*d
ans =
737388162
possible vectors s (give or take a few, haven't thought about +1's etc.). ~740 million evaluations of your relatively simple f(s) shouldn't take more than 2 seconds, and having found all s that maximize f(s), you are left with the problem of finding linear combinations in your vector set that add up to one of those solutions s.
Of course, this finding of combinations is no easy feat, and the whole method breaks down anyway if you are dealing with
int16: ans = 2.311325368800510e+018
int32: ans = 4.253529737045237e+037
int64: ans = 1.447401115466452e+076
So, I'll discuss a more direct and more general approach here.
Since we're talking integers and a fairly large search space, I'd suggest using a branch-and-bound algorithm. But unlike the bintprog algorithm, you'd have to use different branching strategies, and of course, these should be based on a non-linear objective function.
Unfortunately, there is nothing like this in the optimization toolbox (or the File Exchange as far as I could find). fmincon is a no-go, since it uses gradient and Hessian information (which will usually be all-zero for integers), and fminsearch is a no-go, since you'll need a really good initial estimate, and the rate of convergence is (roughly) O(N), meaning, for this 20-dimensional problem you'll have to wait quite long before convergence, without the guarantee of having found the global solution.
An interval method could be a possibility, however, I personally have very little experience with this. There is no native interval-related stuff in MATLAB or any of its toolboxes, but there's the freely available INTLAB.
So, if you're not feeling like implementing your own non-linear binary integer programming algorithm, or are not in the mood for an adventure with INTLAB, there's really only one thing left: heuristic methods. In this link there is a similar situation, with an outline of the solution: use the genetic algorithm (ga) from the Global Optimization toolbox.
I would implement the problem roughly like so:
function [sol, fval, exitflag] = bintprog_nonlinear()
%// insert your data here
%// Any sparsity you may have here will only make this more
%// *memory* efficient, not *computationally*
data = [...
... %// this will be an array with size 4-by-20-by-12
... %// (or some permutation of that you find more intuitive)
];
%// offsets into the 3D array to facilitate indexing a bit
offsets = bsxfun(#plus, ...
repmat(1:size(data,1), size(data,3),1), ...
(0:size(data,3)-1)' * size(data,1)*size(data,2)); %//'
%// your objective function
function val = obj(X)
%// limit "X" to integers in [1 20]
X = min(max(round(X),1),size(data,3));
%// "X" will be a collection of 12 integers between 0 and 20, which are
%// indices into the data matrix
%// form "s" from "X"
s = sum(bsxfun(#plus, offsets, X*size(data,1) - size(data,1)));
%// XxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxX
%// Compute the NEGATIVE VALUE of your function here
%// XxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxX
end
%// your "non-linear" constraint function
function [C, Ceq] = nonlcon(X)
%// limit "X" to integers in [1 20]
X = min(max(round(X),1),size(data,3));
%// form "s" from "X"
s = sum(bsxfun(#plus, offsets, X(:)*size(data,1) - size(data,1)));
%// we have no equality constraints
Ceq = [];
%// Compute inequality constraints
%// NOTE: solver is trying to solve C <= 0, so:
C = [...
40 - s(1)
s(2) - 100
-20 - s(4)
];
end
%// useful GA options
options = gaoptimset(...
'UseParallel', 'always'...
...
);
%// The rest really depends on the specifics of the problem.
%// Useful to look at will be at least 'TolCon', 'Vectorized', and of course,
%// 'PopulationType', 'Generations', etc.
%// THE OPTIMZIATION
[sol, fval, exitflag] = ga(...
#obj, size(data,3), ... %// objective function, taking a vector of 20 values
[],[], [],[], ... %// no linear (in)equality constraints
1,size(data,2), ... %// lower and upper limits
#nonlcon, options); %// your "nonlinear" constraints
end
Note that even though your constraints are essentially linear, the way by which you must compute the value for your s necessitates the use of a custom constraint function (nonlcon).
Especially note that this is currently (probably) a sub-optimal way to use ga -- I don't know the specifics of your objective function, so a lot more may be possible. For instance, I currently use a simple round() to convert the input X to integers, but using 'PopulationType', 'custom' (with a custom 'CreationFcn', 'MutationFcn' etc.) might produce better results. Also, 'Vectorized' will likely speed things up a lot, but I don't know whether your function is easily vectorized.
And yes, I use nested functions (I just love those things!); it prevents these huge, usually identical lists of input arguments if you use sub-functions or stand-alone functions, and they can really be a performance boost because there is little copying of data. But, I realize that their scoping rules make them somewhat akin to goto constructs, and so they are -ahum- "not everyone's cup of tea"...you might want to convert them to sub-functions to prevent long and useless discussions with your co-workers :)
Anyway, this should be a good place to start. Let me know if this is useful at all.
Unless you define some intelligence on how the vector sets are organized, there will be no intelligent way of solving your problem other then pure brute force.
Say you find s s.t. f(s) is max given constraints of s, you still need to figure out how to build s with twelve 4-element vectors (an overdetermined system if there ever was one), where each vector has 20 possible values. Sparsity may help, although I'm not sure how it is possible to have a vector with four elements be 70-90% zero, and sparsity would only be useful if there was some yet to be described methodology in how the vector are organized
So I'm not saying you can't solve the problem, I'm saying you need to rethink how the problem is set-up.
I know, this answer is reaching you really late.
Unfortunately, the problem, as is, show not many patterns to be exploited, besides of brute force -Branch&Bound, Master& Slave, etc.- Trying a Master Slave approach -i.e. solving first the function continuous nonlinear problem as master, and solving the discrete selection as slave could help, but with as many combinations, and without any more information over the vectors, there is not too much space for work.
But based on the given continuous almost everywhere functions, based on combinations of sums and multiplication operators and their inverses, the sparsity is a clear point to be exploited here. If 70-90% of vectors are zero, almost a good part of the solution space will be close to zero, or close to infinite. Hence a 80-20 pseudo solution would discard easily the 'zero' combinations, and use only the 'infinite' ones.
This way, the brute-force could be guided.