Optimization under constraints - matlab

I have a question regarding optimization.
I have a matrix x with 3 columns and a certain number of rows (max 200). Each row represents a candidate. The column one contains a score (between 0 and 1) , the column 2 contains the kind of candidate (there are 10 kinds in total labeled from 1 to 10) and the column 3 contains the amount of each candidate. There is one thing to take into consideration: the amount can be NEGATIVE
What I would like to do is to select max 35 elements among these candidates which would maximize the function which sum over their respective score (column 1) under the constraints that there can be a maximum of 10% of each kind computed in the following way: percenteage of kind 1: sum amount of kind 1 divided by sum all amount.
At the end, I would like to have a set of max 35 candidates which satisfy the constraints and optimize the sum of their scores.
Here is a the code I have come up with so far but I am struggling on the 10% constraint as it seems not to be taken into account:
rng('default');
clc;
clear;
n = 100;
maxSize = 35;
%%%TOP BASKET
nbCandidates = 100;
score = rand(100,1)/10+0.9;
quantity = rand(100,1)*100000;
type = ceil(rand(100,1)*10)
typeMask = zeros(n,10);
for i=1:10
typeMask(:,i) = type(:,1) == i;
end
fTop = -score;
intconTop = [1:1:n];
%Write the linear INEQUALITY constraints:
A = [ones(1,n);bsxfun(#times,typeMask,quantity)'/sum(type.*quantity)];
b = [maxSize;0.1*ones(10,1)];
%Write the linear EQUALITY constraints:
Aeq = [];
beq = [];
%Write the BOUND constraints:
lb = zeros(n,1);
ub = ones(n,1); % Enforces i1,i2,...in binary
x = intlinprog(fTop,intconTop,A,b,Aeq,beq,lb,ub);
I would be grateful to some advice where I m doing it wrong!

A linear program for your model might look something like this:
n is the number of candidates.
S[x] is candidate x's score.
A[i][x] is the amount of candidate x for kind i (A[i][x] can be positive or negative, like you said).
T[i] is the total amount of all candidates for kind i.
I[x] is 1 if element x is to be included, and 0 if element x is to be excluded.
The function f which you want to optimize is a function of S[x] and I[x]. You can think of S and I as n-dimensional vectors, so the function you want to optimize is their dot-product.
f() = DotProduct(I, S)
This is equivalent to the linear function I1 * S1 + I2 * S2 + ... + In * Sn.
We can formulate all of the constraints in this way to get a set of linear functions whose coeffecients are the components in an n dimensional vector that we can dot with I, the parameters to optimize.
For the constraint that we can only take 35 elements at most, let C1() be a function which computes the total number of elements.
Then the first constraint can be formalized as C1() <= 35 and C1() is a linear function which can be computed thusly:
Let j be an n dimensional vector with each component equal to 1: j = <1,1,...,1>.
C1() = DotProduct(I, j)
So C1() <= 35 is a linear inequality equivalent to:
I1 * 1 + I2 * 1 + ... + In * 1 <= 35
I1 + I2 + ... + In <= 35
We need to add a slack variable x1 here to turn this into and equivalence relation:
I1 + I2 + ... + In + x1 = 35
For the constraint that we can only take 10% of each kind, we will have a function C2[i]() for each kind i (you said there are 10 in all). C2[i]() Computes the amount of students taken for kind i given the students we have selected:
C21() <= .1 * T1
C22() <= .1 * T2
...
C210() <= .1 * T10
We compute C2[i]() like this:
Let k be an n dimensional vector equal to <A[i]1, A[i]2, ..., A[i]n>, each component is the amount of each candidate for kind i.
Then DotProduct(I, k) = I1 * A[i]1 + I2 * A[i]2 + ... + In * A[i]n, is the total amount we are taking of kind i given I, the vector which captures what elements we are including.
So C2[i]() = DotProduct(I, k)
Now that we know how to compute C2[i](), we need to add a slack variable to turn this into an equality relation:
C2[i]() + x[i + 1] = .1 * T[i]
Here x's subscript is [i + 1] because x1 is already used as a slack variable for the previous constraint.
In summary, the linear program would look like this (adding 11 slack variables x1, x2, ..., x11 for each constraint that is an inequality):
Let:
V = <I1, I2, ..., In, x1, x2, ..., x11> (variables)
|S1|
|S2|
|. |
|. |
|. |
P = |Sn| (parameters of objective function)
|0 |
|0 |
|. |
|. |
|. |
|0 |
|35 |
|.1*T1 |
C = |.1*T2 | (right-hand sides of constraining equality relations)
|... |
|.1*T10|
|1 |1 |...|1 |1|0|...|0|
|A1,1 |A1,2 |...|A1,n |0|1|...|0|
CP = |A2,1 |A2,2 |...|A2,n |0|0|...|0| (parameters of constraint functions)
|... |... |...|... |0|0|...|0|
|A10,1|A10,2|...|A10,n|0|0|...|1|
Maximize:
V x P
Subject to:
CP x Transpose(V) = C
Hopefully this is clear, sorry for terrible formatting.

I believe the MIP model can look like:
Here i are the data points and j indicates the type. For simplicity I assumed here every type has the same number of data points (i.e. Amount(i,j), Score(i,j) are matrices). It is easy to handle the more irregular case by restricting the summations.
The 10% rule is simply applied on the sum of the amounts. I hope that is the correct interpretation. Not sure if this is true if we have negative sums.

Related

Construct a Matlab array through nested loops

Suppose I have three matrices A_1, A_2, A_3 each of dimension mxn, where m and n are large. These matrices contain strictly positive numbers.
I want to construct a matrix check of dimension mx1 such that, for each i=1,...,m:
check(i)=1 if there exists j,k such that
A_1(i,1)+A_2(j,1)+A_3(k,1)<=quantile(A_1(i,2:end)+A_2(j,2:end)+A_3(k,3:end), 0.95)
In my case m is large (m=10^5) and n=500. Therefore, I would like your help to find an efficient way to do this.
Below I reproduce an example. I impose m smaller than in reality and report my incomplete and probably inefficient attempt to construct check.
clear
rng default
m=4;
n=500;
A_1=betarnd(1,2,m,n);
A_2=betarnd(1,2,m,n);
A_3=betarnd(1,2,m,n);
check=zeros(m,1);
for i=1:m
for j=1:m
for k=1:m
if A_1(i,1)+A_2(j,1)+A_3(k,1)<=quantile(A_1(i,2:end)+A_2(j,2:end)+A_3(k,2:end), 0.95)
check(i)=1;
STOP TO LOOP OVER j AND k, MOVE TO THE NEXT i (INCOMPLETE!)
else
KEEP SEARCHING FOR j,k SUCH THAT THE CONDITION IS SATISFIED (INCOMPLETE!)
end
end
end
end
Given a scalar x and a vector v the expression x <=quantile (v, .95) can be written as sum( x > v) < Q where Q = .95 * numel(v) *.
Also A_1 can be splitted before the loop to avoid extra indexing.
Moreover the most inner loop can be removed in favor of vectorization.
Af_1 = A_1(:,1);
Af_2 = A_2(:,1);
Af_3 = A_3(:,1);
As_1 = A_1(:,2:end);
As_2 = A_2(:,2:end);
As_3 = A_3(:,2:end);
Q = .95 * (n -1);
for i=1:m
for j=1:m
if any (sum (Af_1(i) + Af_2(j) + Af_3 > As_1(i,:) + As_2(j,:) + As_3, 2) < Q)
check(i) = 1;
break;
end
end
end
More optimization can be achieved by rearranging the expressions involved in the inequality and pre-computation:
lhs = A_3(:,1) - A_3(:,2:end);
lhsi = A_1(:,1) - A_1(:,2:end);
rhsj = A_2(:,2:end) - A_2(:,1);
Q = .95 * (n - 1);
for i=1:m
LHS = lhs + lhsi(i,:);
for j=1:m
if any (sum (LHS > rhsj(j,:), 2) < Q)
check(i) = 1;
break;
end
end
end
Note that because of the method that is used in the computation of quantile you may get a slightly different result.
Option 1:
Because all numbers are positive, you can do some optimizations. 95 percentile will be only higher if you add A1 to the mix - if you find the j and k of greatest 95 percentile of A2+A3 on the right side compared to the sum of the first 2 elements, you can simply take that for every i.
maxDif = -inf;
for j = 1 : m
for k = 1 : m
newDif = quantile(A_2..., 0.95) - A_2(j,1)-A_3(k,1);
maxDif = max(newDif, maxDif);
end
end
If even that is too slow, you can first get maxDifA2 and maxDifA3, then estimate that maxDif will be for those particular j and k values and calculate it.
Now, for some numbers you will get that maxDif > A_1, then the check is 1. For some numbers you will get that maxDif + quantile(A1, 0.95) < A_1, here check is 0 (if you estimated maxDif by separate calculation of A2 and A3 this isn't true!). For some (most?) you will unfortunately get values in between and this won't be helpful at all. Then what remains is option 2 (it is also more straightforward):
Option 2:
You could save some time if you can save summation A_2+A_3 on the right side, as that calculation repeats for every different i, but that requires A LOT of memory. But quantile is the more expensive operation anyway, so you aren't saving a lot of time. Something along the lines of
for j = 1 : m
for k = 1 : m
A23R(j,k,:) = A2(j,:)+A3(k,:); % Unlikely to fit in memory.
end
end
Then you can perform your loops, using A23R and avoiding to repeat that sum for every i.

Laguerre's method to obtain poly roots (Matlab)

I must write using Laguerre's method a piece of code to find the real and complex roots of poly:
P=X^5-5*X^4-6*X^3+6*X^2-3*X+1
I have little doubt. I did the algorithm in the matlab, but 3 out of 5 roots are the same and I don't think that is correct.
syms X %Declearing x as a variabl
P=X^5-5*X^4-6*X^3+6*X^2-3*X+1; %Equation we interest to solve
n=5; % The eq. order
Pd1 = diff(P,X,1); % first differitial of f
Pd2 = diff(P,X,2); %second differitial of f
err=0.00001; %Answear tollerance
N=100; %Max. # of Iterations
x(1)=1e-3; % Initial Value
for k=1:N
G=double(vpa(subs(Pd1,X,x(k))/subs(P,X,x(k))));
H=G^2 - double(subs(Pd2,X,x(k))) /subs(P,X,x(k));
D1= (G+sqrt((n-1)*(n*H-G^2)));
D2= (G-sqrt((n-1)*(n*H-G^2)));
D = max(D1,D2);
a=n/D;
x(k+1)=x(k)-a
Err(k) = abs(x(k+1)-x(k));
if Err(k) <=err
break
end
end
output (roots of polynomial):
x =
0.0010 + 0.0000i 0.1434 + 0.4661i 0.1474 + 0.4345i 0.1474 + 0.4345i 0.1474 + 0.4345i
What you actually see are all the values x(k) which arose in the loop. The last one, 0.1474 + 0.4345i is the end result of this loop - the approximation of the root which is in your given tolerance threshold. The code
syms X %Declaring x as a variable
P = X^5 - 5 * X^4 - 6 * X^3 + 6 * X^2 - 3 * X + 1; %Polynomial
n=5; %Degree of the polynomial
Pd1 = diff(P,X,1); %First derivative of P
Pd2 = diff(P,X,2); %Second derivative of P
err = 0.00001; %Answer tolerance
N = 100; %Maximal number of iterations
x(1) = 0; %Initial value
for k = 1:N
G = double(vpa(subs(Pd1,X,x(k)) / subs(P,X,x(k))));
H = G^2 - double(subs(Pd2,X,x(k))) / subs(P,X,x(k));
D1 = (G + sqrt((n-1) * (n * H-G^2)));
D2 = (G - sqrt((n-1) * (n * H-G^2)));
D = max(D1,D2);
a = n/D;
x(k+1) = x(k) - a;
Err(k) = abs(x(k+1)-x(k));
if Err(k) <=err
fprintf('Initial value %f, result %f%+fi', x(1), real(x(k)), imag(x(k)))
break
end
end
results in
Initial value -2.000000, result -1.649100+0.000000i
If you want to get other roots, you have to use other initial values. For example one can obtain
Initial value 10.000000, result 5.862900+0.000000i
Initial value -2.000000, result -1.649100+0.000000i
Initial value 3.000000, result 0.491300+0.000000i
Initial value 0.000000, result 0.147400+0.434500i
Initial value 1.000000, result 0.147400-0.434500i
These are all zeros of the polynomial.
A method for calculating the next root when you have found another one would be that you divide through the corresponding linear factor and use your loop for the resulting new polynomial. Note that this is in general not very easy to handle since rounding errors can have a big influence on the result.
Problems with the existing code
You do not implement the Laguerre method properly as a method in complex numbers. The denominator candidates D1,D2 are in general complex numbers, it is inadvisable to use the simple max which only has sensible results for real inputs. The aim is to have a=n/D be the smaller of both variants, so that one has to look for the D in [D1,D2] with the larger absolute value. If there were a conditional assignment as in C, this would look like
D = (abs(D_1)>abs(D2)) ? D1 : D2;
As that does not exist, one has to use commands with a similar result
D = D1; if (abs(D_1)<abs(D2)) D=D2; end
The resulting sequence of approximation points is
x(0) = 0.0010000
x(1) = 0.143349512707684+0.466072958423667i
x(2) = 0.164462212064089+0.461399841949893i
x(3) = 0.164466373475316+0.461405404094130i
There is a point where one can not expect the (residual) polynomial value at the root approximation to substantially decrease. The value close to zero is obtained by adding and subtracting rather large terms in the sum expression of the polynomial. The accuracy lost in these catastrophic cancellation events can not be recovered.
The threshold for polynomial values that are effectively zero can be estimated as the machine constant of the double type times the polynomial value where all coefficients and the evaluation point are replaced by their absolute values. This test serves in the code primarily to avoid divisions by zero or near-zero.
Finding all roots
One approach is to apply the method to a sufficiently large number of initial points along some circle containing all the roots, with some strict rules for early termination at too slow convergence. One would have to make the list of the roots found unique, but keep the multiplicity,...
The other standard method is to apply deflation, that is, divide out the linear factor of the root found. This works well in low degrees.
There is no need for the slower symbolic operations as there are functions that work directly on the coefficient array, such as polyval and polyder. Deflation by division with remainder can be achieved using the deconv function.
For real polynomials, we know that the complex conjugate of a root is also a root. Thus initialize the next iteration with the deflated polynomial with it.
Other points:
There is no point in the double conversions as at no point there is a conversion into the single type.
If you don't do anything with it, it makes no sense to create an array, especially not for Err.
Roots of the example
Implementing all this I get a log of
x(0) = 0.001000000000000+0.000000000000000i, |Pn(x(0))| = 0.99701
x(1) = 0.143349512707684+0.466072958423667i, |dx|= 0.48733
x(2) = 0.164462212064089+0.461399841949893i, |dx|=0.021624
x(3) = 0.164466373475316+0.461405404094130i, |dx|=6.9466e-06
root found x=0.164466373475316+0.461405404094130i with value P0(x)=-2.22045e-16+9.4369e-16i
Deflation
x(0) = 0.164466373475316-0.461405404094130i, |Pn(x(0))| = 2.1211e-15
root found x=0.164466373475316-0.461405404094130i with value P0(x)=-2.22045e-16-9.4369e-16i
Deflation
x(0) = 0.164466373475316+0.461405404094130i, |Pn(x(0))| = 4.7452
x(1) = 0.586360702193454+0.016571894375927i, |dx|= 0.61308
x(2) = 0.562204173408499+0.000003168181059i, |dx|=0.029293
x(3) = 0.562204925474889+0.000000000000000i, |dx|=3.2562e-06
root found x=0.562204925474889+0.000000000000000i with value P0(x)=2.22045e-16-1.33554e-17i
Deflation
x(0) = 0.562204925474889-0.000000000000000i, |Pn(x(0))| = 7.7204
x(1) = 3.332994579372812-0.000000000000000i, |dx|= 2.7708
root found x=3.332994579372812-0.000000000000000i with value P0(x)=6.39488e-14-3.52284e-15i
Deflation
x(0) = 3.332994579372812+0.000000000000000i, |Pn(x(0))| = 5.5571
x(1) = -2.224132251798332+0.000000000000000i, |dx|= 5.5571
root found x=-2.224132251798332+0.000000000000000i with value P0(x)=-3.33067e-14+1.6178e-15i
for the modified code
P = [1, -2, -6, 6, -3, 1];
P0 = P;
deg=length(P)-1; % The eq. degree
err=1e-05; %Answer tolerance
N=10; %Max. # of Iterations
x=1e-3; % Initial Value
for n=deg:-1:1
dP = polyder(P); % first derivative of P
d2P = polyder(dP); %second derivative of P
fprintf("x(0) = %.15f%+.15fi, |Pn(x(0))| = %8.5g\n", real(x),imag(x), abs(polyval(P,x)));
for k=1:N
Px = polyval(P,x);
dPx = polyval(dP,x);
d2Px = polyval(d2P,x);
if abs(Px) < 1e-14*polyval(abs(P),abs(x))
break % if value is zero in relative accuracy
end
G = dPx/Px;
H=G^2 - d2Px / Px;
D1= (G+sqrt((n-1)*(n*H-G^2)));
D2= (G-sqrt((n-1)*(n*H-G^2)));
D = D1;
if abs(D2)>abs(D1) D=D2; end % select the larger denominator
a=n/D;
x=x-a;
fprintf("x(%d) = %.15f%+.15fi, |dx|=%8.5g\n",k,real(x),imag(x), abs(a));
if abs(a) < err*(err+abs(x))
break
end
end
y = polyval(P0,x); % check polynomial value of the original polynomial
fprintf("root found x=%.15f%+.15fi with value P0(x)=%.6g%+.6gi\n", real(x),imag(x),real(y),imag(y));
disp("Deflation");
[ P,R ] = deconv(P,[1,-x]); % division with remainder
x = conj(x); % shortcut for conjugate pairs and clustered roots
end

vectorize lookup values in table of interval limits

Here is a question about whether we can use vectorization type of operation in matlab to avoid writing for loop.
I have a vector
Q = [0.1,0.3,0.6,1.0]
I generate a uniformly distributed random vector over [0,1)
X = [0.11,0.72,0.32,0.94]
I want to know whether each entry of X is between [0,0.1) or [0.1,0.3) or [0.3,0.6), or [0.6,1.0) and I want to return a vector which contains the index of the maximum element in Q that each entry of X is less than.
I could write a for loop
Y = zeros(length(X),1)
for i = 1:1:length(X)
Y(i) = find(X(i)<Q, 1);
end
Expected result for this example:
Y = [2,4,3,4]
But I wonder if there is a way to avoid writing for loop? (I see many very good answers to my question. Thank you so much! Now if we go one step further, what if my Q is a matrix, such that I want check whether )
Y = zeros(length(X),1)
for i = 1:1:length(X)
Y(i) = find(X(i)<Q(i), 1);
end
Use the second output of max, which acts as a sort of "vectorized find":
[~, Y] = max(bsxfun(#lt, X(:).', Q(:)), [], 1);
How this works:
For each element of X, test if it is less than each element of Q. This is done with bsxfun(#lt, X(:).', Q(:)). Note each column in the result corresponds to an element of X, and each row to an element of Q.
Then, for each element of X, get the index of the first element of Q for which that comparison is true. This is done with [~, Y] = max(..., [], 1). Note that the second output of max returns the index of the first maximizer (along the specified dimension), so in this case it gives the index of the first true in each column.
For your example values,
Q = [0.1, 0.3, 0.6, 1.0];
X = [0.11, 0.72, 0.32, 0.94];
[~, Y] = max(bsxfun(#lt, X(:).', Q(:)), [], 1);
gives
Y =
2 4 3 4
Using bsxfun will help accomplish this. You'll need to read about it. I also added a Q = 0 at the beginning to handle the small X case
X = [0.11,0.72,0.32,0.94 0.01];
Q = [0.1,0.3,0.6,1.0];
Q_extra = [0 Q];
Diff = bsxfun(#minus,X(:)',Q_extra (:)); %vectorized subtraction
logical_matrix = diff(Diff < 0); %find the transition from neg to positive
[X_categories,~] = find(logical_matrix == true); % get indices
% output is 2 4 3 4 1
EDIT: How long does each method take?
I got curious about the difference between each solution:
Test Code Below:
Q = [0,0.1,0.3,0.6,1.0];
X = rand(1,1e3);
tic
Y = zeros(length(X),1);
for i = 1:1:length(X)
Y(i) = find(X(i)<Q, 1);
end
toc
tic
result = arrayfun(#(x)find(x < Q, 1), X);
toc
tic
Q = [0 Q];
Diff = bsxfun(#minus,X(:)',Q(:)); %vectorized subtraction
logical_matrix = diff(Diff < 0); %find the transition from neg to positive
[X_categories,~] = find(logical_matrix == true); % get indices
toc
Run it for yourself, I found that when the size of X was 1e6, bsxfun was much faster, while for smaller arrays the differences were varying and negligible.
SAMPLE: when size X was 1e3
Elapsed time is 0.001582 seconds. % for loop
Elapsed time is 0.007324 seconds. % anonymous function
Elapsed time is 0.000785 seconds. % bsxfun
Octave has a function lookup to do exactly that. It takes a lookup table of sorted values and an array, and returns an array with indices for values in the lookup table.
octave> Q = [0.1 0.3 0.6 1.0];
octave> x = [0.11 0.72 0.32 0.94];
octave> lookup (Q, X)
ans =
1 3 2 3
The only issue is that your lookup table has an implicit zero which be fixed easily with:
octave> lookup ([0 Q], X) # alternatively, just add 1 at the results
ans =
2 4 3 4
You can create an anonymous function to perform the comparison, then apply it to each member of X using arrayfun:
compareFunc = #(x)find(x < Q, 1);
result = arrayfun(compareFunc, X, 'UniformOutput', 1);
The Q array will be stored in the anonymous function ( compareFunc ) when the anonymous function is created.
Or, as one line (Uniform Output is the default behavior of arrayfun):
result = arrayfun(#(x)find(x < Q, 1), X);
Octave does a neat auto-vectorization trick for you if the vectors you have are along different dimensions. If you make Q a column vector, you can do this:
X = [0.11, 0.72, 0.32, 0.94];
Q = [0.1; 0.3; 0.6; 1.0; 2.0; 3.0];
X <= Q
The result is a 6x4 matrix indicating which elements of Q each element of X is less than. I made Q a different length than X just to illustrate this:
0 0 0 0
1 0 0 0
1 0 1 0
1 1 1 1
1 1 1 1
1 1 1 1
Going back to the original example you have, you can do
length(Q) - sum(X <= Q) + 1
to get
2 4 3 4
Notice that I have semicolons instead of commas in the definition of Q. If you want to make it a column vector after defining it, do something like this instead:
length(Q) - sum(X <= Q') + 1
The reason that this works is that Octave implicitly applies bsxfun to an operation on a row and column vector. MATLAB will not do this until R2016b according to #excaza's comment, so in MATLAB you can do this:
length(Q) - sum(bsxfun(#le, X, Q)) + 1
You can play around with this example in IDEOne here.
Inspired by the solution posted by #Mad Physicist, here is my solution.
Q = [0.1,0.3,0.6,1.0]
X = [0.11,0.72,0.32,0.94]
Temp = repmat(X',1,4)<repmat(Q,4,1)
[~, ind]= max( Temp~=0, [], 2 );
The idea is that make the X and Q into the "same shape", then use element wise comparison, then we obtain a logical matrix whose row tells whether a given element in X is less than each of the element in Q, then return the first non-zero index of each row of this logical matrix. I haven't tested how fast this method is comparing to other methods

Iterating over all integer vectors summing up to a certain value in MATLAB?

I would like to find a clean way so that I can iterate over all the vectors of positive integers of length, say n (called x), such that sum(x) == 100 in MATLAB.
I know it is an exponentially complex task. If the length is sufficiently small, say 2-3 I can do it by a for loop (I know it is very inefficient) but how about longer vectors?
Thanks in advance,
Here is a quick and dirty method that uses recursion. The idea is that to generate all vectors of length k that sum to n, you first generate vectors of length k-1 that sum to n-i for each i=1..n, and then add an extra i to the end of each of these.
You could speed this up by pre-allocating x in each loop.
Note that the size of the output is (n + k - 1 choose n) rows and k columns.
function x = genperms(n, k)
if k == 1
x = n;
elseif n == 0
x = zeros(1,k);
else
x = zeros(0, k);
for i = 0:n
y = genperms(n-i,k-1);
y(:,end+1) = i;
x = [x; y];
end
end
Edit
As alluded to in the comments, this will run into memory issues for large n and k. A streaming solution is preferable, which generates the outputs one at a time. In a non-strict language like Haskell this is very simple -
genperms n k
| k == 1 = return [n]
| n == 0 = return (replicate k 0)
| otherwise = [i:y | i <- [0..n], y <- genperms (n-i) (k-1)]
viz.
>> mapM_ print $ take 10 $ genperms 100 30
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100]
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,99]
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,98]
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,97]
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,96]
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,95]
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,6,94]
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,7,93]
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,8,92]
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,91]
which runs virtually instantaneously - no memory issues to worry about.
In Python you could achieve something nearly as simple using generators and the yield keyword. In Matlab it is certainly possible, but I leave the translation up to you!
This is one possible method to generate all vectors at once (will give memory problems for moderately large n):
s = 10; %// desired sum
n = 3; %// number of digits
vectors = cell(1,n);
[vectors{:}] = ndgrid(0:s); %// I assume by "integer" you mean non-negative int
vectors = cell2mat(cellfun(#(c) reshape(c,1,[]), vectors, 'uni', 0).');
vectors = vectors(:,sum(vectors)==s); %// each column is a vector
Now you can iterate over those vectors:
for vector = vectors %// take one column at each iteration
%// do stuff with the vector
end
To avoid memory problems it is better to generate each vector as needed, instead of generating all of them initially. The following approach iterates over all possible n-vectors in one for loop (regardless of n), rejecting those vectors whose sum is not the desired value:
s = 10; %// desired sum
n = 3;; %// number of digits
for number = 0: s^n-1
vector = dec2base(number,s).'-'0'; %// column vector of n rows
if sum(vector) ~= s
continue %// reject that vector
end
%// do stuff with the vector
end

Fast way to test whether n^2 + (n+1)^2 is perfect square

I am trying to program a code to test whether n^2 + (n+1)^2 is a perfect.
As i do not have much experience in programming, I only have Matlab at my disposal.
So far this is what I have tried
function [ Liste ] = testSquare(N)
if exist('NumberTheory')
load NumberTheory.mat
else
MaxT = 0;
end
if MaxT > N
return
elseif MaxT > 0
L = 1 + MaxT;
else
L = 1;
end
n = (L:N)'; % Makes a list of numbers from L to N
m = n.^2 + (n+1).^2; % Makes a list of numbers on the form A^2+(A+1)^2
P = dec2hex(m); % Converts this list to hexadecimal
Length = length(dec2hex(P(N,:))); %F inds the maximum number of digits in the hexidecimal number
Modulo = ['0','1','4','9']'; % Only numbers ending on 0,1,4 or 9 can be perfect squares in hex
[d1,~] = ismember(P(:,Length),Modulo); % Finds all numbers that end on 0,1,4 or 9
m = m(d1); % Removes all numbers not ending on 0,1,4 or 9
n = n(d1); % -------------------||-----------------------
mm = sqrt(m); % Takes the square root of all the possible squares
A = (floor(mm + 0.5).^2 == m); % Tests wheter these are actually squares
lA = length(A(A>0)); % Finds the number of such numbers
MaxT = N;
save NumberTheory.mat MaxT;
if lA>0
m = m(A); % makes a list of all the square numbers
n = n(A); % finds the corresponding n values
mm = mm(A); % Finds the squareroot values of m
fid = fopen('Tallteori.txt','wt'); % Writes everything to a simple text.file
for ii = 1:lA
fprintf(fid,'%20d %20d %20d\t',n(ii),m(ii),mm(ii));
fprintf(fid,'\n');
end
fclose(fid);
end
end
Which will write the squares with the corresponding n values to a file. Now I saw that using hexadecimal was a fast way to find perfect squares in C+, and tried to use this in matlab. However I am a tad unsure if this is the best approach.
The code above breaks down when m > 2^52 due to the hexadecimal conversion.
Is there an alternative way/faster to write all the perfect squares on the form n^2 + (n+1)^2 to a text file from 1 to N ?
There is a much faster way that doesn't even require testing. You need a bit of elementary number theory to find that way, but here goes:
If n² + (n+1)² is a perfect square, that means there is an m such that
m² = n² + (n+1)² = 2n² + 2n + 1
<=> 2m² = 4n² + 4n + 1 + 1
<=> 2m² = (2n+1)² + 1
<=> (2n+1)² - 2m² = -1
Equations of that type are easily solved, starting from the "smallest" (positive) solution
1² - 2*1² = -1
of
x² - 2y² = -1
corresponding to the number 1 + √2, you obtain all further solutions by multiplying that with a power of the primitive solution of
a² - 2b² = 1
which is (1 + √2)² = 3 + 2*√2.
Writing that in matrix form, you obtain all solutions of x² - 2y² = -1 as
|x_k| |3 4|^k |1|
|y_k| = |2 3| * |1|
and all x_k are necessarily odd, thus can be written as 2*n + 1.
The first few solutions (x,y) are
(1,1), (7,5), (41,29), (239,169)
corresponding to (n,m)
(0,1), (3,5), (20,29), (119,169)
You can get the next (n,m) solution pair via
(n_(k+1), m_(k+1)) = (3*n_k + 2*m_k + 1, 4*n_k + 3*m_k + 2)
starting from (n_0, m_0) = (0,1).
Quick Haskell code since I don't speak MatLab:
Prelude> let next (n,m) = (3*n + 2*m + 1, 4*n + 3*m + 2) in take 20 $ iterate next (0,1)
[(0,1),(3,5),(20,29),(119,169),(696,985),(4059,5741),(23660,33461),(137903,195025)
,(803760,1136689),(4684659,6625109),(27304196,38613965),(159140519,225058681)
,(927538920,1311738121),(5406093003,7645370045),(31509019100,44560482149)
,(183648021599,259717522849),(1070379110496,1513744654945),(6238626641379,8822750406821)
,(36361380737780,51422757785981),(211929657785303,299713796309065)]
Prelude> map (\(n,m) -> (n^2 + (n+1)^2 - m^2)) it
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
Edit by EitanT:
Here's the MATLAB code to calculate the first N numbers:
res = zeros(1, N);
nm = [0, 1];
for k = 1:N
nm = nm * [3 4; 2 3] + [1, 2];
res(k) = nm(1);
end
The resulting array res should hold the values of n that satisfy the condition of the perfect square.