If I have a set of linear equations (random matrix generated):
2x + 4y + 6z = 4
5x + 3y + 7z = 1
9x + 7y + 3z = 6
and I want to solve for x, y and z I just do a matrix division. But if I want to set a constraint on this matrix, like x > 0 or x = 4, is there a way of doing this?
Is adding another row correct, for example:
2x + 4y + 6z = 4
5x + 3y + 7z = 1
9x + 7y + 3z = 6
1x + 0y + 0z = 1 <---
and is there a general way of applying these constraints with bigger matrices and more complex coefficients?
In MATLAB, use lsqnonneg for non-negativity constraints (on all variables.) If you have the optimization toolbox, then you would use lsqlin to solve problems with inequality constraints, or where only certain variables are bound constrained.
You could of course use a LP solver like linprog, but if you have linprog, then you also have lsqlin! I suppose you could even use the quadprog solver, but why bother? Use the right tool for the problem.
As for the idea of using an explicitly iterative solver to solve it like fmincon, yes, you could do that, but you will be left with a less exact result that takes more time to solve.
Yes, you should investigate either Lagrange multipliers or simplex method to see how it's done.
Related
I have some confusion about the terminologies and simulation of an FIR system. I shall appreciate help in rectifying my mistakes and informing what is correct.
Assuming a FIR filter with coefficient array A=[1,c2,c3,c4]. The number of elements are L so the length of the filter L but the order is L-1.
Confusion1: Is the intercept 1 considered as a coefficient? Is it always 1?
Confusion2: Is my understanding correct that for the given example the length L= 4 and order=3?
Confusion3: Mathematically, I can write it as:
where u is the input data and l starts from zero. Then to simulate the above equation I have done the following convolution. Is it correct?:
N =100; %number of data
A = [1, 0.1, -0.5, 0.62];
u = rand(1,N);
x(1) = 0.0;
x(2) = 0.0;
x(3) = 0.0;
x(4) = 0.0;
for n = 5:N
x(n) = A(1)*u(n) + A(2)*u(n-1)+ A(3)*u(n-3)+ A(4)*u(n-4);
end
Confusion1: Is the intercept 1 considered as a coefficient? Is it always 1?
Yes it is considered a coefficient, and no it isn't always 1. It is very common to include a global scaling factor in the coefficient array by multiplying all the coefficients (i.e. scaling the input or output of a filter with coefficients [1,c1,c2,c2] by K is equivalent to using a filter with coefficients [K,K*c1,K*c2,K*c3]). Also note that many FIR filter design techniques generate coefficients whose amplitude peaks near the middle of the coefficient array and taper off at the start and end.
Confusion2: Is my understanding correct that for the given example the length L= 4 and order = 3?
Yes, that is correct
Confusion3: [...] Then to simulate the above equation I have done the following convolution. Is it correct? [...]
Almost, but not quite. Here are the few things that you need to fix.
In the main for loop, applying the formula you would increment the index of A and decrement the index of u by 1 for each term, so you would actually get x(n) = A(1)*u(n) + A(2)*u(n-1)+ A(3)*u(n-2)+ A(4)*u(n-3)
You can actually start this loop at n=4
The first few outputs should still be using the formula, but dropping the terms u(n-k) for which n-k would be less than 1. So, for x(3) you'd be dropping 1 term, for x(2) you'd be dropping 2 terms and for x(1) you'd be dropping 3 terms.
The modified code would look like the following:
x(1)=A(1)*u(1);
x(2)=A(1)*u(2) + A(2)*u(1);
x(3)=A(1)*u(3) + A(2)*u(2) + A(3)*u(1);
for n = 4:N
x(n) = A(1)*u(n) + A(2)*u(n-1)+ A(3)*u(n-2)+ A(4)*u(n-3);
end
I defined an inline function f that takes as argument a (1,3) vector
a = [3;0.5;1];
b = 3 ;
f = #(x) x*a+b ;
Suppose I have a matrix X of size (N,3). If I want to apply f to each row of X, I can simply write :
f(X)
I verified that f(X) is a (N,1) vector such that f(X)(i) = f(X(i,:)).
Now, if I a add a quadratic term :
f = #(x) x*A*x' + x*a + b ;
the command f(X) raises an error :
Error using +
Matrix dimensions must agree.
Error in #(x) x*A*x' + x*a + b
I guess Matlab is considering the whole matrix X as the input to f. So it does not create a vector with each row, i, being equal to f(X(i,:)). How can I do it ?
I found out that there exist a built-in function rowfun that could help me, but it seems to be available only in versions r2016 (I have version r2015a)
That is correct, and expected.
MATLAB tries to stay close to mathematical notation, and what you are doing (X*A*X' for A 3×3 and X N×3) is valid math, but not quite what you intend to do -- you'll end up with a N×N matrix, which you cannot add to the N×1 matrix x*a.
The workaround is simple, but ugly:
f_vect = #(x) sum( (x*A).*x, 2 ) + x*a + b;
Now, unless your N is enormous, and you have to do this billions of times every minute of every day, the performance of this is more than acceptable.
Iff however this really and truly is your program's bottleneck, than I'd suggest taking a look at MMX on the File Exchange. Together with permute(), this will allow you to use those fast BLAS/MKL operations to do this calculation, speeding it up a notch.
Note that bsxfun isn't going to work here, because that does not support mtimes() (matrix multiplication).
You can also upgrade to MATLAB R2016b, which will have built-in implicit dimension expansion, presumably also for mtimes() -- but better check, not sure about that one.
Adding to the question's description :
I am doing Feature extraction from videos and i am trying to implement this one line of mathematical equation to matlab or even any algorithm .
let's say I have B and C which are 10 x 10 matrices
R is also a 10 x 10 matrix but it should satisfy two conditions
1) M == BxMxC
2) The Values in M should be within a given range [x( which is min) y(max)] given by the user
i can handle the constraints of the range but i have no idea on how to proceed with the 1st condition at all , much less converting it into matlab
I have found several solutions to this type of problem
1)Sylvester's equation which approximates AX+XB = C or AX + XB + C = 0
2)Lyapunov Matrix equation which approximates AX+XA'+Q = 0
both of these implementations of matlab i have found in this link given below
https://github.com/ajt60gaibb/freeLYAP
but i need help in the mathematics area as to how i can use those (if i can) to M = BMC
if anyone can help me get through this ... i'll be really glad
peace out
According to my previous question, I want to optimize an objective function using binary integer linear programming (all of variables are binary) as follows:
Minimize f = (c1*x1) + (c2*x2) + MAX((c3*x3),(c4*x4)) + (c5*x5)
Subject to: some equality and inequality constraints
For MAX operator, I used auxiliary variable x6 and added x6>=(c3*x3) and x6>=(c4*x4) constraints to the problem so the problem turn into:
Minimize f = (c1*x1) + (c2*x2) + x6 + (c5*x5), with added constraints.
I used CPLEX API for MATLAB to optimize the objective function.
Because all of variables are binary except one ((x6) which is continuous) and coefficients have double values, the problem turn into mixed integer linear programming so I used cplexmilp function with this configuration:
Variable types: ctype='BBBBBC' ( B:Binary, C:Continuous)
Lower Bounds: lb=[0 0 0 0 0 0]
Upper Bounds: ub=[0 0 0 0 0 inf]
Function Call:
[fval] = cplexmilp(f, Aineq, bineq, Aeq, beq,[],[],[],lb,ub,ctype)
but sometimes in the results I see x3 and x4 have continuous values(between 0 and 1) and x3+x4=1.
So my questions are:
Can any one tell me what's wrong with x3 and x4?
Is there a solution for not using auxiliary variable and solving this optimization problem with cplexbilp ?
Thanks in advance
[UPDATE]:
One parts of my code had logical errors which I fixed them, now in all cases which x3 and x4 are not binary we have x3*(1-x3)< 10^(-5), x4*(1-x4)< 10^(-5) and x3+x4=1, so #David Nehme were right (according to his useful comment), but my second question still remains here!
David's solution shows you why your formulation has become linearized but non-binary. You could also try printing out the problem in LP or MPS format to see all the resulting constraints.
You asked about a formulation that continues to be purely binary. Here's one way to do that:
Transforming this to a purely binary formulation
Here's a way to keep the problem with Max() also binary. It involves additional aux. variables, but it is relatively straight-forward once you apply the standard if-then IP tricks.
First, let's list out the four possible cases in a simple table, and see what values the max() term can take. These cases are mutually exclusive.
x3 | x4 | max (c3.x4, c4.x3)
-------------------------------
0 | 0 | 0
1 | 0 | c3
0 | 1 | c4
1 | 1 | max(c3, c4) - a constant
Now, let C34 be the max(c3, c4). Note that C34 is a number, not a variable in the problem. We need this for the new Objective function.
Introducing new binary variables
For each of the four cases above, let's introduce an auxiliary BINARY variable. For clarity, call them y0, y3, y4, y34.
Only one of the cases in the table above can hold, so we add:
y0 + y3 + y4 + y34 = 1
yi are BINARY
Now, all that remains is to add linking constraints that ensure:
If x3=0 AND x4=0 then y0=1
If x3=1 AND x4=0 then y3=1
If x3=0 AND x4=1 then y4=1
If x3=1 AND x4=1 then y34=1
We can ensure that by adding a pair of linear constraints for each of the conditions above.
2 y0 <= (1- x3) + (1 -x4)
(1-x3) + (1-x4) <= y0 + 1
2 y3 <= x3 + (1-x4)
x3+(1-x4) <= y3 + 1
2 y4 <= x4 + (1-x3)
x4+(1-x3) <= y4 + 1
2 y34 <= x3 + x4
x3+x4 <= y34 + 1
The new objective function now becomes:
Minimize f = (c1*x1) + (c2*x2) + (c5*x5) + 0*Y0 + C3*Y3 + C4*Y4 + C34*Y34
Notice that we don't have the Max() term anymore in the objective function. And all x and y variables are binary. All your original constraints plus the new ones above (8+1 = 9 of them) should be included. Once you do that, you can use cplexbilp because it is a pure BILP problem.
Hope that helps.
In your case, the auxiliary variable x6 is needed because the "MAX" function is not linear (it has a discontinuous gradient where c3*x3 == c4*x4). By adding the additional variable and constraints you are creating a linear version of the problem with a solution that is equivalent to your original nonlinear problem. The trade-off is that if c3 or c4 have a value other than 0 or 1, then your problem is not a pure binary problem. That is a very good trade-off, especially if you are using a good MIP solver.
I'm desperately trying to avoid a for loop in Matlab, but I cannot figure out how to do it. Here's the situation:
I have two m x n matrices A and B and two vectors v and w of length d. I want to outer multiply A and v so that I get an m x n x d matrix where the (i,j,k) entry is A_(i,j) * v_k, and similarly for B and w.
Afterward, I want to add the resulting m x n x d matrices, and then take the mean along the last dimension to get back an m x n matrix.
I'm pretty sure I could handle the latter part, but the first part has me completely stuck. I tried using bsxfun to no avail. Anyone know an efficient way to do this? Thanks very much!
EDIT: This revision comes after the three great answers below. gnovice has the best answer to the question I asked without a doubt. However,the question that I meant to ask involves squaring each entry before taking the mean. I forgot to mention this part originally. Given this annoyance, both of the other answers work well, but the clever trick of doing algebra before coding doesn't help this time. Thanks for the help, everyone!
EDIT:
Even though the problem in the question has been updated, an algebraic approach can still be used to simplify matters. You still don't have to bother with 3-D matrices. Your result is just going to be this:
output = mean(v.^2).*A.^2 + 2.*mean(v.*w).*A.*B + mean(w.^2).*B.^2;
If your matrices and vectors are large, this solution will give you much better performance due to the reduced amount of memory required as compared to solutions using BSXFUN or REPMAT.
Explanation:
Assuming M is the m-by-n-by-d matrix that you get as a result before taking the mean along the third dimension, this is what a span along the third dimension will contain:
M(i,j,:) = A(i,j).*v + B(i,j).*w;
In other words, the vector v scaled by A(i,j) plus the vector w scaled by B(i,j). And this is what you get when you apply an element-wise squaring:
M(i,j,:).^2 = (A(i,j).*v + B(i,j).*w).^2;
= (A(i,j).*v).^2 + ...
2.*A(i,j).*B(i,j).*v.*w + ...
(B(i,j).*w).^2;
Now, when you take the mean across the third dimension, the result for each element output(i,j) will be the following:
output(i,j) = mean(M(i,j,:).^2);
= mean((A(i,j).*v).^2 + ...
2.*A(i,j).*B(i,j).*v.*w + ...
(B(i,j).*w).^2);
= sum((A(i,j).*v).^2 + ...
2.*A(i,j).*B(i,j).*v.*w + ...
(B(i,j).*w).^2)/d;
= sum((A(i,j).*v).^2)/d + ...
sum(2.*A(i,j).*B(i,j).*v.*w)/d + ...
sum((B(i,j).*w).^2)/d;
= A(i,j).^2.*mean(v.^2) + ...
2.*A(i,j).*B(i,j).*mean(v.*w) + ...
B(i,j).^2.*mean(w.^2);
Try reshaping the vectors v and w to be 1 x 1 x d:
mean (bsxfun(#times, A, reshape(v, 1, 1, [])) ...
+ bsxfun(#times, B, reshape(w, 1, 1, [])), 3)
Here I am using [] in the argument to reshape to tell it to fill that dimension in based on the product of all the other dimensions and the total number of elements in the vector.
Use repmat to tile the matrix in the third dimension.
A =
1 2 3
4 5 6
>> repmat(A, [1 1 10])
ans(:,:,1) =
1 2 3
4 5 6
ans(:,:,2) =
1 2 3
4 5 6
etc.
You still don't have to resort to any explicit loops or indirect looping using bsxfun et al. for your updated requirements. You can achieve what you want by a simple vectorized solution as follows
output = reshape(mean((v(:)*A(:)'+w(:)*B(:)').^2),size(A));
Since OP only says that v and w are vectors of length d, the above solution should work for both row and column vectors. If they are known to be column vectors, v(:) can be replaced by v and likewise for w.
You can check if this matches Lambdageek's answer (modified to square the terms) as follows
outputLG = mean ((bsxfun(#times, A, reshape(v, 1, 1, [])) ...
+ bsxfun(#times, B, reshape(w, 1, 1, []))).^2, 3);
isequal(output,outputLG)
ans =
1