I have a problem that I want to solve with a nonlinear optimization tool of Matlab. So far I only found methods like "fmincon". But for this, you have to have an explicit objective function.
Is there a tool, for that it's not necessary to have an explicit function?
My problem looks like this:
There is the parameter t that should be minimized.
Q, A, A*: 3x3 matrix,
t : 1x1 matrix
A = A* + Q*t
And there is a "sigma" which is calculated with A.
Sigma has to fulfill boundary conditions, e.g. sigma < 100
If someone could give me the right keywords that would be great. Thank you.
Related
I want to minimize a function like below:
Here, n can be 5,10,50 etc. I want to use Matlab and want to use Gradient Descent and Quasi-Newton Method with BFGS update to solve this problem along with backtracking line search. I am a novice in Matlab. Can anyone help, please? I can find a solution for a similar problem in that link: https://www.mathworks.com/help/optim/ug/unconstrained-nonlinear-optimization-algorithms.html .
But, I really don't know how to create a vector-valued function in Matlab (in my case input x can be an n-dimensional vector).
You will have to make quite a leap to get where you want to be -- may I suggest to go through some basic tutorial first in order to digest basic MATLAB syntax and concepts? Another useful read is the very basic example to unconstrained optimization in the documentation. However, the answer to your question touches only basic syntax, so we can go through it quickly nevertheless.
The absolute minimum to invoke the unconstraint nonlinear optimization algorithms of the Optimization Toolbox is the formulation of an objective function. That function is supposed to return the function value f of your function at any given point x, and in your case it reads
function f = objfun(x)
f = sum(100 * (x(2:end) - x(1:end-1).^2).^2 + (1 - x(1:end-1)).^2);
end
Notice that
we select the indiviual components of the x vector by matrix indexing, and that
the .^ notation effects that the operand is to be squared elementwise.
For simplicity, save this function to a file objfun.m in your current working directory, so that you have it available from the command window.
Now all you have to do is to call the appropriate optimization algorithm, say, the quasi Newton method, from the command window:
n = 10; % Use n variables
options = optimoptions(#fminunc,'Algorithm','quasi-newton'); % Use QM method
x0 = rand(n,1); % Random starting guess
[x,fval,exitflag] = fminunc(#objfun, x0, options); % Solve!
fprintf('Final objval=%.2e, exitflag=%d\n', fval, exitflag);
On my machine I see that the algorithm converges:
Local minimum found.
Optimization completed because the size of the gradient is less than
the default value of the optimality tolerance.
Final objval=5.57e-11, exitflag=1
I have a linear program of the form min(f*x) s.t. A1*x < d1; A2*x < d2. The form with one constraint is implemented in Matlab in command linprog. What command can I use to solve linear program with two constrraints?
I could of course create a block diagonal matrix, and double the size of the variable x, but if there is more efficient way I would like to use it, because the size of the matrix is quite large.
Possibly I don't understand the question right but can't you combine the matrixes A1 und A2 by A = [A1; A2]?
You maybe interested in Dantzig-Wolfe Decomposition algorithm for solving linear programming. It takes advantage of this block diagonal structure. However, I don't think there is an out-of-the box implementation of it in commercial softwares.
I do just about everything in Matlab but I have yet to work out a good way to replicate Mathematica's FindInstance function in Matlab. As an example, with Mathematica, I can enter:
FindInstance[x + y == 1 && x > 0 && y > 0, {x, y}]
And it will give me:
{{x -> 1/2, y -> 1/2}}
When no solution exists, it will give me an empty Out. I use this often in my work to check whether or not a solution to a system of inequalities exists -- I don't really care about a particular solution.
It seems like there should be a way to replicate this in Matlab with Solve. There are sections in the help file on solving a set of inequalities for a parametrized solution with conditions. There's another section on spitting out just one solution using PrincipalValue, but this seems to just select from a finite solution set, rather than coming up with one that meets the parameters.
Can anybody come up with a way to replicate the FindInstance functionality in Matlab?
Building on what jlandercy said, you can certainly use MATLAB's linprog function, which is MATLAB's linear programming solver. A linear program in the MATLAB universe can be formulated like so:
You seek to find a solution x in R^n which minimizes the objective function f^{T}*x subject to a set of inequality constraints, equality constraints, and each component in x is bounded between a lower and upper bound. Because you want to find the minimum possible value that satisfies the above constraint given, what you're really after is:
Because MATLAB only supports inequalities of less than, you'll need to take the negative of the first two constraints. In addition, MATLAB doesn't support strict inequalities, and so what you'll have to do is enforce a constraint so that you are checking to see if each variable is lesser than a small number, perhaps something like setting a threshold epsilon to 1e-4. Therefore, with the above, your formulation is now:
Note that we don't have any upper or lower bounds as those conditions are already satisfied in the equality and inequality constraints. All you have to do now is plug this problem into linprog. linprog accepts syntax in the following way:
x = linprog(f,A,b,Aeq,beq);
f is a vector of coefficients that work with the objective function, A is a matrix of coefficients that work with the inequality, b is a vector of coefficients that are for the right-hand side of each inequality constraint, and Aeq,beq, are the same as the inequality but for the equality constraints. x would be the solution to the linear programming problem formulated. If we reformulate your problem into matrix form for the above, we now get:
With respect to the linear programming formulation, we can now see what each variable in the MATLAB universe needs to be. Therefore, in MATLAB syntax, each variable becomes:
f = [1; 1];
A = [-1 0; 0 -1];
b = [1e-4; 1e-4];
Aeq = [1 1];
beq = 1;
As such:
x = linprog(f, A, b, Aeq, beq);
We get:
Optimization terminated.
x =
0.5000
0.5000
If linear programming is not what you're looking for, consider looking at MATLAB's MuPAD interface: http://www.mathworks.com/help/symbolic/mupad_ug/solve-algebraic-equations-and-inequalities.html - This more or less mimics what you see in Mathematica if you're more comfortable with that.
Good luck!
Matlab is not a symbolic solver as Mathematica is, so you will not get exact solutions but numeric approximations. Anyway if you are about to solve linear programming (simplex) such as in your example, you should use linprog function.
I would like to transform the matrix product AX-XB into vector form.
That is Cx where x=vec(X)
Yet I found the last term (XB) is very difficult to vectorize, it would be very sparsy.
Any effective way to do this?
Please see this link for the transformation to vector form
If you don't need C explicitly - like for iterative solvers - you can define an abstract linear operator that returns the vectorized product C*x. Not sure, if there is such a particular function in Matlab as SciPy's LinearOperator, but an anonymous function should do as well:
C_x = #(X) vec(A*X-X*B);
where vec 'vectorises' the matrix, e.g. via X(:) as #Jonas has pointed out.
EDIT: A closed form was suggested by #Eitan T below!!
See Matlab Help for how to use anonymous functions and function handles.
The formula for the explicit C is given here.
I'd like to find the principal components of a data matrix X in Matlab by solving the optimization problem min||X-XBB'||, where the norm is the Frobenius norm, and B is an orthonormal matrix. I'm wondering if anyone could tell me how to do that. Ideally, I'd like to be able to do this using the optimization toolbox. I know how to find the principal components using other methods. My goal is to understand how to set up and solve an optimization problem which has a matrix as the answer. I'd very much appreciate any suggestions or comments.
Thanks!
MJ
The thing about Optimization is that there are different methods to solve a problem, some of which can require extensive computation.
Your solution, given the constraints for B, is to use fmincon. Start by creating a file for the non-linear constraints:
function [c,ceq] = nonLinCon(x)
c = 0;
ceq = norm((x'*x - eye (size(x))),'fro'); %this checks to see if B is orthonormal.
then call the routine:
B = fmincon(#(B) norm(X - X*B*B','fro'),B0,[],[],[],[],[],[],#nonLinCon)
with B0 being a good guess on what the answer will be.
Also, you need to understand that this algorithms tries to find a local minimum, which may not be the solution you ultimately want. For instance:
X = randn(1,2)
fmincon(#(B) norm(X - X*B*B','fro'),rand(2),[],[],[],[],[],[],#nonLinCon)
ans =
0.4904 0.8719
0.8708 -0.4909
fmincon(#(B) norm(X - X*B*B','fro'),rand(2),[],[],[],[],[],[],#nonLinCon)
ans =
0.9864 -0.1646
0.1646 0.9864
So be careful, when using these methods, and try to select a good starting point
The Statistics toolbox has a built-in function 'princomp' that does PCA. If you want to learn (in general, without the optimization toolbox) how to create your own code to do PCA, this site is a good resource.
Since you've specifically mentioned wanting to use the Optimization Toolbox and to set this up as an optimization problem, there is a very well-trusted 3rd-party package known as CVX from Stanford University that can solve the optimization problem you are referring to at this site.
Do you have the optimization toolbox? The documentation is really good, just try one of their examples: http://www.mathworks.com/help/toolbox/optim/ug/brg0p3g-1.html.
But in general the optimization function look like this:
[OptimizedMatrix, OptimizedObjectiveFunction] = optimize( (#MatrixToOptimize) MyObjectiveFunction(MatrixToOptimize), InitialConditionsMatrix, ...optional constraints and options... );
You must create MyObjectiveFunction() yourself, it must take the Matrix you want to optimize as an input and output a scalar value indicating the cost of the current input Matrix. Most of the optimizers will try to minimise this cost. Note that the cost must be a scalar.
fmincon() is a good place to start, once you are used to the toolbox you and if you can you should choose a more specific optimization algorithm for your problem.
To optimize a matrix rather than a vector, reshape the matrix to a vector, pass this vector to your objective function, and then reshape it back to the matrix within your objective function.
For example say you are trying to optimize the 3 x 3 matrix M. You have defined objective function MyObjectiveFunction(InputVector). Pass M as a vector:
MyObjectiveFunction(M(:));
And within the MyObjectiveFunction you must reshape M (if necessary) to be a matrix again:
function cost = MyObjectiveFunction(InputVector)
InputMatrix = reshape(InputVector, [3 3]);
%Code that performs matrix operations on InputMatrix to produce a scalar cost
cost = %some scalar value
end