I'm using cplex linear programming in matlab (cplexlp) to solve the problem
min f'u s.t. Au>=b, u>=lb
using
[u,minima,flag] = cplexlp(f,-A,-b,[],[],lb);
but I need a solution tolerance of below 1e-9, which is the minimal tolerance in documentation.
I figured I can just scale the problem (e.g. by 10000) and achieve an effective tolerance of 1e-13.
scale=10000;
tolerance=1e-9;
options = cplexoptimset('cplex');
options.simplex.tolerances.feasibility = tolerance;
options.simplex.tolerances.optimality = tolerance;
[u,minima,flag] = cplexlp(scale*f,-scale*A,-scale*b,[],[],scale*lb,[], [], options);
minima = minima / scale;
It does not work, and the tolerance is improved to 1e-11 but not less. The image below show in log10 scale the 'real solution' (found using a different method) and the solution returned by the algorithm for different parameters (each color is different A,b and the x axes is some parameter of the problem which control the solution). As you can see, the real solution is achieved as long as it is above 1e-11.
Any suggestion for why is it so or how to avoid this problem?
CPLEX, like most other codes, uses double precision arithmetic. Within this environment you usually can trust your results only up to the first 9 or 10, maybe 11, digits. That's also the reason why you cannot set a smaller tolerance in CPLEX.
To get more accurate results you would have to use a solver that uses rational arithmetic. QSopt_ex and SoPlex (with iterative refinement) are two possibilities. I do not know how this is possible from within Matlab.
Additional answers offered in the cplex forum, but none solved it.
Related
I would like to solve the following optimization problem with Matlab: minimize f(t) when t >= 0.
There's fseminf function but i didn't understand well how to apply it to my case. Also it seems a bit overkill to use such a powerful tool for such a seemingly easy problem. I'll be grateful for any tips on how to apply fseminf here and any suggestions how else it can be solved.
Matlab is a numerical software so one 'simple' way to solve this problem is to calculate the value of f(t) for values of t>0 and then find the minimum of that. Depending on the function the number of ts that you want to evaluate might be smaller or larger.
One possible solution could be:
t = 0:0.001:10; % create values from 0 to 10 in steps of 0.001
f = t.^3+5; % evaluate the function for each value of 't'
[minF, locT] = min(f);
minF % this is the smallest value of the function
t(locT) % the minimum value occurred at this 't'
You should define t to be in the region where you expect the minimum to, if you define it wrong this would only find the local minimum. If the spacing between separate 't's is too large the minimum might also fall between them, that is why I choose a relatively small step of '0.001'.
I'm using Pulp to solve a linear program (also getting the same result with scipy). So something wrong with my linear program formulation, or I don't know some tricky details on how simplex algorithm works.
Here is objective function for minimisation, notice that multiplier for x2 is 0, so I don't expect x1 or x2 to have any value except 0, because x3 doesn't have max constraint and -1 * x3 is able to provide more value for minimisation:
System of linear equations:
As a solution I'm getting x2 = 20 even if its multiplier in objective function is 0.
If in objective function I set -2 * x3, then it works just fine.
The solution you've posted gives the objective = -380. Check x=[20,0,0,20,0,20,20,0], the objective function is -380 as well, which implies it's optimal too, hence, you have infinitely many solutions (it's easy to show that any convex combination of this two points is optimal, see any Linear Programming book). The problem is your PuLP solver stopped when it's encountered one optimal extreme point. If you are interested in getting all the optimal extreme points I would recommend you to use Cplex (it's not free but you might be eligible for IBM academic initiatives). Also, you can set a solution method to Dual Simlex in your PuLP solver to make it go in different direction and there is a chance you will get the other extreme point.
I am trying to solve two coupled algebraic equation
f1(x,y) = 0;
f2(x,y) = 0;
typical order of magnitude of the functions f1 and f2 are 10^42 . I ran the matlab code but it said no solution found. I figured that the problem is because scales involved is very high. Rescaling the whole equation is pretty tedious. I want to stop the root finding function (fsolve) when delta(f)/f < epsilon(say 1e-6) . How can this condition implemented in matlab? Any alternative solution to the scaling problem is also welcome.
RTFM (friendly of course), https://de.mathworks.com/help/optim/ug/fsolve.html
The options that you can provide to the solver contain the parameter TolFun with default value 1e-6 that is the absolute tolerance for the function value. Apparently there is no provision for relative tolerance, so you need to compute the function value scale from the initial point or more global considerations to set TolFun = scale * epsilon.
This is my first post to stackoverflow, so if this isn't the correct area I apologize. I am working on minimizing a L1-Regularized System.
This weekend is my first dive into optimization, I have a basic linear system Y = X*B, X is an n-by-p matrix, B is a p-by-1 vector of model coefficients and Y is a n-by-1 output vector.
I am trying to find the model coefficients, I have implemented both gradient descent and coordinate descent algorithms to minimize the L1 Regularized system. To find my step size I am using the backtracking algorithm, I terminate the algorithm by looking at the norm-2 of the gradient and terminating if it is 'close enough' to zero(for now I'm using 0.001).
The function I am trying to minimize is the following (0.5)*(norm((Y - X*B),2)^2) + lambda*norm(B,1). (Note: By norm(Y,2) I mean the norm-2 value of the vector Y) My X matrix is 150-by-5 and is not sparse.
If I set the regularization parameter lambda to zero I should converge on the least squares solution, I can verify that both my algorithms do this pretty well and fairly quickly.
If I start to increase lambda my model coefficients all tend towards zero, this is what I expect, my algorithms never terminate though because the norm-2 of the gradient is always positive number. For example, a lambda of 1000 will give me coefficients in the 10^(-19) range but the norm2 of my gradient is ~1.5, this is after several thousand iterations, While my gradient values all converge to something in the 0 to 1 range, my step size becomes extremely small (10^(-37) range). If I let the algorithm run for longer the situation does not improve, it appears to have gotten stuck somehow.
Both my gradient and coordinate descent algorithms converge on the same point and give the same norm2(gradient) number for the termination condition. They also work quite well with lambda of 0. If I use a very small lambda(say 0.001) I get convergence, a lambda of 0.1 looks like it would converge if I ran it for an hour or two, a lambda any greater and the convergence rate is so small it's useless.
I had a few questions that I think might relate to the problem?
In calculating the gradient I am using a finite difference method (f(x+h) - f(x-h))/(2h)) with an h of 10^(-5). Any thoughts on this value of h?
Another thought was that at these very tiny steps it is traveling back and forth in a direction nearly orthogonal to the minimum, making the convergence rate so slow it is useless.
My last thought was that perhaps I should be using a different termination method, perhaps looking at the rate of convergence, if the convergence rate is extremely slow then terminate. Is this a common termination method?
The 1-norm isn't differentiable. This will cause fundamental problems with a lot of things, notably the termination test you chose; the gradient will change drastically around your minimum and fail to exist on a set of measure zero.
The termination test you really want will be along the lines of "there is a very short vector in the subgradient."
It is fairly easy to find the shortest vector in the subgradient of ||Ax-b||_2^2 + lambda ||x||_1. Choose, wisely, a tolerance eps and do the following steps:
Compute v = grad(||Ax-b||_2^2).
If x[i] < -eps, then subtract lambda from v[i]. If x[i] > eps, then add lambda to v[i]. If -eps <= x[i] <= eps, then add the number in [-lambda, lambda] to v[i] that minimises v[i].
You can do your termination test here, treating v as the gradient. I'd also recommend using v for the gradient when choosing where your next iterate should be.
I have several equations and each have their own individual frequencies and amplitudes. I would like to sum the equations together and adjust the individual phases, phase1,phase2, and phase3 to keep the total amplitude value of eq_total under a specific value like 0.8. I know I can normalize the signal or change the vertical offset, but for my purposes I need to have the amplitude controlled by changing/finding the values for just the phases in phase1,phase2, and phase3 that will limit the maximum amplitude when the equations are summed.
Note: I'm using constructive and destructive phase interference to adjust the maximum amplitude of the summed equations.
Example:
eq1=0.2*cos(2pi*t*3+phase1)+vertical offset1
eq2=0.7*cos(2pi*t*9+phase2)+vertical offset2
eq3=0.8*cos(2pi*t*5+phase3)+vertical offset3
eq_total=eq1+eq2+eq3
Is there a way to solve for phase1,phase2, and phase3 so that the amplitude of the summed signals in eq_total never goes over 0.8 by just adjusting/finding the values of phase1,phase2,and phase3?
Here's a picture of a geogebra applet I tested this idea with.
Here's the geogebra ggb file I used to edit/test idea with. (I used this to see if my idea would work) Java is required if you want to dynamically interact with the applet
http://dl.dropbox.com/u/6576402/questions/ggb/sin_find_phases_example.ggb
I'm using matlab/octave
Thanks
Your example
eq1=0.2*cos(2pi*t*3+phase1)+vertical offset1
eq2=0.7*cos(2pi*t*9+phase2)+vertical offset2
eq3=0.8*cos(2pi*t*5+phase3)+vertical offset3
eq_total=eq1+eq2+eq3
where the maximum amplitude should be less than 0.8, has infinitely many solutions. Unless you have some additional objective you'd like to achieve, I suggest that you modify the problem such that you find the combination of phase angles that has a maximum amplitude of exactly 0.8 (or 0.79, such that you're guaranteed to be below).
Furthermore only two out of three phase angles are independent; if you increase all by, say, pi/3, the solution still holds. Thus, you have only two unknowns in eq_total.
You can solve the nonlinear optimization problem using e.g. FMINSEARCH. You formulate the problem such that max(abs(eq_total(phase1,phase2))) should equal 0.79.
Thus:
%# define the vector t, verticalOffset here
%# objectiveFunction is (eq_total-0.79)^2, so the phase shifts 1 and 2 that
%# satisfy this (approximately) should guarantee that signal never exceeds 0.8
objectiveFunction = #(phase)(max(abs(0.2*cos(2*pi*t+phase(1))+0.7*cos(2*pi*t*9+phase(2))+0.8*cos(2*pi*t*5)+verticalOffset)) - 0.79)^2;
%# search for optimal phase shift, starting at no shift
solution = fminsearch(objectiveFunction,[0;0]);
EDIT
Unfortunately when I Try this code and plot the results the maximum amplitude is not 0.79 it's over 1. Am I doing something wrong? see code below t=linspace(0,1,8000); verticalOffset=0; objectiveFunction = #(phase)(max(abs(0.2*cos(2*pi*t+phase(1))+0.7*cos(2*pi*t*9+phase(2))+0.8*cos(2*pi*t*5)+verticalOffset)) - 0.79)^2; s1 = fminsearch(objectiveFunction,[0;0]) eqt=0.2*cos(2*pi*t+s1(1))+0.7*cos(2*pi*t*9+s1(2))+0.8*cos(2*pi*t*5)+verticalOffset; plot(eqt)
fminsearch will find a minimum of the objective function. Whether this solution satisfies all your conditions is something you have to test. In this case, the solution given by fminsearch with the starting value [0;0] gives a maximum of ~1.3, which is obviously not good enough. However, when you plot the maximum for a range of phase angles from 0 to 2pi, you'll see that `fminsearch didn't get stuck in a bad local minimum. Rather, there is no good solution at all (z-axis is the maximum).
If I understand you correctly, you are trying to find a phase to vary the amplitude of a signal. To my knowledge, this is not possible.
For a signal
s = A * cos (w*t + phi)
only A allows you to change the amplitude. With w you change the frequency of the signal and phi regulates the "horizontal shift".
Furthermore, I think you are missing a "moving variable" like the time t in the equation above.
Maybe this article clarifies things a little.
If you set all the vertical offsets to be equal to -1, then it solves your problem because each eq# will never be > 0, so the sum can never be >0.8.
I know that this isn't that helpful, but I'm hoping that this will help you understand your problem better.