How explicitly does Matlab solve the rightmost equation in c1, specifically ((x-1)\y)?
I am well aware what happens when you use a matrix, e.g. A\b (where A is a matrix and b is a column vector), but what happens when you use backslash on two vectors with equal rows?
The problem:
x = (1:3)';
y = ones(3,1);
c1 = ((x-1)\y) % why does this =0.6?
You're doing [0;1;2]\[1;1;1]. Essentially x=0.6 is the least-squares solution to
[0;1;2]*x=[1;1;1]
The case you have from the documentation is the following:
If A is a rectangular m-by-n matrix with m ~= n, and B is a matrix with m rows, then A\B returns a least-squares solution to the system of equations A*x= B.
(Specifically, you have m=3, n=1).
A = (1:3).' - 1; % = [0;1;2]
B = ones(3,1); % = [1;1;1]
x = A\B; % = 0.6
Algebraically, it's easy to see this is the solution to the least-squares minimisation
% Calculate least squares
leastSquares = sum( ((A*x) - B).^2 )
= sum( ([0;1;2]*x - [1;1;1]).^2 )
= sum( [-1; x-1; 2x-1].^2 )
= 1 + (x-1)^2 + (2x-1)^2
= 1 + x^2 - 2*x + 1 + 4*x^2 - 4*x + 1
= 5*x^2 - 6*x + 3
% Minimum least squares -> derivative = 0
d(leastSquares)/dx = 10*x - 6 = 0
10*x = 6
x = 0.6
I have no doubt that MATLAB uses a more sophisticated algorithm to come to the same conclusion, but this lays out the mathematics in a fairly plain way.
You can see experimentally that there is no better solution by testing the following for various values of x... 0.6 gives the smallest error:
sum( ([0;1;2]*x - [1;1;1]).^2 )
I am trying to solve an equation for x in Matlab, but keep getting the error:
Empty sym: 0-by-1
The equation has an integral, where x is the upper bound and also part of the integrand 1. The code I use is the following:
a = 0.2; b= 10; c = -10; d = 15; mu = 3; sig = 1;
syms x t
eqn = 0 == a + b*normcdf(x,mu,sig)+c*int( normcdf(d + x - t,mu,sig)*normpdf(t,mu,sig),t,0,x);
A = vpasolve(eqn,x)
Any hints on where I am wrong?
I believe that the symbolic toolbox may not be good enough to solve that integral... Maybe some assume or some other trick can do the job, I personally could not find the way.
However, to test if this is solvable, I tried Wolfram Alpha. It gives a result, that you can use.
eq1=a + b*normcdf(x,mu,sig);
resint=c*(t^3*(d - t + x)*erfc((mu - x)/(sqrt(2)*sig)))/(4*sig*exp((-mu + x)^2/(2*sig^2))*sqrt(2*pi));
A=vpasolve(eq1+subs(resint,t,x)-subs(resint,t,0) ==0)
gives 1.285643225712432599485355373093 in my PC.
I am trying to get solution for variable x using cplex in Matlab , but I'm confused how to write the syntax since I wanna the solution x∈{a,b}.
For example, assume that I have this LP problem:
Maximize x1 + 2 x2 + 3 x3
Subject to
- x1 + x2 + x3 <= 20
x1 - 3 x2 + x3 <= 30
Bounds
0 <= x1 <= 40
0 <= x2
0 <= x3
Retrieved from: http://www-01.ibm.com/support/knowledgecenter/SSSA5P_12.6.3/ilog.odms.cplex.help/CPLEX/MATLAB/topics/example_cplexlpex.html
But, in this problem, I wanna add the constraint x∈{20,30} s.t solution values are 20 or 30. How could I write the syntax for this additional constraint? I wanna use ctype, but ctype can only be used for 'I,B,S,N,C'.
I'm not overly familiar with the capabilities cplex in Matlab, but if all else fails you can always define
x = 20 + 10*B,
where B is a binary variable. That way x can only take 2 values. This approach gets messy when you want more options though, such as
x \in {20, 23, 30, 103}.
Then you would have to define
x = 20*B_1 + 23*B_2 etc
sum_i (B_i) = 1.
It works, but your solution speed would deteriorate fast.
This question is connected to this one. Suppose again the following code:
syms x
f = 1/(x^2+4*x+9)
Now taylor allows the function f to be expanded about infinity:
ts = taylor(f,x,inf,'Order',100)
But the following code
c = coeffs(ts)
produces errors, because the series does not contain positive powers of x (it contains negative powers of x).
In such a case, what code should be used?
Since the Taylor Expansion around infinity was likely performed with the substitution y = 1/x and expanded around 0, I would explicitly make that substitution to make the power positive for use on coeffs:
syms x y
f = 1/(x^2+4x+9);
ts = taylor(f,x,inf,'Order',100);
[c,ty] = coeffs(subs(ts,x,1/y),y);
tx = subs(ty,y,1/x);
The output from taylor is not a multivariate polynomial, so coeffs won't work in this case. One thing you can try is using collect (you may get the same or similar result from using simplify):
syms x
f = 1/(x^2 + 4*x + 9);
ts = series(f,x,Inf,'Order',5) % 4-th order Puiseux series of f about 0
c = collect(ts)
which returns
ts =
1/x^2 - 4/x^3 + 7/x^4 + 8/x^5 - 95/x^6
c =
(x^4 - 4*x^3 + 7*x^2 + 8*x - 95)/x^6
Then you can use numden to extract the numerator and denominator from either c or ts:
[n,d] = numden(ts)
which returns the following polynomials:
n =
x^4 - 4*x^3 + 7*x^2 + 8*x - 95
d =
x^6
coeffs can then be used on the numerator. You may find other functions listed here helpful as well.
Suppose we are given a training dataset {yᵢ, xᵢ}, for i = 1, ..., n, where yᵢ can either be -1 or 1 and xᵢ can be e.g. a 2D or 3D point.
In general, when the input points are linearly separable, the SVM model can be defined as follows
min 1/2*||w||²
w,b
subject to the constraints (for i = 1, ..., n)
yᵢ*(w*xᵢ - b) >= 1
This is often called the hard-margin SVM model, which is thus a constrained minimization problem, where the unknowns are w and b. We can also omit 1/2 in the function to be minimized, given it's just a constant.
Now, the documentation about Matlab's quadprog states
x = quadprog(H, f, A, b) minimizes 1/2*x'*H*x + f'*x subject to the restrictions A*x ≤ b. A is a matrix of doubles, and b is a vector of doubles.
We can implement the hard-margin SVM model using quadprog function, to get the weight vector w, as follows
H becomes an identity matrix.
f' becomes a zeros matrix.
A is the left-hand side of the constraints
b is equal to -1 because the original constraint had >= 1, it becomes <= -1 when we multiply with -1 on both sides.
Now, I am trying to implement a soft-margin SVM model. The minimization equation here is
min (1/2)*||w||² + C*(∑ ζᵢ)
w,b
subject to the constraints (for i = 1, ..., n)
yᵢ*(w*xᵢ - b) >= 1 - ζᵢ
such that ζᵢ >= 0, where ∑ is the summation symbol, ζᵢ = max(0, 1 - yᵢ*(w*xᵢ - b)) and C is a hyper-parameter.
How can this optimization problem be solved using the Matlab's quadprog function? It's not clear to me how the equation should be mapped to the parameters of the quadprog function.
The "primal" form of the soft-margin SVM model (i.e. the definition above) can be converted to a "dual" form. I did that, and I am able to get the Lagrange variable values (in the dual form). However, I would like to know if I can use quadprog to solve directly the primal form without needing to convert it to the dual form.
I don't see how it can be a problem. Let z be our vector of (2n + 1) variables:
z = (w, eps, b)
Then, H becomes diagonal matrix with first n values on the diagonal equal to 1 and the last n + 1 set to zero:
H = diag([ones(1, n), zeros(1, n + 1)])
Vector f can be expressed as:
f = [zeros(1, n), C * ones(1, n), 0]'
First set of constrains becomes:
Aineq = [A1, eye(n), zeros(n, 1)]
bineq = ones(n, 1)
where A1 is a the same matrix as in primal form.
Second set of constraints becomes lower bounds:
lb = (inf(n, 1), zeros(n, 1), inf(n, 1))
Then you can call MATLAB:
z = quadprog(H, f, Aineq, bineq, [], [], lb);
P.S. I can be mistaken in some small details, but the general idea is right.
I wanted to clarify #vharavy answer because you could get lost while trying to deduce what 'n' means in his code. Here is my version according to his answer and SVM wikipedia article. I assume we have a file named "test.dat" which holds coordinates of test points and their class membership in the last column.
Example content of "test.dat" with 3D points:
-3,-3,-2,-1
-1,3,2,1
5,4,1,1
1,1,1,1
-2,5,4,1
6,0,1,1
-5,-5,-3,-1
0,-6,1,-1
-7,-2,-2,-1
Here is the code:
data = readtable("test.dat");
tableSize = size(data);
numOfPoints = tableSize(1);
dimension = tableSize(2) - 1;
PointsCoords = data(:, 1:dimension);
PointsSide = data.(dimension+1);
C = 0.5; %can be changed
n = dimension;
m = numOfPoints; %can be also interpretet as number of constraints
%z = [w, eps, b]; number of variables in 'z' is equal to n + m + 1
H = diag([ones(1, n), zeros(1, m + 1)]);
f = [zeros(1, n), C * ones(1, m), 0];
Aineq = [-diag(PointsSide)*table2array(PointsCoords), -eye(m), PointsSide];
bineq = -ones(m, 1);
lb = [-inf(1, n), zeros(1, m), -inf];
z = quadprog(H, f, Aineq, bineq, [], [], lb);
If let z = (w; w0; eps)T be a the long vector with n+1+m elements.(m the number of points)
Then,
H= diag([ones(1,n),zeros(1,m+1)]).
f = [zeros(1; n + 1); ones(1;m)]
The inequality constraints can be specified as :
A = -diag(y)[X; ones(m; 1); zeroes(m;m)] -[zeros(m,n+1),eye(m)],
where X is the n x m input matrix in the primal form.Out of the 2 parts for A, the first part is for w0 and the second part is for eps.
b = ones(m,1)
The equality constraints :
Aeq = zeros(1,n+1 +m)
beq = 0
Bounds:
lb = [-inf*ones(n+1,1); zeros(m,1)]
ub = [inf*ones(n+1+m,1)]
Now, z=quadprog(H,f,A,b,Aeq,beq,lb,ub)
Complete code. The idea is the same as above.
n = size(X,1);
m = size(X,2);
H = diag([ones(1, m), zeros(1, n + 1)]);
f = [zeros(1,m+1) c*ones(1,n)]';
p = diag(Y) * X;
A = -[p Y eye(n)];
B = -ones(n,1);
lb = [-inf * ones(m+1,1) ;zeros(n,1)];
z = quadprog(H,f,A,B,[],[],lb);
w = z(1:m,:);
b = z(m+1:m+1,:);
eps = z(m+2:m+n+1,:);