So I want to determine the alpha, beta and gamma values of my SVM classifier, I trained my SVM classifier and determined the alpha and support vectors. My support vectors have 3 variables per row, below are just ten rows from my support vector:
0.0904235536887480 -0.269325475875919 -0.678528701392414
-0.321039098061280 -0.507618180664821 -1.42365662798284
-0.0737761304021185 -0.269641311369441 -0.647521877863172
0.00105779420640393 -0.311226557946309 -0.667506146498475
0.0913098589312967 -0.289462325547514 -0.391261050348894
0.00622693949845773 -0.166248587146820 -0.149546793127464
-0.292302915842567 -0.564676268888150 -1.60153093563523
0.112997393643248 -0.310512134534035 -0.725281274142312
-0.135361511770186 -0.456321702624641 -1.26973221898260
-0.173160731078767 -0.434439033384469 -1.22687774941370
and similarly below are just ten rows from my alpha:
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
So what I basically doing is to determine the coefficients like so:
c = (-1)*[0.0904*x1 + (-0.2693)*x2 + (-0.6785)*x3] + (-1)*[(-0.321)*x1 + (-0.5076)*x2 + (-1.4236)*x3].....
and so on until the size of my alpha.
So I coded the following in Matlab
syms x1, syms x2, syms x3;
alpha = SVMStruct.Alpha;
svm_vec = SVMStruct.SupportVectors;
for i = 1:size(alpha,1)
c(i) = alpha(i)*(svm_vec(i,1)*x1 + svm_vec(i,2)*x2 + svm_vec(i,3)*x3);
end
sum_it = sum(c);
But this created a very strange output:
(107845064549358722206080751595348329973204613074833920445585562521882937008164658045489239834546021458299139*x1)/50534761550197893278639420198779799540396107395587434771118149413836407509953624874438129483687080755200 + (95720990302914087945142311872326568914380675701489099929103269189530321664249312169660240242394455632803627*x2)/134178504805697854567421908803656709124500009291732154392279224305703564767807900528680550698065697177600 + (90626366614084720573448362168042659133754200934323766866906741825007634289583081638045482944881264585156183*x3)/125521827076297992982426946945356276277758073208394596044390242092432367040852552107475353878835652198400
is this the expected output? Why am I getting it as fractions?
Looking by the output, did I implement my equation correctly?
Your inputs to your calculation are symbolic variables, so any calculations using those variables are going to yield an exact value that is also a symbolic variable. If you want the approximate floating point representation of the result, you'll want to cast the sum as a double
double(sum_it)
In order to do this though, you'll need to specify actual values for x1, x2, and x3
double(subs(sum(c), [x1, x2, x3], [1, 2, 3]))
That being said, I think what you actually want is to solve the system of equations. You can use the \ operator to do this
variables = svm_vec \ alpha;
Related
i have a polynomial of the form 𝑓(𝑥) = 𝑎0 + 𝑎1𝑥 + ⋯ +𝑎𝑛−1𝑥𝑛−1+𝑎𝑛𝑥𝑛.I want to take a0,a1...an from user and n = length(a0,a1...an) example:
f = 3*x^2 - 4*x + 2.Help me please,thank you.
a0=1;a1=3;a2=4;
A = [a0 a1 a2];
polyval(A,3)
ans =
22
using polyval. Note that the order of input arguments goes from highest to lowest, so in this example the polynomial is x^2+3x+4.
Since MATLAB is a numeric package, you'd generally want the numeric values of the polynomial evaluated at discrete values of x, and not a symbolic expression.
I just calculated a summation of two exponential distritbution with different lambda.
It's known that summmation of exponential distributions is Erlang(Gamma) distribution.
However, when lamdbas are different, result is a litte bit different.
Anyway look at the following equations.
Now, problem is (alpha_1 λ_2-alpha_2 λ_1).
(alpha_1 λ_2-alpha_2 λ_1) becomes 0
Thus, last two terms go to infinite....
Is that true??
I make some simple matlab code for verification.
clc;
clear;
mu=[1 2];
a1 = mu(1)/(mu(1)+mu(2));
a2 = mu(2)/(mu(1)+mu(2));
n = 10^6;
x = exprnd(mu(1), [1, n]);
y = exprnd(mu(2), [1, n]);
z = a1*x + a2*y;
figure
histfit(z, 100 ,'gamma')`
The figure is pdf of Z=alpha_1 * X + alpha_2 * Y.
This case is λ_1 = 1, λ_1=2. (The red line is gamma distribution.)
The result of matlab shows random variable Z is not infinite value.
What is the problom in my calculations??
I got the problem in my integral calculation. In the 6th row, e^-(lambda2-alpha2*lambda1/alpha1) = 1, thus, there is no term alpha1/(alpha1*lambda2-alpha2*lambda1) in the 7th row.
I am trying to implement a map / function which has the equation Bernoulli Shift Map
x_n+1 = 2* x_n mod 1
The output of this map will be a binary number which will be either 0/1.
So, I generated the first sample x_1 using rand. The following is the code. The problem is I am getting real numbers. When using a digital calculator, I can get binary, whereas when using Matlab, I am getting real numbers. Please help where I am going wrong. Thank you.
>> x = rand();
>> x
x =
0.1647
>> y = mod(2* x,1)
y =
0.3295
The dyadic transformation seems to be a transformation from [0,1) continuous to [0,1) continuous. I see nothing wrong with your test code if you are trying to implement the dyadic mapping. You should be expecting output in the [0,1)
I misunderstood your question because I focused on the assumption you had that the output should be binary [0 or 1], which is wrong.
To reproduce the output of the dyadic transformation as in the link you provided, your code works fine (for 1 value), and you can use this function to calculate N terms (assuming a starting term x0) :
function x = dyadic(x0,n)
x = zeros(n,1) ; %// preallocate output vector
x(1) = x0 ; %// assign first term
for k=2:n
x(k) = mod( 2*x(k-1) , 1) ; %// calculate all terms of the serie
end
Note that the output does not have to be binary, it has to be between 0 and 1.
In the case of integers, the result of mod(WhateverInteger,1) is always 0, but in the case of Real numbers (which is what you use here), the result of mod(AnyRealNumber,1) will be the fractional part, so a number between 0 and 1. (1 is mathematically excluded, 0 is possible by the mod(x,1) operation, but in the case of your serie it means all the successive term will be zero too).
I have a linear equation:
vt = v1*x1 + v2*x2 + v3*x3
vt, v1, v2, v3 are scalars with values between 0 and 1. What is the best way to generate one set (any set will be fine) of x1, x2 and x3 that satisfy the equation above. and also satisfy
x1>0
x2>0
x3>0
I have couple thousand sets of vt,v1,v2 and v3, therefore I need to be able to generate x1, x2 and x3 programmatically.
There are two ways you could approach this:
From the method that you have devised in your post. Randomly generate x1 and x2 and ensure that vt < v1*x1 + v2*x2, then go ahead and solve for x3.
Formulate this into linear program. A linear program is basically solving a system of equations that are subject to inequality or equality constraints. In other words:
As such, we can translate your problem to be of a linear programming problem. The "maximize" statement is what is known as the objective function - the overall goal of what you are trying to accomplish. In linear programming problems, we are trying to minimize or maximize this objective. To do this, we must satisfy the inequalities seen in the subject to condition. Usually, this program is represented in canonical form, and so the constraints on each variable should be positive.
The maximize condition can be arbitrary as you don't care about the objective. You just care about any solution. This whole paradigm can be achieved by linprog in MATLAB. What you should be careful with is how linprog is specified. In fact, the objective is minimized instead of maximized. The conditions, however, are the same with the exception of ensuring that all of the variables are positive. We will have to code that in ourselves.
In terms of the arbitrary objective, we can simply do x1 + x2 + x3. As such, c = [1 1 1]. Our equality constraint is: v1*x1 + v2*x2 + v3*x3 = vt. We also must make sure that x is positive. In order to code this in, what we can do is choose a small constant so that all values of x are greater than this value. Right now, linprog does not support strict inequalities (i.e. x > 0) and so we have to circumvent this by doing this trick. Also, to ensure that the values are positive, linprog assumes that the Ax <= b. Therefore, a common trick that is used is to negate the inequality of x >= 0, and so this is equivalent to -x <= 0. To ensure the values are non-zero, we would actually do: -x <= -eps, where eps is a small constant. However, when I was doing experiments, by doing it this way, two of the variables end up to be the same solution. As such, what I would recommend we do is to generate good solutions that are random each time, let's draw b to be from a uniform random distribution as you said. This will then give us a starting point every time we want to solve this problem.
Therefore, our inequalities are:
-x1 <= -rand1
-x2 <= -rand2
-x3 <= -rand3
rand1, rand2, rand3 are three randomly generated numbers that are between 0 and 1. In matrix form, this is:
[-1 0 0][x1] [-rand1]
[0 -1 0][x2] <= [-rand2]
[0 0 -1][x3] [-rand3]
Finally, our equality constraint from before is:
[v1 v2 v3][x1] [vt]
[x2] =
[x3]
Now, to use linprog, you would do this:
X = linprog(c, A, b, Aeq, beq);
c is a coefficient array that is defined for the objective. In this case, it would be defined as [1 1 1], A and b is the matrix and column vector defined for the inequality constraints and Aeq and beq is the matrix and column vector defined for the equality constraints. X would thus give us the solution after linprog converges (i.e. x1, x2, x3). As such, you would do this:
A = -eye(3,3);
b = -rand(3,1);
Aeq = [v1 v2 v3];
beq = vt;
c = [1 1 1];
X = linprog(c, A, b, Aeq, beq);
As an example, suppose v1 = 0.33, v2 = 0.5, v3 = 0.2, and vt = 2.5. Therefore:
rng(123); %// Set seed for reproducibility
v1 = 0.33; v2 = 0.5; v3 = 0.2;
vt = 2.5;
A = -eye(3,3);
b = -rand(3,1);
Aeq = [v1 v2 v3];
beq = vt;
c = [1 1 1];
X = linprog(c, A, b, Aeq, beq);
I get:
X =
0.6964
4.4495
0.2268
To verify that this equals vt, we would do:
s = Aeq*X
s = 2.5000
The above simply does v1*x1 + v2*x2 + v3*x3. This is computed in a dot product form to make things easy as X is a column vector and v1, v2, v3 are already set in Aeq and is a row vector.
As such, either way is good, but at least with linprog, you don't have to keep looping until you get that condition to be satisfied!
Small Caveat
One small caveat that I forgot to mention in the above approach is that you need to make sure that vt >= v1*rand1 + v2*rand2 + v3*rand3 to ensure convergence. Since you said that v1,v2,v3 are bounded between 0 and 1, the worst case is when v1,v2,v3 are all equal to 1. As such, we really need to make sure that vt > rand1 + rand2 + rand3. If this is not the case, then simply take each value of rand1, rand2, rand3, and divide by (rand1 + rand2 + rand3) / vt. As such, this will ensure that the total summation will equal vt assuming that all of the weights are 1, and this will allow the linear program to converge properly.
If you don't, then the solution will not converge due to the inequality conditions placed in for b, and you won't get the right answer. Just some food for thought! As such, do this for b before you run linprog
if sum(-b) > vt
b = b ./ (sum(-b) / vt);
end
Good luck!
According to my previous question, I want to optimize an objective function using binary integer linear programming (all of variables are binary) as follows:
Minimize f = (c1*x1) + (c2*x2) + MAX((c3*x3),(c4*x4)) + (c5*x5)
Subject to: some equality and inequality constraints
For MAX operator, I used auxiliary variable x6 and added x6>=(c3*x3) and x6>=(c4*x4) constraints to the problem so the problem turn into:
Minimize f = (c1*x1) + (c2*x2) + x6 + (c5*x5), with added constraints.
I used CPLEX API for MATLAB to optimize the objective function.
Because all of variables are binary except one ((x6) which is continuous) and coefficients have double values, the problem turn into mixed integer linear programming so I used cplexmilp function with this configuration:
Variable types: ctype='BBBBBC' ( B:Binary, C:Continuous)
Lower Bounds: lb=[0 0 0 0 0 0]
Upper Bounds: ub=[0 0 0 0 0 inf]
Function Call:
[fval] = cplexmilp(f, Aineq, bineq, Aeq, beq,[],[],[],lb,ub,ctype)
but sometimes in the results I see x3 and x4 have continuous values(between 0 and 1) and x3+x4=1.
So my questions are:
Can any one tell me what's wrong with x3 and x4?
Is there a solution for not using auxiliary variable and solving this optimization problem with cplexbilp ?
Thanks in advance
[UPDATE]:
One parts of my code had logical errors which I fixed them, now in all cases which x3 and x4 are not binary we have x3*(1-x3)< 10^(-5), x4*(1-x4)< 10^(-5) and x3+x4=1, so #David Nehme were right (according to his useful comment), but my second question still remains here!
David's solution shows you why your formulation has become linearized but non-binary. You could also try printing out the problem in LP or MPS format to see all the resulting constraints.
You asked about a formulation that continues to be purely binary. Here's one way to do that:
Transforming this to a purely binary formulation
Here's a way to keep the problem with Max() also binary. It involves additional aux. variables, but it is relatively straight-forward once you apply the standard if-then IP tricks.
First, let's list out the four possible cases in a simple table, and see what values the max() term can take. These cases are mutually exclusive.
x3 | x4 | max (c3.x4, c4.x3)
-------------------------------
0 | 0 | 0
1 | 0 | c3
0 | 1 | c4
1 | 1 | max(c3, c4) - a constant
Now, let C34 be the max(c3, c4). Note that C34 is a number, not a variable in the problem. We need this for the new Objective function.
Introducing new binary variables
For each of the four cases above, let's introduce an auxiliary BINARY variable. For clarity, call them y0, y3, y4, y34.
Only one of the cases in the table above can hold, so we add:
y0 + y3 + y4 + y34 = 1
yi are BINARY
Now, all that remains is to add linking constraints that ensure:
If x3=0 AND x4=0 then y0=1
If x3=1 AND x4=0 then y3=1
If x3=0 AND x4=1 then y4=1
If x3=1 AND x4=1 then y34=1
We can ensure that by adding a pair of linear constraints for each of the conditions above.
2 y0 <= (1- x3) + (1 -x4)
(1-x3) + (1-x4) <= y0 + 1
2 y3 <= x3 + (1-x4)
x3+(1-x4) <= y3 + 1
2 y4 <= x4 + (1-x3)
x4+(1-x3) <= y4 + 1
2 y34 <= x3 + x4
x3+x4 <= y34 + 1
The new objective function now becomes:
Minimize f = (c1*x1) + (c2*x2) + (c5*x5) + 0*Y0 + C3*Y3 + C4*Y4 + C34*Y34
Notice that we don't have the Max() term anymore in the objective function. And all x and y variables are binary. All your original constraints plus the new ones above (8+1 = 9 of them) should be included. Once you do that, you can use cplexbilp because it is a pure BILP problem.
Hope that helps.
In your case, the auxiliary variable x6 is needed because the "MAX" function is not linear (it has a discontinuous gradient where c3*x3 == c4*x4). By adding the additional variable and constraints you are creating a linear version of the problem with a solution that is equivalent to your original nonlinear problem. The trade-off is that if c3 or c4 have a value other than 0 or 1, then your problem is not a pure binary problem. That is a very good trade-off, especially if you are using a good MIP solver.