Using svmtrain and svmmodel, we're supposed to plot a hyperplane to separate two collections of data. Through the sample, I have the data:
c = [1 1; 2 1.5; 2 1;3 1.5];
N = 10; X = []; sigma = 0.2;
for i = 1:4
X = [X; sigma*randn(N,2) + repmat(c(i,:),N,1)];
end
Y = [ones(1,2*N) -ones(1,2*N)];
plot(X(1:end/2,1),X(1:end/2,2),'+')
hold all
plot(X(end/2+1:end,1),X(end/2+1:end,2),'o')
hold on
So, from my understanding, I have to use the primal vector to obtain the hyperplane as: y = w'x + b, which is the following:
model = svmtrain(Y',X,'-s 0 -t 0 -c 1')
w = model.SVs' * model.sv_coef;
b = -model.rho;
if model.Label(1) == -1
w = -w;
b = -b;
end
However, when I add
plot(w+b)
I get:
Which is not the desired hyperplane. I've played with the values as much as I can and looked up different samples of this kind, but I can't figure out how I'm supposed to use the primal and bias to produce the proper hyperplane.
Related
I am trying to test streamline with a very simple 3D vector field. I fill in a mesh using 3 for loops (not the best, but this is reminescent of a different expression for "v" which I couldn't easily vectorise). Then I define the vector field v as r. Simple radial field. My full code is below. quiver3 is fine, but unfortunately streamline gives me the following error:
Error using griddedInterpolant
Interpolation requires at least two
sample points in each dimension.
N = 5;
L = 2;
dl = 2*L/N;
for i = 1:N+1
for j = 1:N+1
for k = 1:N+1
x = -L + (i-1)*dl;
y = -L + (j-1)*dl;
z = -L + (k-1)*dl;
X(i,j,k) = x;
Y(i,j,k) = y;
Z(i,j,k) = z;
r = [x,y,z];
v = r-r0;
Vx(i,j,k) = v(1);
Vy(i,j,k) = v(2);
Vz(i,j,k) = v(3);
end
end
end
quiver3(X,Y,Z,Vx,Vy,Vz);
[sx,sy,sz] = meshgrid(0:0.1:1,0:1:5,0:1:5);
streamline(X,Y,Z,Vx,Vy,Vz,sx,sy,sz);
hold on;
streamslice(X,Y,Z,Vx,Vy,Vz,[],[],5);
pbaspect([1,1,1])
It returns back to gridded X, Y variables, if you use transposed version of X and Y, you will not get this interpolation error in the streamline function. To transpose a N-D array in MATLAB, use permute function, like:
X = permute(X, [2,1,3]); % to rearrange X and Y dimension
or just define correct form of X and Y at the first place [in the loop]
I have a set of N points in k dimensions as a matrix of size N X k.
How can I find the best fitting line through these points? The line will be a plane (hyerpplane) in k dimensions. It will have k coefficients and one bias term.
Existing functions like fit seem to be usable only for points in 2 or 3 dimension.
You can fit a hyperplane (or any lower dimensional affine space) to a set of D dimensional data using Principal Component Analysis. Here's an example of fitting a plane to a set of 3D data. This is explained in more detail in the MATLAB documentation but I tried to construct the simplest example I could.
% generate some random correlated data
D = 3;
mu = zeros(1,D);
sqrt_sig = randn(D);
sigma = sqrt_sig'*sqrt_sig;
% generate 50 points in a D x 50 matrix
X = mvnrnd(mu, sigma, 50)';
% perform PCA
coeff = pca(X');
% The last principal component is normal to the best fit plane and plane goes through mean of X
a = coeff(:,D);
b = -mean(X,2)'*a;
% plane defined by a'*x + b = 0
dist = abs(a'*X+b) / norm(a);
mse = mean(dist.^2)
Edit: Added example plot of results for D = 3. I take advantage of the orthogonality of the other principal components here. Ignore the code if you want it's just to demonstrate that the plane does in fact fit the data pretty well.
% plot in 3D
X0 = bsxfun(#minus,X,mean(X,2));
b1 = coeff(:,1); b2 = coeff(:,2);
y1 = b1'*X0; y2 = b2'*X0;
y1_min = min(y1); y1_max = max(y1);
y1_span = y1_max - y1_min;
y2_min = min(y2); y2_max = max(y2);
y2_span = y2_max - y2_min;
pad = 0.2;
y1_min = y1_min - pad*y1_span;
y1_max = y1_max + pad*y1_span;
y2_min = y2_min - pad*y2_span;
y2_max = y2_max + pad*y2_span;
[y1_m,y2_m] = meshgrid(linspace(y1_min,y1_max,5), linspace(y2_min,y2_max,5));
grid = bsxfun(#plus, bsxfun(#times,y1_m(:)',b1) + bsxfun(#times,y2_m(:)',b2), mean(X,2));
x = reshape(grid(1,:),size(y1_m));
y = reshape(grid(2,:),size(y1_m));
z = reshape(grid(3,:),size(y1_m));
figure(1); clf(1);
surf(x,y,z,'FaceColor','black','FaceAlpha',0.3,'EdgeAlpha',0.6);
hold on;
plot3(X(1,:),X(2,:),X(3,:),' .');
axis equal;
axis vis3d;
Edit2: When I say "principal component" I'm being a bit sloppy (or just plain wrong) with the wording. I'm actually referring to the orthogonal basis vectors that the principal components are expressed in.
Here's a simpler solution, that just uses MATLAB's \ operator. We start with defining a plane in k dimensions:
% 0 = a + x(1) * b(1) + x(2) * b(2) + ... + x(k) * 1
k = 8;
a = randn(1);
b = randn(k-1,1);
(note that we assume b(k)=1, you can always multiply the plane parameters by any value without changing the plane).
Next we generate N random points within this plane:
N = 1000;
x = rand(N,k-1);
x(:,k) = -(a + x * b);
...sorry, it's not the best way to generate random points on the plane, but it's good enough for the demonstration here. Add noise to the points:
x = x + 0.05*randn(size(x));
To find the parameters of the plane, we solve the system of equations
% a + x(1:k-1) * b == -x(k)
in the least-squares sense. a and b are the unknowns there. We can rewrite the left-hand side as [1,x(1:k-1)] * [a;b]. If we have a matrix equation M*p=v we can solve for p by writing p=M\v:
p = [ones(N,1),x(:,1:k-1)]\(-x(:,k));
disp(['ground truth: [a,b,1] = ',mat2str([a,b',1],3)]);
disp(['estimated : [a,b,1] = ',mat2str([p',1],3)]);
This gives as output:
ground truth: [a,b,1] = [-1.35 -1.44 -1.48 1.17 0.226 -0.214 0.234 -1.59 1]
estimated : [a,b,1] = [-1.41 -1.38 -1.43 1.14 0.219 -0.195 0.221 -1.54 1]
The less noise or the more points in the dataset, the smaller the error will be of course!
Could you please help me with the following question:
I want to solve a second order equation with two unknowns and use the results to plot an ellipse.
Here is my function:
fun = #(x) [x(1) x(2)]*V*[x(1) x(2)]'-c
V is 2x2 symmetric matrix, c is a positive constant and there are two unknowns, x1 and x2.
If I solve the equation using fsolve, I notice that the solution is very sensitive to the initial values
fsolve(fun, [1 1])
Is it possible to get the solution to this equation without providing an exact starting value, but rather a range? For example, I would like to see the possible combinations for x1, x2 \in (-4,4)
Using ezplot I obtain the desired graphical output, but not the solution of the equation.
fh= #(x1,x2) [x1 x2]*V*[x1 x2]'-c;
ezplot(fh)
axis equal
Is there a way to have both?
Thanks a lot!
you can take the XData and YData from ezplot:
c = rand;
V = rand(2);
V = V + V';
fh= #(x1,x2) [x1 x2]*V*[x1 x2]'-c;
h = ezplot(fh,[-4,4,-4,4]); % plot in range
axis equal
fun = #(x) [x(1) x(2)]*V*[x(1) x(2)]'-c;
X = fsolve(fun, [1 1]); % specific solution
hold on;
plot(x(1),x(2),'or');
% possible solutions in range
x1 = h.XData;
x2 = h.YData;
or you can use vector input to fsolve:
c = rand;
V = rand(2);
V = V + V';
x1 = linspace(-4,4,100)';
fun2 = #(x2) sum(([x1 x2]*V).*[x1 x2],2)-c;
x2 = fsolve(fun2, ones(size(x1)));
% remove invalid values
tol = 1e-2;
x2(abs(fun2(x2)) > tol) = nan;
plot(x1,x2,'.b')
However, the easiest and most straight forward approach is to rearrange the ellipse matrix form in a quadratic equation form:
k = rand;
V = rand(2);
V = V + V';
a = V(1,1);
b = V(1,2);
c = V(2,2);
% rearange terms in the form of quadratic equation:
% a*x1^2 + (2*b*x2)*x1 + (c*x2^2) = k;
% a*x1^2 + (2*b*x2)*x1 + (c*x2^2 - k) = 0;
x2 = linspace(-4,4,1000);
A = a;
B = (2*b*x2);
C = (c*x2.^2 - k);
% solve regular quadratic equation
dicriminant = B.^2 - 4*A.*C;
x1_1 = (-B - sqrt(dicriminant))./(2*A);
x1_2 = (-B + sqrt(dicriminant))./(2*A);
x1_1(dicriminant < 0) = nan;
x1_2(dicriminant < 0) = nan;
% plot
plot(x1_1,x2,'.b')
hold on
plot(x1_2,x2,'.g')
hold off
plotted two different dataset
I have 2 different dataset that have different mu and sigma, and X vector such as [1.8; 1.8]. And also I know probability of each classes that P(ω1)= P(ω2) = 1/2
I want to ploting linear decision boundary between these two dataset but I don't have any idea have to do. My code is down below, here
X = [1.8; 1.8];
u1 = [1;1]; u2 = [3;3];
s1 = [1 0;0 1]; s2 = [1 0;0 1];
Pr1 = 1/2;
Pr2 = 1/2;
r = mvnrnd(u1,s1,500);
plot(r(:,1), r(:,2), '+r');
hold on
r = mvnrnd(u2,s2,500);
plot(r(:,1), r(:,2), '+b');
hold on
grid on
W1 = (u1')/(s1(1,1))^2;
W10 = (u1'*u1)/(-2*s1(1,1)) + log(Pr1);
g1 = W1'.*X + W10;
W2 = (u2')/(s2(1,1))^2;
W20 = (u2'*u2)/(-2*s2(1,1)) + log(Pr2);
g2 = W2'.*X + W20;
Is there someone who can give any idea to me please?
The trick is to calculate two points of the decision boundary you want to plot.
W1_W2 = W2 - W1; % vector from W1 to W2
W1_W2_average = (W2 + W1)/2; % point in the middle between W1 and W2
W1_W2_orthogonal = [-W1_W2(2) W1_W2(1)]; % vector orthogonal to W1_W2
points = [W1_W2_average - 2*W1_W2_orthogonal; W1_W2_average + 2*W1_W2_orthogonal]; % Two points on the line you want to plot
plot(points(:, 1), points(:, 2)); %plot the line
Note that I am not very familiar with classification problems. I might have forgotten some terms in the calculation of the decision boundary.
I solved decision problem, in detailed
First I defined parametric function with parameters x1 and x2 such as
g = #(x1,x2)
then in order to plotting decision boundary realized that g1-g2 = 0 equation such as
e = #(x1,x2) (W1*[x1;x2] + w10 - W2*[x1;x2] - w20)
ezplot(g, [-xlim xlim -ylim ylim])
and done
I have two vectors:
x = [0; 1; 2]
y = [2.0000; 0; -14.7781]
If I will plot x and y I will see three points on the xy-plane. But I want to connect those three points and get them as a continuous function:
y = f(x),
y(0) = 2;
y(1) = 0;
y(2) = -14.7781;
y(0.5) = value between 2 and 0.
For example y can be treated as a ZOH (zero order held) continuous signal.
I saw that MATLAB has a function called d2c, which converts a model from discrete to continuous time. But have no idea how to link it with the vector I have already. How to do
this with MATLAB?
OK, your latest edit improves the situation a lot.
However, you still do not demarcate the problem sufficiently.
ZOH would be as simple as
>> x = [0; 1; 2];
>> y = [2.0000; 0; -14.7781];
>> f = #(new_x) y(find(x <= new_x, 1, 'last'));
>> f(0.5)
ans =
2
However, this is not what I think you mean, as the y(0.5) = value between 2 and 0 part of your question indicates.
Perhaps you want a linearly interpolated value:
>> f = #(new_x) interp1(x,y, new_x);
>> f(0.5)
ans =
1
Or a cubic splines interpolation:
>> f = #(new_x) interp1(x,y, new_x, 'spline');
>> f(0.5)
ans =
2.5973
What I'm asking is: what model best describes your signal when the sample time would decrease to infinitesmal values?
An nth degree polynomial can have at most n-1 turning points. Thus, we can do a polynomial regression:
% Input data
yy = [2.0000; 0; -14.7781];
% Parameters
n = length(yy)-1;
x = (0:1:n).';
% Regression
p = polyfit(x,yy,n);
% Plot
f = polyval(p,x);
figure
plot(x,yy,'o',x,f,'-')