Related
I am trying to define I function which I will derivate later and the function is
My code is as below:
syms i q j
syms f(i,q)
f=symsum(((1-q)^(i-j))*j*q,j,0,i)
The f I get shows me 4 terms but in fact I do not know this number. Is it correct or how do I enter this correctly?
First of all, please don't use i and j, because they correspond to the imaginary unit; that causes misunderstandings.
syms N q k
syms f(i,q)
f=symsum(((1-q)^(N-k))*k*q,k,0,N)
piecewise(q == 0, 0, q ~= 0, (q + N*q - q*(1 - q)^N + (1 - q)^N - 1)/q)
The result is a piecewise, and it says that the result is 0 if q is 0, and otherwise the formula given on the right.
You can also evaluate Sum[(1 - q)^(M - k) k q, {k, 0, M}] with WolframAlpha, where you get the same result. As you see, your formula can be written without the sum. You get the derivative with D[Sum[(1 - q)^(M - k) k q, {k, 0, M}],q] on WolframAlpha.
below is my code to perform jacobi iterations to solve Ax=b.
I try this code on the matrix A =[4 -1 1; 4 -8 1; -2 1 5] and b=[7 -21 15].
and x is a first guess 1 x 3 vector. Are not these dimensions correct? It gives me the error in the code that calculates: r = b - x*A
and M\(x*N + b)
What am I missing?!? how do I fix this? please help!
function [x, error, iter, flag] = jacobi(A, x, b, maxiter, tol)
%implement jacobi iterations
%[x, error, iter, flag] = jacobi(A, x, b, maxiter, tol)
%jacobi.m solves the linear system Ax=b using the Jacobi iteration
%
%
%INPUT A the matrix of the system Ax=b
% x the first guess vector Ax=b
% b the vector in the system
% maxiter the maximum number of iteration to perform
% tol the tolerance
%
%OUTPUT x the solution vector
% error error norm
% niter the number of iterations it took
% flag indicates whether a solution was found. 0 means there was a
% solution and 1 means there was not a solution
iter = 0;
flag = 0;
bnrm2 = norm(b);
if (bnrm2 == 0)
bnrm2 = 1.0;
end
r = b - x*A;
error = norm(r) / bnrm2;
if (error<tol)
return;
end
[m,n] = size(A);
M = diag(diag(A));
N = diag(diag(A)) - A;
for iter = 1:maxiter,
oldx = x;
x = M\(x*N + b);
error = norm(x - oldx) / norm(x);
if (error <= tol)
break;
end
end
if (error > tol)
flag = 1;
end
Since, in the code, you're solving what I'll call (not sure if it's proper since I never do it) the left-multiply problem, the operator and order of matrices are, in some sense, reversed.
If you were solving the problem A*x = b with the residual r = b - A*x (i.e., x and b are column vectors), you would perform right-vector multiplies and left-matrix divides. Therefore, the update line in the loop would be
x = M \ (N*x + b);
Conversely, if you were solving the problem x*A = b with the residual r = b - x*A (i.e., x and b are row vectors), you would perform left-vector multiplies and right-matrix divides. Therefore, the update line in the loop would be
x = (x*N + b) / M;
Note that \ resolves to the function mldivide and / resolves to mrdivide. There is no function distinction for the multiply.
It appears your current updater mixes the two, which is bad news bears for dimension matching.
I am using matlab to simulate an accumulation process with several random walks that accumulate towards threshold in parallel. To select which random walk will increase at time t, randsample is used. If the vector V represents the active random walks and vector P represents the probability with which each random walk should be selected then the call to randsample looks like this:
randsample(V, 1, true, P);
The problem is that the simulations are slow, and randsample is the bottleneck. Approximately 80% of the runtime is dedicated to resolving the randsample call.
Is there a relatively straightforward way to improve upon the efficiency of randsample? Are there other alternatives that might improve the speed?
Like I mentioned in the comments, the bottleneck is properly caused by the fact that you are sampling one value at a time, it would be faster if you vectorize the randsample call (of course I am assuming that the probabilities vector is constant).
Here is a quick benchmark:
function testRandSample()
v = 1:5;
w = rand(numel(v),1); w = w ./ sum(w);
n = 50000;
% timeit
t(1) = timeit(#() func1(v, w, n));
t(2) = timeit(#() func2(v, w, n));
t(3) = timeit(#() func3(v, w, n));
disp(t)
% check distribution of samples (should be close to w)
tabulate(func1(v, w, n))
tabulate(func2(v, w, n))
tabulate(func3(v, w, n))
disp(w*100)
end
function s = func1(v, w, n)
s = randsample(v, n, true, w);
end
function s = func2(v, w, n)
[~,idx] = histc(rand(n,1), [0;cumsum(w(:))./sum(w)]);
s = v(idx);
end
function s = func3(v, w, n)
cw = cumsum(w) / sum(w);
s = zeros(n,1);
for i=1:n
s(i) = find(rand() <= cw, 1, 'first');
end
s = v(s);
%s = v(arrayfun(#(~)find(rand() <= cw, 1, 'first'), 1:n));
end
The output (annotated):
% measured elapsed times for func1/2/3 respectively
0.0016 0.0015 0.0790
% distribution of random sample from func1
Value Count Percent
1 4939 9.88%
2 15049 30.10%
3 7450 14.90%
4 11824 23.65%
5 10738 21.48%
% distribution of random sample from func2
Value Count Percent
1 4814 9.63%
2 15263 30.53%
3 7479 14.96%
4 11743 23.49%
5 10701 21.40%
% distribution of random sample from func3
Value Count Percent
1 4985 9.97%
2 15132 30.26%
3 7275 14.55%
4 11905 23.81%
5 10703 21.41%
% true population distribution
9.7959
30.4149
14.7414
23.4949
21.5529
As you can see, randsample is pretty well optimized. The bottleneck you observed in your code is probably due lack of vectorization as I explained.
To see how slow it can get, replace func1 with a looped version sampling one value at-a-time:
function s = func1(v, w, n)
s = zeros(n,1);
for i=1:n
s(i) = randsample(v, 1, true, w);
end
end
Maybe this will be faster:
find(rand <= cumsum(P), 1) %// gives the same as randsample(V, 1, true, P)
I'm assuming P are probabilities, i.e. their sum is 1. Otherwise normalize P:
find(rand <= cumsum(P)/sum(P), 1) %// gives the same as randsample(V, 1, true, P)
If P is always the same, precompute cumsum(P)/sum(P) to save time:
cp = cumsum(P)/sum(P); %// precompute (just once)
find(rand <= cP, 1) %// gives the same as randsample(V, 1, true, P)
Consider a difference equation with its initial conditions.
5y(n) + y(n-1) - 3y(n-2) = (1/5^n) u(n), n>=0
y(n-1) = 2, y(n-2) = 0
How can I determine y(n) in Matlab?
Use an approach similar to this (using filter), but specifying initial conditions as done here (using filtic).
I'm assuming your initial conditions are: y(-1)=2, y(-2)=0.
num = 1; %// numerator of transfer function (from difference equation)
den = [5 1 -3]; %// denominator of transfer function (from difference equation)
n = 0:100; %// choose as desired
x = (1/5).^n; %// n is >= 0, so u(n) is 1
y = filter(num, den, x, filtic(num, den, [2 0], [0 0]));
%// [2 0] reflects initial conditions on y, and [0 0] those on x.
Here's a plot of the result, obtained with stem(n,y).
The second line of your code does not give initial conditions, because it refers to the index variable n. Since Matlab only allows positive integer indices, I'll assume that you mean y(1) = 0 and y(2) = 2.
You can get an iteration rule out of your first equation by simple algebra:
y(n) = ( (1/5^n) u(n) - y(n-1) + 3y(n-2) ) / 5
Code to apply this rule in Matlab:
n_max = 100;
y = nan(n_max, 1);
y(1) = 0;
y(2) = 2;
for n = 3 : n_max
y(n) = ( (1/5^n) * u(n) - y(n-1) + 3 * y(n-2) ) / 5;
end
This code assumes that the array u is already defined. n_max specifies how many elements of y to compute.
Suppose we are given a training dataset {yᵢ, xᵢ}, for i = 1, ..., n, where yᵢ can either be -1 or 1 and xᵢ can be e.g. a 2D or 3D point.
In general, when the input points are linearly separable, the SVM model can be defined as follows
min 1/2*||w||²
w,b
subject to the constraints (for i = 1, ..., n)
yᵢ*(w*xᵢ - b) >= 1
This is often called the hard-margin SVM model, which is thus a constrained minimization problem, where the unknowns are w and b. We can also omit 1/2 in the function to be minimized, given it's just a constant.
Now, the documentation about Matlab's quadprog states
x = quadprog(H, f, A, b) minimizes 1/2*x'*H*x + f'*x subject to the restrictions A*x ≤ b. A is a matrix of doubles, and b is a vector of doubles.
We can implement the hard-margin SVM model using quadprog function, to get the weight vector w, as follows
H becomes an identity matrix.
f' becomes a zeros matrix.
A is the left-hand side of the constraints
b is equal to -1 because the original constraint had >= 1, it becomes <= -1 when we multiply with -1 on both sides.
Now, I am trying to implement a soft-margin SVM model. The minimization equation here is
min (1/2)*||w||² + C*(∑ ζᵢ)
w,b
subject to the constraints (for i = 1, ..., n)
yᵢ*(w*xᵢ - b) >= 1 - ζᵢ
such that ζᵢ >= 0, where ∑ is the summation symbol, ζᵢ = max(0, 1 - yᵢ*(w*xᵢ - b)) and C is a hyper-parameter.
How can this optimization problem be solved using the Matlab's quadprog function? It's not clear to me how the equation should be mapped to the parameters of the quadprog function.
The "primal" form of the soft-margin SVM model (i.e. the definition above) can be converted to a "dual" form. I did that, and I am able to get the Lagrange variable values (in the dual form). However, I would like to know if I can use quadprog to solve directly the primal form without needing to convert it to the dual form.
I don't see how it can be a problem. Let z be our vector of (2n + 1) variables:
z = (w, eps, b)
Then, H becomes diagonal matrix with first n values on the diagonal equal to 1 and the last n + 1 set to zero:
H = diag([ones(1, n), zeros(1, n + 1)])
Vector f can be expressed as:
f = [zeros(1, n), C * ones(1, n), 0]'
First set of constrains becomes:
Aineq = [A1, eye(n), zeros(n, 1)]
bineq = ones(n, 1)
where A1 is a the same matrix as in primal form.
Second set of constraints becomes lower bounds:
lb = (inf(n, 1), zeros(n, 1), inf(n, 1))
Then you can call MATLAB:
z = quadprog(H, f, Aineq, bineq, [], [], lb);
P.S. I can be mistaken in some small details, but the general idea is right.
I wanted to clarify #vharavy answer because you could get lost while trying to deduce what 'n' means in his code. Here is my version according to his answer and SVM wikipedia article. I assume we have a file named "test.dat" which holds coordinates of test points and their class membership in the last column.
Example content of "test.dat" with 3D points:
-3,-3,-2,-1
-1,3,2,1
5,4,1,1
1,1,1,1
-2,5,4,1
6,0,1,1
-5,-5,-3,-1
0,-6,1,-1
-7,-2,-2,-1
Here is the code:
data = readtable("test.dat");
tableSize = size(data);
numOfPoints = tableSize(1);
dimension = tableSize(2) - 1;
PointsCoords = data(:, 1:dimension);
PointsSide = data.(dimension+1);
C = 0.5; %can be changed
n = dimension;
m = numOfPoints; %can be also interpretet as number of constraints
%z = [w, eps, b]; number of variables in 'z' is equal to n + m + 1
H = diag([ones(1, n), zeros(1, m + 1)]);
f = [zeros(1, n), C * ones(1, m), 0];
Aineq = [-diag(PointsSide)*table2array(PointsCoords), -eye(m), PointsSide];
bineq = -ones(m, 1);
lb = [-inf(1, n), zeros(1, m), -inf];
z = quadprog(H, f, Aineq, bineq, [], [], lb);
If let z = (w; w0; eps)T be a the long vector with n+1+m elements.(m the number of points)
Then,
H= diag([ones(1,n),zeros(1,m+1)]).
f = [zeros(1; n + 1); ones(1;m)]
The inequality constraints can be specified as :
A = -diag(y)[X; ones(m; 1); zeroes(m;m)] -[zeros(m,n+1),eye(m)],
where X is the n x m input matrix in the primal form.Out of the 2 parts for A, the first part is for w0 and the second part is for eps.
b = ones(m,1)
The equality constraints :
Aeq = zeros(1,n+1 +m)
beq = 0
Bounds:
lb = [-inf*ones(n+1,1); zeros(m,1)]
ub = [inf*ones(n+1+m,1)]
Now, z=quadprog(H,f,A,b,Aeq,beq,lb,ub)
Complete code. The idea is the same as above.
n = size(X,1);
m = size(X,2);
H = diag([ones(1, m), zeros(1, n + 1)]);
f = [zeros(1,m+1) c*ones(1,n)]';
p = diag(Y) * X;
A = -[p Y eye(n)];
B = -ones(n,1);
lb = [-inf * ones(m+1,1) ;zeros(n,1)];
z = quadprog(H,f,A,B,[],[],lb);
w = z(1:m,:);
b = z(m+1:m+1,:);
eps = z(m+2:m+n+1,:);