I have an equation for which I am trying to write a code in Matlab, but I am not sure if my code is right. The equation is as follows:
Where I think the iteration is over the superscript i.e. k, k+1 etc and the dimensions are marked by subscripts m, n, n'. The notations are not well defined in the literature so I think this is how it should be.
My code segment for this equation is as follows:
c_n = [1,2,3,4]'; % c^(0)_n (nx1) vector
K = 50;
d = [0.5,0.9]';
for k = 1:1:K
c_n = c_n.*((sum(A_mn'*d/(sum(A_mn*c_n,2)),2))./sum(A_mn',2)) ;
end
Is this code correct for the above equation?. The summations in the equation are confusing me.
If A is a matrix with m rows and n columns, then the sum is just the sum of the nth row in AT. This is the same as the corresponding sum of the nth column in A: . The matrix multiplication it represents works out nicer with the transpose because matrix multiplications are just sums of weighted rows.
Similarly, is the mth row of A, weighted element-wise by by c.
We can assume that c and d are column vectors of size n and m, respectively. (d' will be represented by just plain d in the code). In that case, most of the operations can be reduced to matrix operations:
is just the matrix product A * c, which yields a column vector of size m.
then becomes the element-wise ratio d ./ (A * c), also of size m.
The ratio is used to scale the elements of the sum of AT in the numerator, making it the matrix product A.' * (d ./ (A * c)) of size n.
Each element of that is scaled by , which can be represented by either A.' * ones(m, 1) or sum(A, 1).', so the final matrix product is just c .* (A.' * (d ./ (A * c)) ./ sum(A, 1).').
You can pre-calculate sum(A, 1).', call it e to get the following:
c = [1; 2; 3; 4];
d = [0.5; 0.9];
A = ... some 2x4 matrix;
e = sum(A, 1).';
k = 50;
for i = 1 : k
c = c .* (A.' * (d ./ (A * c)) ./ e);
end
If you want to retain the intermediate values of c for each k, you can allocate a matrix of size n, k + 1 and fill that in with each column representing a new iteration of c:
c = zeros(4, 51);
c(:, 1) = [1; 2; 3; 4];
for i = 1 : k
c(:, k + 1) = c(:, k) .* (A.' * (d ./ (A * c(:, k))) ./ e);
end
Related
Suppose there are three n * n matrices X, Y, S. How to fast compute the the following scalars b
for i = 1:n
b = b + sum(sum((X(i,:)' * Y(i,:) - S).^2));
end
The computation cost is O(n^3). There exists a fast way to compute the outer product of two matrices. Specifically, the matrix C
for i = 1:n
C = C + X(i,:)' * Y(i,:);
end
can be calculated without for loop C = A.'*B which is only O(n^2). Is there exists a faster way to compute b?
You can use:
X2 = X.^2;
Y2 = Y.^2;
S2 = S.^2;
b = sum(sum(X2.' * Y2 - 2 * (X.' * Y ) .* S + n * S2));
Given your example
b=0;
for i = 1:n
b = b + sum(sum((X(i,:).' * Y(i,:) - S).^2));
end
We can first bring the summation out of the loop:
b=0;
for i = 1:n
b = b + (X(i,:).' * Y(i,:) - S).^2;
end
b=sum(b(:))
Knowing that we can write (a - b)^2 as a^2 - 2*a*b + b^2
b=0;
for i = 1:n
b = b + (X(i,:).' * Y(i,:)).^2 - 2.* (X(i,:).' * Y(i,:)) .*S + S.^2;
end
b=sum(b(:))
And we know that (a * b) ^ 2 is the same as a^2 * b^2:
X2 = X.^2;
Y2 = Y.^2;
S2 = S.^2;
b=0;
for i = 1:n
b = b + (X2(i,:).' * Y2(i,:)) - 2.* (X(i,:).' * Y(i,:)) .*S + S2;
end
b=sum(b(:))
Now we can compute each term separately:
b = sum(sum(X2.' * Y2 - 2 * (X.' * Y ) .* S + n * S2));
Here is the result of a test in Octave that compares my method and two other methods provided by #AndrasDeak and the original loop based solution for inputs of size 500*500:
===rahnema1 (B)===
Elapsed time is 0.0984299 seconds.
===Andras Deak (B2)===
Elapsed time is 7.86407 seconds.
===Andras Deak (B3)===
Elapsed time is 2.99158 seconds.
===Loop solution===
Elapsed time is 2.20357 seconds
n=500;
X= rand(n);
Y= rand(n);
S= rand(n);
disp('===rahnema1 (B)===')
tic
X2 = X.^2;
Y2 = Y.^2;
S2 = S.^2;
b=sum(sum(X2.' * Y2 - 2 * (X.' * Y ) .* S + n * S2));
toc
disp('===Andras Deak (B2)===')
tic
b2 = sum(reshape((permute(reshape(X, [n, 1, n]).*Y, [3,2,1]) - S).^2, 1, []));
toc
disp('===Andras Deak (B3)===')
tic
b3 = sum(reshape((reshape(X, [n, 1, n]).*Y - reshape(S.', [1, n, n])).^2, 1, []));
toc
tic
b=0;
for i = 1:n
b = b + sum(sum((X(i,:)' * Y(i,:) - S).^2));
end
toc
You probably can't spare time complexity, but you can make use of vectorization to get rid of the loop and make use of low-level code and caching as much as possible. Whether it's actually faster depends on your dimensions, so you need to do some timing tests to see if this is worth it:
% dummy data
n = 3;
X = rand(n);
Y = rand(n);
S = rand(n);
% vectorize
b2 = sum(reshape((permute(reshape(X, [n, 1, n]).*Y, [3,2,1]) - S).^2, 1, []));
% check
b - b2 % close to machine epsilon i.e. zero
What happens is that we insert a new singleton dimension in one of the arrays, ending up with an array of size [n, 1, n] against one with [n, n], the latter being implicitly the same as [n, n, 1]. The overlapping first index corresponds to the i in your loop, the remaining two indices correspond to the matrix indices of the dyadic product you have for each i. Then we permute the indices in order to put the "i" index last, so that we can again broadcast the result with S of (implicit) size [n, n, 1]. What we then have is a matrix of size [n, n, n] where the first two indices are matrix indices in your original and the last one corresponds to i. We then just have to take the square and sum each term (instead of summing twice I reshaped the array to a row and summed once).
A slight variation of the above transposes S instead of the 3d array which might be faster (again, you should time it):
b3 = sum(reshape((reshape(X, [n, 1, n]).*Y - reshape(S.', [1, n, n])).^2, 1, []));
In terms of performance, reshape is free (it only reinterprets data, it doesn't copy) but permute/transpose will often lead to a perforance hit when data gets copied.
I have two matrices X and Y, both of order mxn. I want to create a new matrix Z of order mx1 such that each i th entry in this new matrix is computed by applying a function to ith and ith row of X and Y respectively. In my case m = 100000 and n = 2. I tried using a loop but it takes forever.
for i = 1:m
Z = function(X(1,:),Y(1,:), constant_parameters)
end
Is there an efficient way to vectorize it?
EDIT 1
This is the function
function [peso] = fxPesoTexturaCN(a,b, img, r, L)
ac = num2cell(a);
bc = num2cell(b);
imgint1 = img(sub2ind(size(img),ac{:}));
imgint2 = img(sub2ind(size(img),bc{:}));
peso = (sum((a - b) .^ 2) + (r/L) * (imgint2 - imgint1)) / (2*r^2);
Where img, r, L are constats. a is X(1,:) and b is Y(1,:)
And the call of this function is
peso = bsxfun(#(a,b) fxPesoTexturaCN(a,b,img,r,L), a, b);
Suppose I have a m-by-n-by-p matrix "A", each indices stores a real number, now I want to create another matrix "B" and B(i, j, k) = f(A(i, j, k), i, j, k, otherVars), is there a faster way to do it in matlab rather than looping through all the elements? (notice the function requires the index number (i, j, k))
An example is as follows(The actual function f could be more complex):
A = rand(3, 4, 5);
B = zeros(size(A));
C = 10;
for x = 1:size(A, 1)
for y = 1:size(A, 2)
for z = 1:size(A, 3)
B(x, y, z) = A(x,y,z) + x - y * z + C;
end
end
end
I've tried creating a cell "B", and
B{i, j, k} = [A(i, j, k), i, j, k];
I then applied cellfun() to do the parallel computing, but it's even slower than a for-loop over each elements in A.
In my real implementation, function f is much more complex than B = A + X - Y.*Z + C; it takes four scaler values and I don't want to modify it since it's a function written in an external package. Any suggestions?
Vectorize it by building an ndgrid of the appropriate values:
[X,Y,Z] = ndgrid(1:size(A,1), 1:size(A,2), 1:size(A,3));
B = A + X - Y.*Z + C;
The formula I have to translate to Octave/Matlab goes something like this:
\sum (v_i - m) (v_i - m)^T
I have a matrix, and I need to take each row, subtract m from it and then multiply it with its own transpose. I wrote the inner part as a function:
function w = str(v, m)
y = v - m
w = y * transpose(y)
end
My matrix is like this
xx = [1 2 3 4 5; 1 2 3 4 5; 1 2 3 4 5]
Now I have no idea how to apply this function to each row in a matrix and then sum them up to a new matrix. Maybe someone can help me here.
EDIT: The result is not the dot product. I'm looking for v * v^T, which has a matrix as result!
Probably you need this
X = bsxfun( #minus, A, m );
Y = X'* X;
Suppose the matrix is A, then the solution is
total = sum(sum((A-m).*(A-m),2));
A.*A is an element wise multiplication, hence sum(A.*A,2) returns a column vector, with each element being the self dot product of each row in A.
If m is a vector then, it is slightly more complicated.
[p,~]=size(A);
total = sum(sum((A-repmat(m,p,1)).*(A-repmat(m,p,1)),2));
Cheers.
In the end, I wrote this:
function w = str(v, m)
y = v - m;
w = y' * y;
end
y = zeros(5,5);
for i=1:12
y = y + str(A(i,:), m);
end
Surely not the most elegant way to do this, but it seems to work.
You can subtract the mean using bsxfun
>> v_m = bsxfun( #minus, v, m );
For the sum of outer product of all vectors you can use bsxfun again
>> op = bsxfun( #times, permute( v, [3 1 2]), permute( v, [1 3 2] ) );
>> op = sum( op, 3 );
There are two ways to solve this issue:
Assume A is your matrix:
sum(drag(A' * A))
will do the job. However, it is slightly more efficient with the following:
sum((A .* A)(:))
Suppose we are given a training dataset {yᵢ, xᵢ}, for i = 1, ..., n, where yᵢ can either be -1 or 1 and xᵢ can be e.g. a 2D or 3D point.
In general, when the input points are linearly separable, the SVM model can be defined as follows
min 1/2*||w||²
w,b
subject to the constraints (for i = 1, ..., n)
yᵢ*(w*xᵢ - b) >= 1
This is often called the hard-margin SVM model, which is thus a constrained minimization problem, where the unknowns are w and b. We can also omit 1/2 in the function to be minimized, given it's just a constant.
Now, the documentation about Matlab's quadprog states
x = quadprog(H, f, A, b) minimizes 1/2*x'*H*x + f'*x subject to the restrictions A*x ≤ b. A is a matrix of doubles, and b is a vector of doubles.
We can implement the hard-margin SVM model using quadprog function, to get the weight vector w, as follows
H becomes an identity matrix.
f' becomes a zeros matrix.
A is the left-hand side of the constraints
b is equal to -1 because the original constraint had >= 1, it becomes <= -1 when we multiply with -1 on both sides.
Now, I am trying to implement a soft-margin SVM model. The minimization equation here is
min (1/2)*||w||² + C*(∑ ζᵢ)
w,b
subject to the constraints (for i = 1, ..., n)
yᵢ*(w*xᵢ - b) >= 1 - ζᵢ
such that ζᵢ >= 0, where ∑ is the summation symbol, ζᵢ = max(0, 1 - yᵢ*(w*xᵢ - b)) and C is a hyper-parameter.
How can this optimization problem be solved using the Matlab's quadprog function? It's not clear to me how the equation should be mapped to the parameters of the quadprog function.
The "primal" form of the soft-margin SVM model (i.e. the definition above) can be converted to a "dual" form. I did that, and I am able to get the Lagrange variable values (in the dual form). However, I would like to know if I can use quadprog to solve directly the primal form without needing to convert it to the dual form.
I don't see how it can be a problem. Let z be our vector of (2n + 1) variables:
z = (w, eps, b)
Then, H becomes diagonal matrix with first n values on the diagonal equal to 1 and the last n + 1 set to zero:
H = diag([ones(1, n), zeros(1, n + 1)])
Vector f can be expressed as:
f = [zeros(1, n), C * ones(1, n), 0]'
First set of constrains becomes:
Aineq = [A1, eye(n), zeros(n, 1)]
bineq = ones(n, 1)
where A1 is a the same matrix as in primal form.
Second set of constraints becomes lower bounds:
lb = (inf(n, 1), zeros(n, 1), inf(n, 1))
Then you can call MATLAB:
z = quadprog(H, f, Aineq, bineq, [], [], lb);
P.S. I can be mistaken in some small details, but the general idea is right.
I wanted to clarify #vharavy answer because you could get lost while trying to deduce what 'n' means in his code. Here is my version according to his answer and SVM wikipedia article. I assume we have a file named "test.dat" which holds coordinates of test points and their class membership in the last column.
Example content of "test.dat" with 3D points:
-3,-3,-2,-1
-1,3,2,1
5,4,1,1
1,1,1,1
-2,5,4,1
6,0,1,1
-5,-5,-3,-1
0,-6,1,-1
-7,-2,-2,-1
Here is the code:
data = readtable("test.dat");
tableSize = size(data);
numOfPoints = tableSize(1);
dimension = tableSize(2) - 1;
PointsCoords = data(:, 1:dimension);
PointsSide = data.(dimension+1);
C = 0.5; %can be changed
n = dimension;
m = numOfPoints; %can be also interpretet as number of constraints
%z = [w, eps, b]; number of variables in 'z' is equal to n + m + 1
H = diag([ones(1, n), zeros(1, m + 1)]);
f = [zeros(1, n), C * ones(1, m), 0];
Aineq = [-diag(PointsSide)*table2array(PointsCoords), -eye(m), PointsSide];
bineq = -ones(m, 1);
lb = [-inf(1, n), zeros(1, m), -inf];
z = quadprog(H, f, Aineq, bineq, [], [], lb);
If let z = (w; w0; eps)T be a the long vector with n+1+m elements.(m the number of points)
Then,
H= diag([ones(1,n),zeros(1,m+1)]).
f = [zeros(1; n + 1); ones(1;m)]
The inequality constraints can be specified as :
A = -diag(y)[X; ones(m; 1); zeroes(m;m)] -[zeros(m,n+1),eye(m)],
where X is the n x m input matrix in the primal form.Out of the 2 parts for A, the first part is for w0 and the second part is for eps.
b = ones(m,1)
The equality constraints :
Aeq = zeros(1,n+1 +m)
beq = 0
Bounds:
lb = [-inf*ones(n+1,1); zeros(m,1)]
ub = [inf*ones(n+1+m,1)]
Now, z=quadprog(H,f,A,b,Aeq,beq,lb,ub)
Complete code. The idea is the same as above.
n = size(X,1);
m = size(X,2);
H = diag([ones(1, m), zeros(1, n + 1)]);
f = [zeros(1,m+1) c*ones(1,n)]';
p = diag(Y) * X;
A = -[p Y eye(n)];
B = -ones(n,1);
lb = [-inf * ones(m+1,1) ;zeros(n,1)];
z = quadprog(H,f,A,B,[],[],lb);
w = z(1:m,:);
b = z(m+1:m+1,:);
eps = z(m+2:m+n+1,:);