I'd like to approximate a given n x m matrix A with n >> m as a weighted sum W of some k rows B (ideally selected from A, but could also be arbitrary). The weights must sum up to 1 and need to be positive.
import numpy as np
n = 1000 # rows
m = 3 # columns
k = 2 # hidden rank
# create random matrix with rank k
A = np.random.rand(n, k).dot(np.random.rand(k, m))
# estimate hidden rank
u, s, vt = np.linalg.svd(A, full_matrices=False, compute_uv=True)
k_est = np.count_nonzero(~np.isclose(s, 0))
# truncate to k_est
B = np.diag(s[:k_est]) # vt[..., :k_est, :]
W = u[..., :k_est]
# do some magic with B and W to come up with
assert np.all(W >= 0)
assert np.all(np.isclose(W.sum(1), 1))
assert np.all(np.isclose(A, W # B))
I tried with SVD which is able to reproduce A by W # B, but the weights are negative and don't sum up to 1.
From my gut feeling it seems like I'm searching for a convex hull of A, but with only k_est points.
I have an equation for which I am trying to write a code in Matlab, but I am not sure if my code is right. The equation is as follows:
Where I think the iteration is over the superscript i.e. k, k+1 etc and the dimensions are marked by subscripts m, n, n'. The notations are not well defined in the literature so I think this is how it should be.
My code segment for this equation is as follows:
c_n = [1,2,3,4]'; % c^(0)_n (nx1) vector
K = 50;
d = [0.5,0.9]';
for k = 1:1:K
c_n = c_n.*((sum(A_mn'*d/(sum(A_mn*c_n,2)),2))./sum(A_mn',2)) ;
end
Is this code correct for the above equation?. The summations in the equation are confusing me.
If A is a matrix with m rows and n columns, then the sum is just the sum of the nth row in AT. This is the same as the corresponding sum of the nth column in A: . The matrix multiplication it represents works out nicer with the transpose because matrix multiplications are just sums of weighted rows.
Similarly, is the mth row of A, weighted element-wise by by c.
We can assume that c and d are column vectors of size n and m, respectively. (d' will be represented by just plain d in the code). In that case, most of the operations can be reduced to matrix operations:
is just the matrix product A * c, which yields a column vector of size m.
then becomes the element-wise ratio d ./ (A * c), also of size m.
The ratio is used to scale the elements of the sum of AT in the numerator, making it the matrix product A.' * (d ./ (A * c)) of size n.
Each element of that is scaled by , which can be represented by either A.' * ones(m, 1) or sum(A, 1).', so the final matrix product is just c .* (A.' * (d ./ (A * c)) ./ sum(A, 1).').
You can pre-calculate sum(A, 1).', call it e to get the following:
c = [1; 2; 3; 4];
d = [0.5; 0.9];
A = ... some 2x4 matrix;
e = sum(A, 1).';
k = 50;
for i = 1 : k
c = c .* (A.' * (d ./ (A * c)) ./ e);
end
If you want to retain the intermediate values of c for each k, you can allocate a matrix of size n, k + 1 and fill that in with each column representing a new iteration of c:
c = zeros(4, 51);
c(:, 1) = [1; 2; 3; 4];
for i = 1 : k
c(:, k + 1) = c(:, k) .* (A.' * (d ./ (A * c(:, k))) ./ e);
end
I have two matrices X and Y, both of order mxn. I want to create a new matrix Z of order mx1 such that each i th entry in this new matrix is computed by applying a function to ith and ith row of X and Y respectively. In my case m = 100000 and n = 2. I tried using a loop but it takes forever.
for i = 1:m
Z = function(X(1,:),Y(1,:), constant_parameters)
end
Is there an efficient way to vectorize it?
EDIT 1
This is the function
function [peso] = fxPesoTexturaCN(a,b, img, r, L)
ac = num2cell(a);
bc = num2cell(b);
imgint1 = img(sub2ind(size(img),ac{:}));
imgint2 = img(sub2ind(size(img),bc{:}));
peso = (sum((a - b) .^ 2) + (r/L) * (imgint2 - imgint1)) / (2*r^2);
Where img, r, L are constats. a is X(1,:) and b is Y(1,:)
And the call of this function is
peso = bsxfun(#(a,b) fxPesoTexturaCN(a,b,img,r,L), a, b);
Suppose I have a m-by-n-by-p matrix "A", each indices stores a real number, now I want to create another matrix "B" and B(i, j, k) = f(A(i, j, k), i, j, k, otherVars), is there a faster way to do it in matlab rather than looping through all the elements? (notice the function requires the index number (i, j, k))
An example is as follows(The actual function f could be more complex):
A = rand(3, 4, 5);
B = zeros(size(A));
C = 10;
for x = 1:size(A, 1)
for y = 1:size(A, 2)
for z = 1:size(A, 3)
B(x, y, z) = A(x,y,z) + x - y * z + C;
end
end
end
I've tried creating a cell "B", and
B{i, j, k} = [A(i, j, k), i, j, k];
I then applied cellfun() to do the parallel computing, but it's even slower than a for-loop over each elements in A.
In my real implementation, function f is much more complex than B = A + X - Y.*Z + C; it takes four scaler values and I don't want to modify it since it's a function written in an external package. Any suggestions?
Vectorize it by building an ndgrid of the appropriate values:
[X,Y,Z] = ndgrid(1:size(A,1), 1:size(A,2), 1:size(A,3));
B = A + X - Y.*Z + C;
Suppose we are given a training dataset {yᵢ, xᵢ}, for i = 1, ..., n, where yᵢ can either be -1 or 1 and xᵢ can be e.g. a 2D or 3D point.
In general, when the input points are linearly separable, the SVM model can be defined as follows
min 1/2*||w||²
w,b
subject to the constraints (for i = 1, ..., n)
yᵢ*(w*xᵢ - b) >= 1
This is often called the hard-margin SVM model, which is thus a constrained minimization problem, where the unknowns are w and b. We can also omit 1/2 in the function to be minimized, given it's just a constant.
Now, the documentation about Matlab's quadprog states
x = quadprog(H, f, A, b) minimizes 1/2*x'*H*x + f'*x subject to the restrictions A*x ≤ b. A is a matrix of doubles, and b is a vector of doubles.
We can implement the hard-margin SVM model using quadprog function, to get the weight vector w, as follows
H becomes an identity matrix.
f' becomes a zeros matrix.
A is the left-hand side of the constraints
b is equal to -1 because the original constraint had >= 1, it becomes <= -1 when we multiply with -1 on both sides.
Now, I am trying to implement a soft-margin SVM model. The minimization equation here is
min (1/2)*||w||² + C*(∑ ζᵢ)
w,b
subject to the constraints (for i = 1, ..., n)
yᵢ*(w*xᵢ - b) >= 1 - ζᵢ
such that ζᵢ >= 0, where ∑ is the summation symbol, ζᵢ = max(0, 1 - yᵢ*(w*xᵢ - b)) and C is a hyper-parameter.
How can this optimization problem be solved using the Matlab's quadprog function? It's not clear to me how the equation should be mapped to the parameters of the quadprog function.
The "primal" form of the soft-margin SVM model (i.e. the definition above) can be converted to a "dual" form. I did that, and I am able to get the Lagrange variable values (in the dual form). However, I would like to know if I can use quadprog to solve directly the primal form without needing to convert it to the dual form.
I don't see how it can be a problem. Let z be our vector of (2n + 1) variables:
z = (w, eps, b)
Then, H becomes diagonal matrix with first n values on the diagonal equal to 1 and the last n + 1 set to zero:
H = diag([ones(1, n), zeros(1, n + 1)])
Vector f can be expressed as:
f = [zeros(1, n), C * ones(1, n), 0]'
First set of constrains becomes:
Aineq = [A1, eye(n), zeros(n, 1)]
bineq = ones(n, 1)
where A1 is a the same matrix as in primal form.
Second set of constraints becomes lower bounds:
lb = (inf(n, 1), zeros(n, 1), inf(n, 1))
Then you can call MATLAB:
z = quadprog(H, f, Aineq, bineq, [], [], lb);
P.S. I can be mistaken in some small details, but the general idea is right.
I wanted to clarify #vharavy answer because you could get lost while trying to deduce what 'n' means in his code. Here is my version according to his answer and SVM wikipedia article. I assume we have a file named "test.dat" which holds coordinates of test points and their class membership in the last column.
Example content of "test.dat" with 3D points:
-3,-3,-2,-1
-1,3,2,1
5,4,1,1
1,1,1,1
-2,5,4,1
6,0,1,1
-5,-5,-3,-1
0,-6,1,-1
-7,-2,-2,-1
Here is the code:
data = readtable("test.dat");
tableSize = size(data);
numOfPoints = tableSize(1);
dimension = tableSize(2) - 1;
PointsCoords = data(:, 1:dimension);
PointsSide = data.(dimension+1);
C = 0.5; %can be changed
n = dimension;
m = numOfPoints; %can be also interpretet as number of constraints
%z = [w, eps, b]; number of variables in 'z' is equal to n + m + 1
H = diag([ones(1, n), zeros(1, m + 1)]);
f = [zeros(1, n), C * ones(1, m), 0];
Aineq = [-diag(PointsSide)*table2array(PointsCoords), -eye(m), PointsSide];
bineq = -ones(m, 1);
lb = [-inf(1, n), zeros(1, m), -inf];
z = quadprog(H, f, Aineq, bineq, [], [], lb);
If let z = (w; w0; eps)T be a the long vector with n+1+m elements.(m the number of points)
Then,
H= diag([ones(1,n),zeros(1,m+1)]).
f = [zeros(1; n + 1); ones(1;m)]
The inequality constraints can be specified as :
A = -diag(y)[X; ones(m; 1); zeroes(m;m)] -[zeros(m,n+1),eye(m)],
where X is the n x m input matrix in the primal form.Out of the 2 parts for A, the first part is for w0 and the second part is for eps.
b = ones(m,1)
The equality constraints :
Aeq = zeros(1,n+1 +m)
beq = 0
Bounds:
lb = [-inf*ones(n+1,1); zeros(m,1)]
ub = [inf*ones(n+1+m,1)]
Now, z=quadprog(H,f,A,b,Aeq,beq,lb,ub)
Complete code. The idea is the same as above.
n = size(X,1);
m = size(X,2);
H = diag([ones(1, m), zeros(1, n + 1)]);
f = [zeros(1,m+1) c*ones(1,n)]';
p = diag(Y) * X;
A = -[p Y eye(n)];
B = -ones(n,1);
lb = [-inf * ones(m+1,1) ;zeros(n,1)];
z = quadprog(H,f,A,B,[],[],lb);
w = z(1:m,:);
b = z(m+1:m+1,:);
eps = z(m+2:m+n+1,:);