I have an array A (I have written so as to make it similar to the matrix that I am using) :
%%%%%%%%%%%%% This is Matrix %%%%%%%%%%%%%%%%%%%%
a = 3; b = 240; c = 10; d = 30; e = 1;
mtx1 = a.*rand(30,1) + a;
mtx2 = round((b-c).*rand(30,1));
mtx3 = round((d-e).*rand(30,1));
mtx4 = -9999.*ones(30,1);
A = [mtx1 mtx2 mtx3 mtx4];
for i = 10:12
for ii = 17 :19
A(i,:)= -9999;
A(ii,:)= 999;
end
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
I would calculate some statistical values, excluding from the calculation the values **-9999 and 999.
the statistical values must be calculated with respect to each column.
the columns represent respectively: the wind speed, direction, and
other parameters
I wrote a code but it is not correct
[nr,ncc]=size(A);
for i=1:ncc
B = A(:,i); %// Temp Vector
Oup=1; Odw=1; %// for Vector Control
while Oup>0 %// || Odw>0 % Oup>0 OR Odw>0 , Oup>0 && (AND) Odw>0
B=sort(B,'descend');
U = find(B<999 & B>-9999); % find for each column of the temp
%vector
Oup = length(U); % Calculates the length
B(U)=[]; % Delete values -9999 and 9999
end
% calculates parameters with the vector temp
count(i)=length(B);
med(i)=mean(B);
devst(i)=std(B);
mediana(i)=median(B);
vari(i)=var(B);
kurt(i)=kurtosis(B);
Asimm(i)=skewness(B);
Interv(i)=range(B);
Mass(i)=max(B);
Mini(i)=min(B);
if length(B)<nr
B(length(B)+1:nr)=nan;
end
C(:,i)=B(:); %//reconstruction of the original matrix
end
would you have any suggestions?
If your data set is in A, and you want to operate on it with a function f, just use logical indexing, i.e.:
f(A( ~(A==999 & A==-9999) )) =...
Alternatively, use find and linear indexing:
ind = find( ~(A==999 & A==-9999) );
f(A(ind)) = ....
Related
Is there a function in MATLAB that generates the following matrix for a given scalar r:
1 r r^2 r^3 ... r^n
0 1 r r^2 ... r^(n-1)
0 0 1 r ... r^(n-2)
...
0 0 0 0 ... 1
where each row behaves somewhat like a power analog of the CUMSUM function?
You can compute each term directly using implicit expansion and element-wise power, and then apply triu:
n = 5; % size
r = 2; % base
result = triu(r.^max((1:n)-(1:n).',0));
Or, maybe a little faster because it doesn't compute unwanted powers:
n = 5; % size
r = 2; % base
t = (1:n)-(1:n).';
u = find(t>=0);
t = t(u);
result = zeros(n);
result(u) = r.^t;
Using cumprod and triu:
% parameters
n = 5;
r = 2;
% Create a square matrix filled with 1:
A = ones(n);
% Assign the upper triangular part shifted by one with r
A(triu(A,1)==1)=r;
% cumprod along the second dimension and get only the upper triangular part
A = triu(cumprod(A,2))
Well, cumsum accumulates the sum of a vector but you are asking for a specially design matrix, so the comparison is a bit problematic....
Anyway, it might be that there is a function for this if this is a common special case triangular matrix (my mathematical knowledge is limited here, sorry), but we can also build it quite easily (and efficiently=) ):
N = 10;
r = 2;
% allocate arry
ary = ones(1,N);
% initialize array
ary(2) = r;
for i = 3:N
ary(i) = ary(i-1)*r;
end
% build matrix i.e. copy the array
M = eye(N);
for i = 1:N
M(i,i:end) = ary(1:end-i+1);
end
This assumes that you want to have a matrix of size NxN and r is the value that you want calculate the power of.
FIX: a previous version stated in line 13 M(i,i:end) = ary(i:end);, but the assignment needs to start always at the first position of the ary
I have the following data:
N = 10^3;
x = randn(N,1);
y = randn(N,1);
z = randn(N,1);
f = x.^2+y.^2+z.^2;
Now I want to split this continuous 3D space into nB bins.
nB = 20;
[~,~,x_bins] = histcounts(x,nB);
[~,~,y_bins] = histcounts(y,nB);
[~,~,z_bins] = histcounts(z,nB);
And put in each cube average f or nan if no observations happen in the cube:
F = nan(50,50,50);
for iX = 1:20
for iY = 1:20
for iZ = 1:20
idx = (x_bins==iX)&(y_bins==iY)&(z_bins==iZ);
F(iX,iY,iZ) = mean(f(idx));
end
end
end
isosurface(F,0.5)
This code does what I want. My problem is the speed. This code is extremely slow when N > 10^5 and nB = 100.
How can I speed up this code?
I also tried the accumarray() function:
subs=([x_bins,y_bins,z_bins]);
F2 = accumarray(subs,f,[],#mean);
all(F(:) == F2(:)) % false
However, this code produces a different result.
The problem with the code in the OP is that it tests all elements of the data for each element in the output array. The output array has nB^3 elements, the data has N elements, so the algorithm is O(N*nB^3). Instead, one can loop over the N elements of the input, and set the corresponding element in the output array, which is an operation O(N) (2nd code block below).
The accumarray solution in the OP needs to use the fillvals parameter, set it to NaN (3rd code block below).
To compare the results, one needs to explicitly test that both arrays have NaN in the same locations, and have equal non-NaN values elsewhere:
all( ( isnan(F(:)) & isnan(F2(:)) ) | ( F(:) == F2(:) ) )
% \-------same NaN values------/ \--same values--/
Here is code. All three versions produce identical results. Timings in Octave 4.4.1 (no JIT), in MATLAB the loop code should be faster. (Using input data from OP, with N=10^3 and nB=20).
%% OP's code, O(N*nB^3)
tic
F = nan(nB,nB,nB);
for iX = 1:nB
for iY = 1:nB
for iZ = 1:nB
idx = (x_bins==iX)&(y_bins==iY)&(z_bins==iZ);
F(iX,iY,iZ) = mean(f(idx));
end
end
end
toc
% Elapsed time is 1.61736 seconds.
%% Looping over input, O(N)
tic
s = zeros(nB,nB,nB);
c = zeros(nB,nB,nB);
ind = sub2ind([nB,nB,nB],x_bins,y_bins,z_bins);
for ii=1:N
s(ind(ii)) = s(ind(ii)) + f(ii);
c(ind(ii)) = c(ind(ii)) + 1;
end
F2 = s ./ c;
toc
% Elapsed time is 0.0606539 seconds.
%% Other alternative, using accumarray
tic
ind = sub2ind([nB,nB,nB],x_bins,y_bins,z_bins);
F3 = accumarray(ind,f,[nB,nB,nB],#mean,NaN);
toc
% Elapsed time is 0.14113 seconds.
To compute the mean of every bins along a dimension of a nd array in matlab, for example, average every 10 elements along dim 4 of a 4d array
x = reshape(1:30*30*20*300,30,30,20,300);
n = 10;
m = size(x,4)/10;
y = nan(30,30,20,m);
for ii = 1 : m
y(:,:,:,ii) = mean(x(:,:,:,(1:n)+(ii-1)*n),4);
end
It looks a bit silly. I think there must be better ways to average the bins?
Besides, is it possible to make the script applicable to general cases, namely, arbitray ndims of array and along an arbitray dim to average?
For the second part of your question you can use this:
x = reshape(1:30*30*20*300,30,30,20,300);
dim = 4;
n = 10;
m = size(x,dim)/10;
y = nan(30,30,20,m);
idx1 = repmat({':'},1,ndims(x));
idx2 = repmat({':'},1,ndims(x));
for ii = 1 : m
idx1{dim} = ii;
idx2{dim} = (1:n)+(ii-1)*n;
y(idx1{:}) = mean(x(idx2{:}),dim);
end
For the first part of the question here is an alternative using cumsum and diff, but it may not be better then the loop solution:
function y = slicedmean(x,slice_size,dim)
s = cumsum(x,dim);
idx1 = repmat({':'},1,ndims(x));
idx2 = repmat({':'},1,ndims(x));
idx1{dim} = slice_size;
idx2{dim} = slice_size:slice_size:size(x,dim);
y = cat(dim,s(idx1{:}),diff(s(idx2{:}),[],dim))/slice_size;
end
Here is a generic solution, using the accumarray function. I haven't tested how fast it is. There might be some room for improvement though.
Basically, accumarray groups the value in x following a matrix of customized index for your question
x = reshape(1:30*30*20*300,30,30,20,300);
s = size(x);
% parameters for averaging
dimAv = 4;
n = 10;
% get linear index
ix = (1:numel(x))';
% transform them to a matrix of index per dimension
% this is a customized version of ind2sub
pcum = [1 cumprod(s(1:end-1))];
sub = zeros(numel(ix),numel(s));
for i = numel(s):-1:1,
ixtmp = rem(ix-1, pcum(i)) + 1;
sub(:,i) = (ix - ixtmp)/pcum(i) + 1;
ix = ixtmp;
end
% correct index for the given dimension
sub(:,dimAv) = floor((sub(:,dimAv)-1)/n)+1;
% run the accumarray to compute the average
sout = s;
sout(dimAv) = ceil(sout(dimAv)/n);
y = accumarray(sub,x(:), sout, #mean);
If you need a faster and memory efficient operation, you'll have to write your own mex function. It shouldn't be so difficult, I think !
I'm using data set with 200 data points that is used to draw B-Spline curve and I want to extract the 100 original control points from this curve to use it in one algorithm to solve one problem. The result of control points it's too small compared with the value of the data points of B-Spline curve so I don't know if I make something wrong in the following code or not I need help to know that because I must used these control points to complete my study in one algorithm
link of set of data points:
https://drive.google.com/open?id=0B_2BUqaJptbqUkRWLWdmbmpQakk
Code :
% read data set
dataset = importdata("path of data set here");
x = dataset(:,1);
y = dataset(:,2);
for i=1:200
controlpoints(i,1) = x(i);
controlpoints(i,2) = y(i);
controlpoints(i,3) = 0;
end
% Create Q with some points from originla matrix controlpoints ( I take only 103 points)
counter =1;
for i=1:200
if (i==11) || (i==20) || (i==198)
Q(counter,:) = F(i,:);
counter = counter +1;
end
if ne(mod(i,2),0)
Q(counter,:) = F(i,:);
counter = counter+1;
end
end
I used Centripetal method to find control points from curve like the following picture
Complete my code:
% 2- Create Centripetal Nodes array from Q
CP(1) = 0;
CP(103) =1;
for i=2:102
sum = 0;
for j=2:102
sum = sum + sqrt(sqrt((Q(j,1)-Q(j-1,1))^2+(Q(j,2)-Q(j-1,2))^2));
end
CP(i) = CP(i-1) + (sqrt(sqrt((Q(i,1)-Q(i-1,1))^2+(Q(i,2)-Q(i-1,2))^2))/sum);
end
p=3; % degree
% 3- Create U_K array from CP array
for i=1:103
U_K(i) = CP(i);
end
To calculate control points we must follow this equation P=Qx(R') --> R' is inverse of R matrix so we must find R matrix then fins P(control points matrix) by the above equation. The following scenario is used to find R matrix
and to calculate N in B-Spline we must use these recursive function
Complete my code :
% 5- Calculate R_i_p matrix
for a=1:100
for b=1:100
R_i_p(a,b) = NCalculate(b,p,U_K(a),U_K);
end
end
% 6- Find inverse of R_i_p matrix
R_i_p_invers = inv(R_i_p);
% 7- Find Control points ( 100 points because we have curve with 3 degree )
for i=1:100
for k=1:100
PX(i) = R_inv(i,k) * Q(k,1);
PY(i) = R_inv(i,k) * Q(k,2);
end
end
PX2 = transpose(PX);
PY2 = transpose(PY);
P = horzcat(PX2,PY2); % The final control points you can see the values is very small compared with the original data points vlaues
My Recursive Function to find the previous R matrix:
function z = NCalculate(j,k,u,U)
if (k == 1 )
if ( (u > U(j)) && (u <= U(j+1)) )
z = 1;
else
z = 0;
end
else
z = (u-U(j)/U(j+k-1)-U(j)* NCalculate(j,k-1,u,U) ) + (U(j+k)-u/U(j+k)-U(j+1) * NCalculate(j+1,k-1,u,U));
end
end
Really I need to this help so much , I tried in this problem from one week :(
Updated:
Figure 1 for the main B-spline Curve , Figure 2 for the result control points after applied reverse engineering on this curve so the value is so far and so small compared with the original data points value
Updated(2):
I made some updates on my code but the problem now in the inverse of R matrix it gives me infinite value at all time
% 2- Create Centripetal Nodes array from Q
CP(1) = 0;
CP(100) =1;
sum = 0;
for i=2:100
sum = sum + sqrt(sqrt((Q(i,1)-Q(i-1,1))^2+(Q(i,2)-Q(i-1,2))^2));
end
for i=2:99
CP(i) = CP(i-1) + (sqrt(sqrt((Q(i,1)-Q(i-1,1))^2+(Q(i,2)-Q(i-1,2))^2))/sum);
end
% 3- Create U_K array from CP array
for i=1:100
U_K(i) = CP(i);
end
p=3;
% create Knot vector
% The first elements
for i=1:p+1
U(i) = 0;
end
% The last elements
for i=100:99+p+1
U(i) = 1;
end
% The remain elements
for j=2:96
sum = 0;
for i=j:(j+p-1)
sum = sum + U_K(i);
end
U(j+p) = (1/p)* sum;
end
% 5- Calculate R_i_p matrix
for a=1:100
for b=1:100
R_i_p(a,b) = NCalculate(b,p,U_K(a),U);
end
end
R_i_p_invers = inv(R_i_p);
% 7- Find Control points ( 100 points )
for i=1:100
for k=1:100
if isinf(R_inv(i,k))
R_inv(i,k) = 0;
end
PX(i) = R_inv(i,k) * Q(k,1);
PY(i) = R_inv(i,k) * Q(k,2);
end
end
PX2 = transpose(PX);
PY2 = transpose(PY);
P = horzcat(PX2,PY2);
Note: I updated my NCalculate recursive function to give me 0 if the result is NaN (not a number )
function z = NCalculate(j,k,u,U)
if (k == 1 )
if ( (u >= U(j)) && (u < U(j+1)) )
z = 1;
else
z = 0;
end
else
z = (u-U(j)/U(j+k-1)-U(j)* NCalculate(j,k-1,u,U) ) + (U(j+k)-u/U(j+k)-U(j+1) * NCalculate(j+1,k-1,u,U));
end
if isnan(z)
z =0;
end
end
I think there are a few dubious issues in your approach:
First of all, if you try to create a b-spline curve interpolating 103 input points (and no other boundary conditions are imposed), the b-spline curve will have 103 control points regardless what degree the b-spline curve is.
The U_K array is the parameter assigned to each input point. They are not the same as the knot sequence ti used by the Cox DeBoor recursive formula. If the b-spline curve is of degree 3, you shall have (103+3+1) knot values in the knot sequence. You can create the knot values in the following way:
0) Denote the parameters as p[i], where i = 0 to (n-1), p[0]=0.0 and n is number of points.
1) Create the knot values as
knot[0] = (p[1]+p[2]+p[3])/D (where D is degree)
knot[1] = (p[2]+p[3]+p[4])/D
knot[2] = (p[3]+p[4]+p[5])/D
......
These are the interior knot values. You should notice that p[0] and p[n-1] will not be used in this step. You will have (n-D-1) interior knots.
2) Now, add p[0] to the front of the knot values (D+1) times and add p[n-1] to the end of the knot values (D+1) times and you are done. At the end, you will have (N+D+1) knots in total.
I have two matrices of big sizes, which are something similar to the following matrices.
m; with size 1000 by 10
n; with size 1 by 10.
I would like to subtract each element of n from all elements of m to get ten different matrices, each has size of 1000 by 10.
I started as follows
clc;clear;
nrow = 10000;
ncol = 10;
t = length(n)
for i = 1:nrow;
for j = 1:ncol;
for t = 1:length(n);
m1(i,j) = m(i,j)-n(1);
m2(i,j) = m(i,j)-n(2);
m3(i,j) = m(i,j)-n(3);
m4(i,j) = m(i,j)-n(4);
m5(i,j) = m(i,j)-n(5);
m6(i,j) = m(i,j)-n(6);
m7(i,j) = m(i,j)-n(7);
m8(i,j) = m(i,j)-n(8);
m9(i,j) = m(i,j)-n(9);
m10(i,j) = m(i,j)-n(10);
end
end
end
can any one help me how can I do it without writing the ten equations inside the loop? Or can suggest me any convenient way especially when the two matrices has many columns.
Why can't you just do this:
m01 = m - n(1);
...
m10 = m - n(10);
What do you need the loop for?
Even better:
N = length(n);
m2 = cell(N, 1);
for k = 1:N
m2{k} = m - n(k);
end
Here we go loopless:
nrow = 10000;
ncol = 10;
%example data
m = ones(nrow,ncol);
n = 1:ncol;
M = repmat(m,1,1,ncol);
N = permute( repmat(n,nrow,1,ncol) , [1 3 2] );
result = bsxfun(#minus, M, N );
%or just
result = M-N;
Elapsed time is 0.018499 seconds.
or as recommended by Luis Mendo:
M = repmat(m,1,1,ncol);
result = bsxfun(#minus, m, permute(n, [1 3 2]) );
Elapsed time is 0.000094 seconds.
please make sure that your input vectors have the same orientation like in my example, otherwise you could get in trouble. You should be able to obtain that by transposements or you have to modify this line:
permute( repmat(n,nrow,1,ncol) , [1 3 2] )
according to your needs.
You mentioned in a comment that you want to count the negative elements in each of the obtained columns:
A = result; %backup results
A(A > 0) = 0; %set non-negative elements to zero
D = sum( logical(A),3 );
which will return the desired 10000x10 matrix with quantities of negative elements. (Please verify it, I may got a little confused with the dimensions ;))
Create the three dimensional result matrix. Store your results, for example, in third dimension.
clc;clear;
nrow = 10000;
ncol = 10;
N = length(n);
resultMatrix = zeros(nrow, ncol, N);
neg = zeros(ncol, N); % amount of negative values
for j = 1:ncol
for i = 1:nrow
for t = 1:N
resultMatrix(i,j,t) = m(i,j) - n(t);
end
end
for t = 1:N
neg(j,t) = length( find(resultMatrix(:,j,t) < 0) );
end
end