Related
I have the following code:
x = VarName3;
y = VarName4;
x = (x/6000)/60;
plot(x, y)
Where VarName3 and VarName4 are 3000x1. I would like to apply a median filter to this in MATLAB. However, the problem I am having is that, if I use medfilt1, then I can only enter a single array of variables as the first argument. And for medfilt2, I can only enter a matrix as the first argument. But the data looks very obscured if I convert x and y into a matrix.
The x is time and y is a list of integers. I'd like to be able to filter out spikes and dips. How do I go about doing this? I was thinking of just eliminating the erroneous data points by direct manipulation of the data file. But then, I don't really get the effect of a median filter.
I found a solution using sort.
Median is the center element, so you can sort three elements, and take the middle element as median.
sort function also returns the index of the previous syntaxes.
I used the index information for restoring the matching value of X.
Here is my code sample:
%X - simulates time.
X = [1 2 3 4 5 6 7 8 9 10];
%Y - simulates data
Y = [0 1 2 0 100 1 1 1 2 3];
%Create three vectors:
Y0 = [0, Y(1:end-1)]; %Left elements [0 0 1 2 0 2 1 1 1 2]
Y1 = Y; %Center elements [0 1 2 0 2 1 1 1 2 3]
Y2 = [Y(2:end), 0]; %Right elements [1 2 0 2 1 1 1 2 3 0]
%Concatenate Y0, Y1 and Y2.
YYY = [Y0; Y1; Y2];
%Sort YYY:
%sortedYYY(2, :) equals medfilt1(Y)
%I(2, :) equals the index: value 1 for Y0, 2 for Y1 and 3 for Y2.
[sortedYYY, I] = sort(YYY);
%Median is the center of sorted 3 elements.
medY = sortedYYY(2, :);
%Corrected X index of medY
medX = X + I(2, :) - 2;
%Protect X from exceeding original boundries.
medX = min(max(medX, min(X)), max(X));
Result:
medX =
1 2 2 3 6 7 7 8 9 9
>> medY
medY =
0 1 1 2 1 1 1 1 2 2
Use a sliding window on the data vector centred at a given time. The value of your filtered output at that time is the median value of the data in the sliding window. The size of the sliding window is an odd value, not necessarily fixed to 3.
Let z = [1 3 5 6] and by getting all the difference between each elements:
we get:
bsxfun(#minus, z', z)
ans =
0 -2 -4 -5
2 0 -2 -3
4 2 0 -1
5 3 1 0
I now want to order these values in ascending order and remove the duplicates. So:
sort(reshape(bsxfun(#minus, z', z),1,16))
ans =
Columns 1 through 13
-5 -4 -3 -2 -2 -1 0 0 0 0 1 2 2
Columns 14 through 16
3 4 5
C = unique(sort(reshape(bsxfun(#minus, z', z),1,16)))
C =
-5 -4 -3 -2 -1 0 1 2 3 4 5
But by looking at -5 in [-5 -4 -3 -2 -1 0 1 2 3 4 5],
how can I tell where -5 comes from. By reading myself the matrix,
0 -2 -4 -5
2 0 -2 -3
4 2 0 -1
5 3 1 0
I know it comes from z(1) - z(4), i.e. row 1 column 4.
Also 2 comes from both z(3) - z(2) and z(2) - z(1), which comes from two cases. Without reading the originally matrix itself, how can we know that the 2 in [-5 -4 -3 -2 -1 0 1 2 3 4 5] is originally in row 3 column 2 and row 2 column 1 of the original matrix?
So by looking at each element in [-5 -4 -3 -2 -1 0 1 2 3 4 5], how do we know, for example, where -5 comes from in the original matrix index efficiently. I want to know as I need to do operation on ,e.g.,-5 and two indices that produce this: for example, for each difference, say -5, i do (-5)*1*6, as z(1)- z(6) = -5. But for 2, I need to do 2*(3*2+2*1) as z(3) - z(2) = 2, z(2) - z(1) = 2 which is not distinct.
Thinking hard, I think i should not reshape bsxfun(#minus, z', z) to array. I will also create two index array such that I can do operations like (-5)*1*6 stated above effectively. However, this is easier said than done and I also have to take care of nondistinct sources. Or should I do the desired operations first?
Use the third output from unique. And don't sort, unique will do that for you.
[sortedOutput,~,linearIndices] = unique(reshape(bsxfun(#minus, z', z),[1 16]))
You can reconstruct the result from bsxfun like so:
distances = reshape(sortedOutput(linearIndices),[4 4]);
If you want to know where a certain value appears, you write
targetValue = -5;
targetValueIdx = find(sortedOutput==targetValue);
linearIndexIntoDistances = find(targetValueIdx==linearIndices);
[row,col] = ind2sub([4 4],linearIndexIntoDistances);
Because linearIndices is 1 wherever the first value in sortedOutput appears in the original vector.
If you save the result of bsxfun in an intermediate variable:
distances=bsxfun(#minus, z', z)
Then you can look for the values of C in distances using find iteratively.
[rows,cols]=find(C(i)==distances)
This will give all rows and cols if the values are repeated. You just need to then use them for your equation.
You can use accumarray to collect all row and column indices that correspond to the same value in the matrix of differences:
z = [1 3 5 6]; % data vector
zd = bsxfun(#minus, z.', z); % matrix of differences
[C, ~, ind] = unique(zd); % unique values and indices
[rr, cc] = ndgrid(1:numel(z)); % template for row and col indices
f = #(x){x}; % anonymous function to collect row and col indices
row = accumarray(ind, rr(:), [], f); % group row indices according to ind
col = accumarray(ind, cc(:), [], f); % same for col indices
For example, C(6) is value 0, which appears four times in zd, at positions given by row{6} and col{6}:
>> row{6}.'
ans =
3 2 1 4
>> col{6}.'
ans =
3 2 1 4
As you see, the results are not guaranteed to be sorted. If you need to sort them in linear order:
rowcol = cellfun(#(r,c)sortrows([r c]), row, col, 'UniformOutput', false);
so now
>> rowcol{6}
ans =
1 1
2 2
3 3
4 4
I'm not sure I've followed exactly but some points to consider:
unique will sort the data for you by default so you don't need to call sort first
unique actually has three outputs and you can recover your original vector (i.e. with duplicates) using the third output so
[C,~,ic] = unique(reshape(bsxfun(#minus, z', z),1,16))
now you can get back to bsxfun(#minus, z', z),1,16) by calling
reshape(C(ic), numel(z), numel(z))
You might be more interested in the second output of unique which tells you what index each unique value was at in your 1-by-16 vector. It really depends on what you're trying to do though. But with this you could get a list of row column pairs to match your unique values:
[rows, cols] = ndgrid(1:4);
coords = [rows(:), cols(:)];
[C, ia] = unique(reshape(bsxfun(#minus, z', z),1,16));
coords_pairs = coords(ia,:)
which results in
coords_pairs =
1 4
1 3
2 4
2 3
3 4
4 4
4 3
3 2
4 2
3 1
4 1
I have a matrix that I'd like to create a new ordering of, for example,
vals = [1 2; 3 4]
I also have two matrices, new_x and new_y such that new_x(a,b) = j and new_x(a,b) = k means that I want the value at vals (a,b) to be mapped to new_vals(j,k).
For example, given
new_x = [1 2; 2 1]
new_y = [2 2; 1 1]
I'd want
new_vals = [4 3; 1 2]
I understand that I could just write two for loops to build the new array, but matlab is notoriously good at providing operations on entire matricies. My question is, how would I build new_vals without the for loops?
Basically you are trying to get a matrix that when indexed with new_x and new_y would give us vals, i.e. -
output(new_x(1,1),new_y(1,1)) must be equal to vals(1,1),
output(new_x(1,2),new_y(1,2)) must be equal to vals(1,2) and so on.
We will try to verify this later on. For now, here's one solution using linear indexing -
nrows = size(vals,1); %// Store number of rows
%// Calculate linear indices
idx = (new_x + (new_y-1)*nrows);
%// Trace/map back to sorted version of "1:numel(vals)"
[~,traced_back_idx] = sort(idx(:));
%// Index into vals with traced back linear indices & then reshape & transpose
out = reshape(vals(traced_back_idx),[],nrows).'
Here's another and possibly faster way -
out = nan(size(vals));
out((new_x + (new_y-1)*nrows)) = vals;
out = out.'
As discussed earlier for verification, let's index into out with new_x and new_y and that should match up with vals. Here's a code to do so -
for ii = 1:size(out,1)
for jj = 1:size(out,2)
check_back(ii,jj) = out(new_y(ii,jj),new_x(ii,jj));
end
end
Sample runs -
Case #1 (sample from question):
vals =
1 2
3 4
new_x =
1 2
2 1
new_y =
2 2
1 1
new_vals =
4 3
1 2
out =
4 3
1 2
check_back = (must be same as vals)
1 2
3 4
Case #2:
vals =
1 2 5
3 4 5
6 8 3
new_x =
1 2 3
3 1 2
3 2 1
new_y =
2 2 3
2 1 1
1 3 3
out =
4 5 6
1 2 3
3 8 5
check_back = (must be same as vals)
1 2 5
3 4 5
6 8 3
I think i see what you are trying to do here. new_x and new_y are just coordinates for the new_val matrix rigth? The problem is tha what you are trying to do only works for vectors, not for matrix, so the only way is to transform the matrix into a vector, reorder the values and then go back to matrix like:
vals = [1 ,2; 3, 4];
A=reshape(vals,1,4); % A is a vector [ 1 3 2 4]
new_coord=[2,3,4,1];
B(new_c)=A; %B is [4 1 3 2]
new_val=reshape(B,2,2) %back to matrix
Obtainig new_val=[4 3; 1 2]. Also B=A(new_c) is also allowed but with different coordinates, eventhoug is much easy to think the rigth coordinates in that way.
I am sure there must be a way to include the new_x matrix and transform everything into new_coord
In Matlab I have a big matrix containing the coordinates (x,y,z) of many points (over 200000). There is an extra column used as identification. I have written this code in order to sort all coordinate points. My final goal is to find duplicated points (rows with same x,y,z). After sorting the coordinate points I use the diff function, two consecutive rows of the matrix with the same coordinates will take value [0 0 0], and then with ismember I can find which rows of that matrix resulting from applying "diff" have the [0 0 0] row. With the indices returned from ismember I can find which points are repeated.
Back to my question...This is the code I wrote to sort properly my coordintes+id matrix. I guess It could be done better. Any suggestion?
%coordinates are always positive
a=[ 1 2 8 4; %sample matrix
1 0 5 6;
2 4 7 1;
3 2 1 0;
2 3 5 0;
3 1 2 8;
1 2 4 8];
b=a; %for checking purposes
%sorting first column
a=sortrows(a,1);
%sorting second column
for i=0:max(a(:,1));
k=find(a(:,1)==i);
if not(isempty(k))
a(k,:)=sortrows(a(k,:),2);
end
end
%Sorting third column
for i=0:max(a(:,2));
k=find(a(:,2)==i);
if not(isempty(k))
%identifying rows with same value on first column
for j=1:length(k)
[rows,~] = ismember(a(:,1:2), [ a(k(j),1),i],'rows');
a(rows,3:end)=sortrows(a(rows,3:end),1);
end
end
end
%Checking that rows remain the same
m=ismember(b,a,'rows');
if length(m)~=sum(m)
disp('Error while sorting!');
end
Why don't you just use unique?
[uniqueRows, ii, jj] = unique(a(:,1:3),'rows');
Example
a = [1 2 3 5
3 2 3 6
1 2 3 9
2 2 2 8];
gives
uniqueRows =
1 2 3
2 2 2
3 2 3
and
jj =
1
3
1
2
meaning third row equals first row.
If you need the full unique rows, including the fourth column: use ii to index a:
fullUniqueRows = a(ii,:);
which gives
fullUniqueRows =
1 2 3 9
2 2 2 8
3 2 3 6
Trying to sort a based on the fourth column? Do this -
a=[ 1 2 8 4; %sample matrix
1 0 5 6;
2 4 7 1;
3 2 1 0;
2 3 5 0;
3 2 1 8;
1 2 4 8];
[x,y] = sort(a(:,4))
sorted_a=a(y,:)
Trying to get the row indices having repeated x-y-z coordinates being represented by the first three columns? Do this -
out = sum(squeeze(all(bsxfun(#eq,a(:,1:3),permute(a(:,1:3),[3 2 1])),2)),2)>1
and use it similarly for sorted_a.
just lets make it simple, assume that I have a 10x3 matrix in matlab. The numbers in the first two columns in each row represent the x and y (position) and the number in 3rd columns show the corresponding value. For instance, [1 4 12] shows that the value of function in x=1 and y=4 is equal to 12. I also have same x, and y in different rows, and I want to average the values with same x,y. and replace all of them with averaged one.
For example :
A = [1 4 12
1 4 14
1 4 10
1 5 5
1 5 7];
I want to have
B = [1 4 12
1 5 6]
I really appreciate your help
Thanks
Ali
Like this?
A = [1 4 12;1 4 14;1 4 10; 1 5 5;1 5 7];
[x,y] = consolidator(A(:,1:2),A(:,3),#mean);
B = [x,y]
B =
1 4 12
1 5 6
Consolidator is on the File Exchange.
Using built-in functions:
sparsemean = accumarray(A(:,1:2), A(:,3).', [], #mean, 0, true);
[i,j,v] = find(sparsemean);
B = [i.' j.' v.'];
A = [1 4 12;1 4 14;1 4 10; 1 5 5;1 5 7]; %your example data
B = unique(A(:, 1:2), 'rows'); %find the unique xy pairs
C = nan(length(B), 1);
% calculate means
for ii = 1:length(B)
C(ii) = mean(A(A(:, 1) == B(ii, 1) & A(:, 2) == B(ii, 2), 3));
end
C =
12
6
The step inside the for loop uses logical indexing to find the mean of rows that match the current xy pair in the loop.
Use unique to get the unique rows and use the returned indexing array to find the ones that should be averaged and ask accumarray to do the averaging part:
[C,~,J]=unique(A(:,1:2), 'rows');
B=[C, accumarray(J,A(:,3),[],#mean)];
For your example
>> [C,~,J]=unique(A(:,1:2), 'rows')
C =
1 4
1 5
J =
1
1
1
2
2
C contains the unique rows and J shows which rows in the original matrix correspond to the rows in C then
>> accumarray(J,A(:,3),[],#mean)
ans =
12
6
returns the desired averages and
>> B=[C, accumarray(J,A(:,3),[],#mean)]
B =
1 4 12
1 5 6
is the answer.