I am trying to compute a moving average on multiple columns of a matrix. After reading some answers on stackoverflow, namely this one, it seemed that the filter function was the way to go. However, it does not ignore NaN elements, and I would like to do this ignoring NaN elements in the spirit of the function nanmean. Below a sample code:
X = rand(100,100); %generate sample matrix
X(sort(randi([1 100],1,10)),sort(randi([1 100],1,10))) = NaN; %put some random NaNs
windowlenght = 7;
MeanMA = filter(ones(1, windowlenght) / windowlenght, 1, X);
Use colfilt with nanmean:
>> A = [1 2 3 4 5; 2 nan nan nan 6; 3 nan nan nan 7; 4 nan nan nan 8; 5 6 7 8 9]
A =
1 2 3 4 5
2 NaN NaN NaN 6
3 NaN NaN NaN 7
4 NaN NaN NaN 8
5 6 7 8 9
>> colfilt(A, [3,3], 'sliding', #nanmean)
ans =
0.6250 1.1429 1.5000 2.5714 1.8750
1.1429 2.2000 3.0000 5.0000 3.1429
1.5000 3.0000 NaN 7.0000 3.5000
2.5714 5.0000 7.0000 7.8000 4.5714
1.8750 3.1429 3.5000 4.5714 3.1250
(if you only care about 'full' blocks, select inner rows / columns appropriately)
Alternatively, you can also use nlfilter, but you then need to be explicit (via an anonymous function handle) about what you'll be doing with the block; in particular, to work with nanmean such that it will produce a scalar output from the whole block, you'll need to convert each block to a column-vector before calling nanmean in your anonymous function:
>> nlfilter(A, [3,3], #(x) nanmean(x(:)))
ans =
0.6250 1.1429 1.5000 2.5714 1.8750
1.1429 2.2000 3.0000 5.0000 3.1429
1.5000 3.0000 NaN 7.0000 3.5000
2.5714 5.0000 7.0000 7.8000 4.5714
1.8750 3.1429 3.5000 4.5714 3.1250
However, for the record, matlab claims colfilt will generally be faster, so generally nlfilter is better reserved for situations where it doesn't make sense for your input to be converted to a column when processing each block.
Also see matlab's manual page/chapter on sliding operations in general.
If you have R2016a or beyond, you can use the movmean function with the 'omitnan' option.
Try
MeanMA = filter(ones(1, windowlenght) / windowlenght, 1, X(find(~isnan(X)));
This will extract the non-nan values from X.
The question is... do you still have a valid filter processing? If X is filled iteratively, one element per timestep, then the "NaN-Elimination" will produce a shorter vector which values are not aligned with the original time vector any more.
EDIT
To still have a valid mean calculation, the filter parameters must be updated according to the number of non-NaN values.
values = X(find(~isnan(X));
templength = length(values);
MeanMA = filter(ones(1, templength ) / templength , 1, values );
Related
Is there a way to calculate a moving mean in a way that the values at the beginning and at the end of the array are averaged with the ones at the opposite end?
For example, instead of this result:
A=[2 1 2 4 6 1 1];
movmean(A,2)
ans = 2.0 1.5 1.5 3.0 5 3.5 1.0
I want to obtain the vector [1.5 1.5 1.5 3 5 3.5 1.0], as the initial array element 2 would be averaged with the ending element 1.
Generalizing to an arbitrary window size N, this is how you can add circular behavior to movmean in the way you want:
movmean(A([(end-floor(N./2)+1):end 1:end 1:(ceil(N./2)-1)]), N, 'Endpoints', 'discard')
For the given A and N = 2, you get:
ans =
1.5000 1.5000 1.5000 3.0000 5.0000 3.5000 1.0000
For an arbitrary window size n, you can use circular convolution with an averaging mask defined as [1/n ... 1/n] (with n entries; in your example n = 2):
result = cconv(A, repmat(1/n, 1, n), numel(A));
Convolution offers some nice ways of doing this. Though, you may need to tweak your input slightly if you are only going to partially average the ends (i.e. the first is averaged with the last in your example, but then the last is not averaged with the first).
conv([A(end),A],[0.5 0.5],'valid')
ans =
1.5000 1.5000 1.5000 3.0000 5.0000 3.5000 1.0000
The generalized case here, for a moving average of size N, is:
conv(A([end-N+2:end, 1:end]),repmat(1/N,1,N),'valid')
i have a `x=(t*n) stock return matrix, that n is number of stock in a portfolio and t is time. I want calculate
c=M{[x(it)-k(x)][y(it)-k(y)]}
where x(it)
return of stock i in time t
and the median M is taken with respect to the joint CDF of x(t)
and y(t), and k(x) and k(y) are the population medians of x(t) and y(t)
for example:
x=[1 2 3;6 7 5;3 5 6;7 8 9]
x =
1 2 3
6 7 5
3 5 6
7 8 9
t=size(x,1)
n=size(x,2)
medianx=median(x)
medianx =
4.5000 6.0000 5.5000
q=x-medianx(ones(t,1),:)
q =
-3.5000 -4.0000 -2.5000
1.5000 1.0000 -0.5000
-1.5000 -1.0000 0.5000
2.5000 2.0000 3.5000
I can do this to here and I don't know how can i reach c matrix in matlab. I calculate c manually that:
c =
4.2500 3.2500 4.0000
3.2500 2.5000 3.2500
4.0000 3.2500 3.2500
where
c(11)=median of(column1*colum1 of matrix q)=4.25
c(22)=median of(column2*colum2 of matrix q)=2.5
c(33)=median of(column3*colum3 of matrix q)=3.25
c(12) & c(21)=median of(column1*column2 of matrix q)=3.25
c(13) & c(31)=median of(column1*column3 of matrix q)=4
c(23) & c(32)=median of(column2*column3 of matrix q)=3.25
notice that i have a t*n matrix and matrix x just is a example. thanks
You can calculate c from q using this:
c=zeros(3,3); %Pre-allocation
for m=1:3
for n=1:3
c(m,n)= median(q(:,m).*q(:,n));
end
end
I have a matrix that looks something like this:
a=[1 1 2 2 3 3 4 4;
1.5 1.5 2.5 2.5 3.5 3.5 4.5 4.5]
what I would like to do is reshape this ie.
What I want is to take the 2x2 matrices next to one another and put them underneath each other.
So get:
b=[1 1;
1.5 1.5;
2 2;
2.5 2.5;
3 3;
3.5 3.5;
4 4;
4.5 4.5]
but I can't seem to manipulate the reshape function to do this for me
edit: the single line version might be a bit complicated, so I've also added one based on a for loop
2 reshapes and a permute should do it (we first split the matrices and store them in 3d), and then stack them. In order to stack them we first need to permute the dimensions (similar to a transpose).
>> reshape(permute(reshape(a,2,2,4),[1 3 2]),8,2)
ans =
1.0000 1.0000
1.5000 1.5000
2.0000 2.0000
2.5000 2.5000
3.0000 3.0000
3.5000 3.5000
4.0000 4.0000
4.5000 4.5000
the for loop based version is a bit more straight forward. We create an empty array of the correct size, and then insert each of the 2x2 matrices separately:
b=zeros(8,2);
for i=1:4,
b((2*i-1):(2*i),:) = a(:,(2*i-1):(2*i));
end
Using MATLAB, how can I find the 3-day moving average of a specific column of a matrix and append the moving average to that matrix? I am trying to compute the 3-day moving average from bottom to top of the matrix. I have provided my code:
Given the following matrix a and mask:
a = [1,2,3;4,5,6;7,8,9;10,11,12;13,14,15;16,17,18];
mask = ones(3,1);
I have tried implementing the conv command but I am receiving an error. Here is the conv command I have been trying to use on the 2nd column of matrix a:
a(:,4) = conv(a(:,2),mask,'valid');
The output I desire is given in the following matrix:
desiredOutput = [1,2,3,5;4,5,6,8;7,8,9,11;10,11,12,14;13,14,15,0;16,17,18,0;]
If you have any suggestions, I would greatly appreciate it. Thank you!
In general it would help if you would show the error. In this case you are doing two things wrong:
First your convolution needs to be divided by three (or the length of the moving average)
c = conv(a(:,2),mask,'valid')/3
c =
5
8
11
14
Second, notice the size of c. You cannot just fit c into a. The typical way of getting a moving average would be to use same:
a(:,4) = conv(a(:,2),mask,'same')/3
a =
1.0000 2.0000 3.0000 2.3333
4.0000 5.0000 6.0000 5.0000
7.0000 8.0000 9.0000 8.0000
10.0000 11.0000 12.0000 11.0000
13.0000 14.0000 15.0000 14.0000
16.0000 17.0000 18.0000 10.3333
but that doesn't look like what you want.
Instead you are forced to use a couple of lines:
c = conv(a(:,2),mask,'valid')/3;
a(1:length(c),4) = c
a =
1 2 3 5
4 5 6 8
7 8 9 11
10 11 12 14
13 14 15 0
16 17 18 0
I have a ~ 100000/2 matrix. I'd like to go down the columns, average each vertically adjacent value, and insert that value in between the two values. For example...
1 2
3 4
4 6
7 8
would become
1 2
2 3
3 4
3.5 5
4 6
5.5 7
7 8
I'm not sure if there is a terse way to do this in matlab. I took a look at http://www.mathworks.com/matlabcentral/fileexchange/9984 but it seems to insert all of the rows in a matrix into the other one at a specific point. Obviously it can still be used, but just wondering if there is a simpler way.
Any help is appreciated, thanks.
Untested:
% Take the mean of adjacent pairs
x_mean = ([x; 0 0] + [0 0; x]) / 2;
% Interleave the two matrices
y = kron(x, [1;0]) + kron(x_mean(1:end-1,:), [0;1]);
%# works for any 2D matrix of size N-by-M
X = rand(100,2);
adjMean = mean(cat(3, X(1:end-1,:), X(2:end,:)), 3);
Y = zeros(2*size(X,1)-1, size(X,2));
Y(1:2:end,:) = X;
Y(2:2:end,:) = adjMean;
octave-3.0.3:57> a = [1,2; 3,4; 4,6; 7,8]
a =
1 2
3 4
4 6
7 8
octave-3.0.3:58> b = (circshift(a, -1) + a) / 2
b =
2.0000 3.0000
3.5000 5.0000
5.5000 7.0000
4.0000 5.0000
octave-3.0.3:60> reshape(vertcat(a', b'), 2, [])'(1:end-1, :)
ans =
1.0000 2.0000
2.0000 3.0000
3.0000 4.0000
3.5000 5.0000
4.0000 6.0000
5.5000 7.0000
7.0000 8.0000