Related
I'm trying to estimate the (unknown) original datapoints that went into calculating a (known) moving average. However, I do know some of the original datapoints, and I'm not sure how to use that information.
I am using the method given in the answers here: https://stats.stackexchange.com/questions/67907/extract-data-points-from-moving-average, but in MATLAB (my code below). This method works quite well for large numbers of data points (>1000), but less well with fewer data points, as you'd expect.
window = 3;
datapoints = 150;
data = 3*rand(1,datapoints)+50;
moving_averages = [];
for i = window:size(data,2)
moving_averages(i) = mean(data(i+1-window:i));
end
length = size(moving_averages,2)+(window-1);
a = (tril(ones(length,length),window-1) - tril(ones(length,length),-1))/window;
a = a(1:length-(window-1),:);
ai = pinv(a);
daily = mtimes(ai,moving_averages');
x = 1:size(data,2);
figure(1)
hold on
plot(x,data,'Color','b');
plot(x(window:end),moving_averages(window:end),'Linewidth',2,'Color','r');
plot(x,daily(window:end),'Color','g');
hold off
axis([0 size(x,2) min(daily(window:end))-1 max(daily(window:end))+1])
legend('original data','moving average','back-calculated')
Now, say I know a smattering of the original data points. I'm having trouble figuring how might I use that information to more accurately calculate the rest. Thank you for any assistance.
You should be able to calculate the original data exactly if you at any time can exactly determine one window's worth of data, i.e. in this case n-1 samples in a window of length n. (In your case) if you know A,B and (A+B+C)/3, you can solve now and know C. Now when you have (B+C+D)/3 (your moving average) you can exactly solve for D. Rinse and repeat. This logic works going backwards too.
Here is an example with the same idea:
% the actual vector of values
a = cumsum(rand(150,1) - 0.5);
% compute moving average
win = 3; % sliding window length
idx = hankel(1:win, win:numel(a));
m = mean(a(idx));
% coefficient matrix: m(i) = sum(a(i:i+win-1))/win
A = repmat([ones(1,win) zeros(1,numel(a)-win)], numel(a)-win+1, 1);
for i=2:size(A,1)
A(i,:) = circshift(A(i-1,:), [0 1]);
end
A = A / win;
% solve linear system
%x = A \ m(:);
x = pinv(A) * m(:);
% plot and compare
subplot(211), plot(1:numel(a),a, 1:numel(m),m)
legend({'original','moving average'})
title(sprintf('length = %d, window = %d',numel(a),win))
subplot(212), plot(1:numel(a),a, 1:numel(a),x)
legend({'original','reconstructed'})
title(sprintf('error = %f',norm(x(:)-a(:))))
You can see the reconstruction error is very small, even using the data sizes in your example (150 samples with a 3-samples moving average).
I have a set of three vectors (stored into a 3xN matrix) which are 'entangled' (e.g. some value in the second row should be in the third row and vice versa). This 'entanglement' is based on looking at the figure in which alpha2 is plotted. To separate the vector I use a difference based approach where I calculate the difference of one value with respect the three next values (e.g. comparing (1,i) with (:,i+1)). Then I take the minimum and store that. The method works to separate two of the three vectors, but not for the last.
I was wondering if you guys can share your ideas with me how to solve this problem (if possible). I have added my coded below.
Thanks in advance!
Problem in figures:
clear all; close all; clc;
%%
alpha2 = [-23.32 -23.05 -22.24 -20.91 -19.06 -16.70 -13.83 -10.49 -6.70;
-0.46 -0.33 0.19 2.38 5.44 9.36 14.15 19.80 26.32;
-1.58 -1.13 0.06 0.70 1.61 2.78 4.23 5.99 8.09];
%%% Original
figure()
hold on
plot(alpha2(1,:))
plot(alpha2(2,:))
plot(alpha2(3,:))
%%% Store start values
store1(1,1) = alpha2(1,1);
store2(1,1) = alpha2(2,1);
store3(1,1) = alpha2(3,1);
for i=1:size(alpha2,2)-1
for j=1:size(alpha2,1)
Alpha1(j,i) = abs(store1(1,i)-alpha2(j,i+1));
Alpha2(j,i) = abs(store2(1,i)-alpha2(j,i+1));
Alpha3(j,i) = abs(store3(1,i)-alpha2(j,i+1));
[~, I] = min(Alpha1(:,i));
store1(1,i+1) = alpha2(I,i+1);
[~, I] = min(Alpha2(:,i));
store2(1,i+1) = alpha2(I,i+1);
[~, I] = min(Alpha3(:,i));
store3(1,i+1) = alpha2(I,i+1);
end
end
%%% Plot to see if separation worked
figure()
hold on
plot(store1)
plot(store2)
plot(store3)
Solution using extrapolation via polyfit:
The idea is pretty simple: Iterate over all positions i and use polyfit to fit polynomials of degree d to the d+1 values from F(:,i-(d+1)) up to F(:,i). Use those polynomials to extrapolate the function values F(:,i+1). Then compute the permutation of the real values F(:,i+1) that fits those extrapolations best. This should work quite well, if there are only a few functions involved. There is certainly some room for improvement, but for your simple setting it should suffice.
function F = untangle(F, maxExtrapolationDegree)
%// UNTANGLE(F) untangles the functions F(i,:) via extrapolation.
if nargin<2
maxExtrapolationDegree = 4;
end
extrapolate = #(f) polyval(polyfit(1:length(f),f,length(f)-1),length(f)+1);
extrapolateAll = #(F) cellfun(extrapolate, num2cell(F,2));
fitCriterion = #(X,Y) norm(X(:)-Y(:),1);
nFuncs = size(F,1);
nPoints = size(F,2);
swaps = perms(1:nFuncs);
errorOfFit = zeros(1,size(swaps,1));
for i = 1:nPoints-1
nextValues = extrapolateAll(F(:,max(1,i-(maxExtrapolationDegree+1)):i));
for j = 1:size(swaps,1)
errorOfFit(j) = fitCriterion(nextValues, F(swaps(j,:),i+1));
end
[~,j_bestSwap] = min(errorOfFit);
F(:,i+1) = F(swaps(j_bestSwap,:),i+1);
end
Initial solution: (not that pretty - Skip this part)
This is a similar solution that tries to minimize the sum of the derivatives up to some degree of the vector valued function F = #(j) alpha2(:,j). It does so by stepping through the positions i and checks all possible permutations of the coordinates of i to get a minimal seminorm of the function F(1:i).
(I'm actually wondering right now if there is any canonical mathematical way to define the seminorm so we get our expected results... I initially was going for the H^1 and H^2 seminorms, but they didn't quite work...)
function F = untangle(F)
nFuncs = size(F,1);
nPoints = size(F,2);
seminorm = #(x,i) sum(sum(abs(diff(x(:,1:i),1,2)))) + ...
sum(sum(abs(diff(x(:,1:i),2,2)))) + ...
sum(sum(abs(diff(x(:,1:i),3,2)))) + ...
sum(sum(abs(diff(x(:,1:i),4,2))));
doSwap = #(x,swap,i) [x(:,1:i-1), x(swap,i:end)];
swaps = perms(1:nFuncs);
normOfSwap = zeros(1,size(swaps,1));
for i = 2:nPoints
for j = 1:size(swaps,1)
normOfSwap(j) = seminorm(doSwap(F,swaps(j,:),i),i);
end
[~,j_bestSwap] = min(normOfSwap);
F = doSwap(F,swaps(j_bestSwap,:),i);
end
Usage:
The command alpha2 = untangle(alpha2); will untangle your functions:
It should even work for more complicated data, like these shuffled sine-waves:
nPoints = 100;
nFuncs = 5;
t = linspace(0, 2*pi, nPoints);
F = bsxfun(#(a,b) sin(a*b), (1:nFuncs).', t);
for i = 1:nPoints
F(:,i) = F(randperm(nFuncs),i);
end
Remark: I guess if you already know that your functions will be quadratic or some other special form, RANSAC would be a better idea for larger number of functions. This could also be useful if the functions are not given with the same x-value spacing.
So I am trying to go through a for loop that will increment .1 every time and will do this until the another variable h is less than or equal to zero. Then I am suppose to graph this h variable along another variable x. The code that I wrote looks like this:
O = 20;
v = 200;
g = 32.2;
for t = 0:.1:12
% Calculate the height
h(t) = (v)*(t)*(sin(O))-(1/2)*(g)*(t^2);
% Calculate the horizontal location
x(t) = (v)*(t)*cos(O);
if t > 0 && h <= 0
break
end
end
The Error that I keep getting when running this code says "Attempted to access h(0); index must be a positive integer or logical." I don't understand what exactly is going on in order for this to happen. So my question is why is this happening and is there a way I can solve it, Thank you in advance.
You're using t as your loop variable as well as your indexing variable. This doesn't work, because you'll try to access h(0), h(0.1), h(0.2), etc, which doesn't make sense. As the error says, you can only access variables using integers. You could replace your code with the following:
t = 0:0.1:12;
for i = 1:length(t)
% use t(i) instead of t now
end
I will also point out that you don't need to use a for loop to do this. MATLAB is optimised for acting on matrices (and vectors), and will in general run faster on vectorised functions rather than for loops. For instance, your equation for h could be replaced with the following:
O = 20;
v = 200;
g = 32.2;
t = 0:0.1:12;
h = v * t * sin(O) - 0.5 * g * t.^2;
The only difference is that you have to use the element-wise square (.^2) rather than the normal square (^2). This means that MATLAB will square each element of the vector t, rather than multiplying the vector t by itself.
In short:
As the error says, t needs to be an integer or logical.
But your t is t=0:0.1:12, therefore a decimal value.
O = 20;
v = 200;
g = 32.2;
for t = 0:.1:12
% Calculate the height
idx_t = 1:numel(t);
h(idx_t) = (v)*(t)*(sin(O))-(1/2)*(g)*(t^2);
% Calculate the horizontal location
x(idx_t) = (v)*(t)*cos(O);
if t > 0 && h <= 0
break
end
end
Look this question's answer for more options: Subscript indices must either be real positive integers or logical error
I have been trying to solve the non dimensional Fisher Kolmagorov equation in Matlab. I am getting a graph which doesn't look at all like it should. Also, I'm getting the equation independent of value of s (the source term in the pdepe solver). No matter what value of s I put in the graph remains the same.
function FK
m = 0;
x = linspace(0,1,100);
t = linspace(0,1,100);
u = pdepe(m,#FKpde,#FKic,#FKbc,x,t);
[X,T] = meshgrid(x,t);
%ANALYTICAL SOLUTION
% a=(sqrt(2))-1;
% q=2;
% s=2/q;
% b= q /((2*(q+2))^0.5);
% c= (q+4)/((2*(q+2))^0.5);
% zeta= X-c*T;
%y = 1/((1+(a*(exp(b*zeta))))^s);
%r=(y(:,:)-u(:,:))./y(:,:); % relative error in numerical and analytical value
figure;
plot(x,u(10,:),'o',x,u(40,:),'o',x,u(60,:),'o',x,u(end,:),'o')
title('Numerical Solutions at different times');
legend('tn=1','tn=5','tn=30','tn=10','ta=20','ta=600','ta=800','ta=1000',0);
xlabel('Distance x');
ylabel('u(x,t)');
% --------------------------------------------------------------------------
function [c,f,s] = FKpde(x,t,u,DuDx)
c = 1;
f = DuDx;
s =u*(1-u);
% --------------------------------------------------------------------------
function u0 = FKic(x)
u0 = 10^(-4);
% --------------------------------------------------------------------------
function [pl,ql,pr,qr] = FKbc(xl,ul,xr,ur,t)
pl = ul-1;
ql = 0;
pr = ur;
qr = 0;
Should maybe be a comment, but putting it as an answer for better formatting. Your analytic solution, which I assume you're using to compare with the numerical answer to say that the graph does not look as it should, does not appear to respect the initial conditions or boundary conditions you are feeding pdepe, so I'd start there if trying to figure out why u does not look like y
The initial and boundary conditions you are setting are:
u(0, t) = 1
u(1, t) = 0
u(x, 0) = 1e-4
Setting aside that the boundary and initial conditions are in conflict, the analytic solution you suggest in the code has
u(0, t) = 1/((1+exp(-b*ct)))
u(1, t) = 1/((1+exp(b*(1-ct)))
u(x, 0) = 1/((1+exp(b*x))
So it seems to me the numerical and analytic solutions should be expected to be different, and the differences you observe are probably due to the IC/BC setup. I suspect that pdepe is probably solving the equation you are giving it.
On increasing the length scale and time scale, I get the answers I want. The problem was to solve for different times, and see the wave propogate. For small lenghts, I could only see part of that wave.
I am new to Matlab and I have to use fixed point iteration to find the x value for the intersection between y = x and y = sqrt(10/x+4), which after graphing it, looks to be around 1.4. I'm using an initial guess of x1 = 0. This is my current Matlab code:
f = #(x)sqrt(10./(x+4));
x1 = 0;
xArray(10) = [];
for i = 1:10
x2 = f(x1);
xArray(i) = x2;
x1 = x1 + 1;
end
plot(xArray);
fprintf('%15.8e\n',xArray);
Now when I run this it seems like my x is approaching 0.8. Can anyone tell me what I am doing wrong?
Well done. You've made a decent start at this.
Lets look at the graphical solution. BTW, this is how I'd have done the graphical part:
ezplot(#(x) x,[-1 3])
hold on
ezplot(#(x) sqrt(10./(x+4)),[-1 3])
grid on
Or, I might subtract the two functions, then looking for a zero of the difference, so where it crosses the x axis.
This is what the fixed point iteration does anyway, trying to solve for x, such that
x = sqrt(10/(x+4))
So how would I change your code to fix it? First of all, I'd want to use more descriptive names for the variables. You don't get charged by the character, and making your code easier to read & follow will pay off greatly in the future for you.
There were a couple of code issues. To initialize a vector, use a form like one of these:
xArray = zeros(1,10);
xArray(1,10) = 0;
Note that if xArray was ALREADY defined because you have been working on this problem, the latter form will only zero out that single element. So the first form is best by a large margin. It affirmatively creates an array, or overwrites an existing array if it is already present in your workspace.
Finally, I like to initialize an array like this with something special, rather than zero, so we can see when an element was overwritten. NaNs are good for this.
Next, there was no need to add one to x1 in your code. Again, I'd strongly suggest using better variable names. It is also a good idea to use comments. Be liberal.
I'd suggest the idea of a convergence tolerance. You can also have an iteration counter.
f = #(x)sqrt(10./(x+4));
% starting value
xcurrent = 0;
% count the iterations, setting a maximum in maxiter, here 25
iter = 0;
maxiter = 25;
% initialize the array to store our iterations
xArray = NaN(1,maxiter);
% convergence tolerance
xtol = 1e-8;
% before we start, the error is set to be BIG. this
% just lets our while loop get through that first iteration
xerr = inf;
% the while will stop if either criterion fails
while (iter < maxiter) && (xerr > xtol)
iter = iter + 1;
xnew = f(xcurrent);
% save each iteration
xArray(iter) = xnew;
% compute the difference between successive iterations
xerr = abs(xnew - xcurrent);
xcurrent = xnew;
end
% retain only the elements of xArray that we actually generated
xArray = xArray(1:iter);
plot(xArray);
fprintf('%15.8e\n',xArray);
What was the result?
1.58113883e+00
1.33856229e+00
1.36863563e+00
1.36479692e+00
1.36528512e+00
1.36522300e+00
1.36523091e+00
1.36522990e+00
1.36523003e+00
1.36523001e+00
1.36523001e+00
For a little more accuracy to see how well we did...
format long g
xcurrent
xcurrent =
1.36523001364783
f(xcurrent)
ans =
1.36523001338436
By the way, it is a good idea to know why the loop terminated. Did it stop for insufficient iterations?
The point of my response here was NOT to do your homework, since you were close to getting it right anyway. The point is to show some considerations on how you might improve your code for future work.
There is no need to add 1 to x1. your output from each iteration is input for next iteration. So, x2 from output of f(x1) should be the new x1. The corrected code would be
for i = 1:10
x2 = f(x1);
xArray(i) = x2;
x1 = x2;
end
f(x)x^3+4*x^2-10 in [1,2] find an approximate root