How to add standrad deviation and moving average - matlab

What I want to is:
I got folder with 32 txt files and 1 excle file, each file contain some data in two columns: time, level.
I already managed to pull the data from the folder and open each file in Matlab and get the data from it. What I need to do is create plot for each data file.
each of the 32 plots should have:
Change in average over time
Standard deviation
With both of this things I am straggling can't make it work.
also I need to make another plot this time the plot should have the average over each minute from all the 32 files.
here is my code until now:
clc,clear;
myDir = 'my path';
dirInfo = dir([myDir,'*.txt']);
filenames = {dirInfo.name};
N = numel(filenames);
data=cell(N,1);
for i=1:N
fid = fopen([myDir,filenames{i}] );
data{i} = textscan(fid,'%f %f','headerlines',2);
fclose(fid);
temp1=data{i,1};
time=temp1{1};
level=temp1{2};
Average(i)=mean(level(1:find(time>60)));
AverageVec=ones(length(time),1).*Average(i);
Standard=std(level);
figure(i);
plot(time,level);
xlim([0 60]);
hold on
plot(time, AverageVec);
hold on
plot(time, Standard);
legend('Level','Average','Standard Deviation')
end
the main problam with this code is that i get only average over all the 60 sec not moving average, and the standard deviation returns nothing.
few things you need to know:
*temp1 is 1x2 cell
*time and level are 22973x1 double.

Apperently you need an alternative to movmean and movstd since they where introduced in 2016a. I combined the suggestion from #bla with two loops that correct for the edge effects.
function [movmean,movstd] = moving_ms(vec,k)
if mod(k,2)==0,k=k+1;end
L = length(vec);
movmean=conv(vec,ones(k,1)./k,'same');
% correct edges
n=(k-1)/2;
movmean(1) = mean(vec(1:n+1));
N=n;
for ct = 2:n
movmean(ct) = movmean(ct-1) + (vec(ct+n) - movmean(ct-1))/N;
N=N+1;
end
movmean(L) = mean(vec((L-n):L));
N=n;
for ct = (L-1):-1:(L-n)
movmean(ct) = movmean(ct+1) + (vec(ct-n) - movmean(ct+1))/N;
N=N+1;
end
%mov variance
movstd = nan(size(vec));
for ct = 1:n
movstd(ct) = sum((vec(1:n+ct)-movmean(ct)).^2);
movstd(ct) = movstd(ct)/(n+ct-1);
end
for ct = n+1:(L-n)
movstd(ct) = sum((vec((ct-n):(ct+n))-movmean(ct)).^2);
movstd(ct) = movstd(ct)/(k-1);
end
for ct = (L-n):L
movstd(ct) = sum((vec((ct-n):L)-movmean(ct)).^2);
movstd(ct) = movstd(ct)/(L-ct+n);
end
movstd=sqrt(movstd);
Someone with matlab >=2016a can compare them using:
v=rand(1,1E3);m1 = movmean(v,101);s1=movstd(v,101);
[m2,s2] = moving_ms(v,101);
x=1:1E3;figure(1);clf;
subplot(1,2,1);plot(x,m1,x,m2);
subplot(1,2,2);plot(x,s1,x,s2);
It should show a single red line since the blue line is overlapped.

Related

Open multiple folders which have about 500 files under them and extract vtk files under them

I am trying to open multiple folders which have about 500 files under them and then use a function called vtkread to read the files in those folders. I am not sure how to set that up.
So here is my function but I am stuggling with setting up the mainscript to select files from a folder
function [Z_displacement,Pressure] = Processing_Code2_Results(filename, reduce_time, timestep_total)
fid = fopen(filename,'r');
Post_all = [];
vv=[1:500];
DANA0 = vtkRead('0_output_000000.vtk'); %extract all data from the vtk file including disp, pressure, points, times
C = [DANA0.points,reshape(DANA0.pointData.displacements,size(DANA0.points)),reshape(DANA0.pointData.pressure,[length(DANA0.points),1])];
disp0 = reshape(DANA0.pointData.displacements,[1,size(C,1),3]);
points = DANA0.points; % This is a matrix of the xyz points
for i = 1:reduce_time:timestep_total %34
DANA = vtkRead(sprintf('0_output_%06d.vtk',i)); % read in each successive timestep
disp(i,:,:) = DANA.pointData.displacements; % store displacement for multiple timesteps
pressure(i,:) = DANA.pointData.pressure; % store pressure for multiple timesteps
% press = pressure';
end
...
I have tried something like this:
clc; clear;
timestep_total = 500;
reduce_time = 100;
cd 'C:\Users\Admin\OneDrive - Kansas State University\PhD\Project\Modeling\SSGF_Model\New_Model_output'
for i = 1:3
filename = sprintf("Gotherm_%d",i)
[Z_displacement_{i},Pressure_{i}] = Processing_Code2_Results(filename, reduce_time, timestep_total);
end

Read all .csv-files in folder and plot their content

By an old post (https://stackoverflow.com/a/13744310/3900582) I have been able to read all the .csv-files in my folder into a cell array. Each .csv-file has the following structure:
0,1024
1,427
2,313
3,492
4,871
5,1376
6,1896
7,2408
8,2851
9,3191
Where the left column is the x-value and the right column is the y-value.
In total, there are almost 200 files and they are each up to 100 000 lines long. I would like to plot the contents of the files in one figure, to allow the data to be more closely inspected.
I was able to use the following code to solve my problem:
dd = dir('*.csv');
fileNames = {dd.name};
data = cell(numel(fileNames),2);
data(:,1) = regexprep(fileNames, '.csv','');
for i = 1:numel(fileNames)
data{i,2} = dlmread(fileNames{i});
end
fig=figure();
hold on;
for j = 1:numel(fileNames)
XY = data{j,2};
X = XY(:,1);
Y = XY(:,2);
plot(X,Y);
end

How to force MATLAB function area to hold on in figure

I'm working on this function which gets axis handler and data, and is supposed to plot it correctly in the axis. The function is called in for loop. It's supposed to draw the multiple data in one figure. My resulted figure is shown below.
There are only two correctly plotted graphs (those with four colors). Others miss areas plotted before the final area (red area is the last plotted area in each graph). But the script is same for every axis. So where can be the mistake? The whole function is written below.
function [] = powerSpectrumSmooth(axis,signal,fs)
N= length(signal);
samplesPer1Hz = N/fs;
delta = int16(3.5*samplesPer1Hz); %last sample of delta frequncies
theta = int16(7.5*samplesPer1Hz); %last sample of theta frequncies
alpha = int16(13*samplesPer1Hz); %last sample of alpha frequncies
beta = int16(30*samplesPer1Hz); %last sample of beta frequncies
x=fft(double(signal));
powerSpectrum = 20*log10(abs(real(x)));
smoothPS=smooth(powerSpectrum,51);
PSmin=min(powerSpectrum(1:beta));
y1=[(smoothPS(1:delta)); zeros(beta-delta,1)+PSmin];
y2=[zeros(delta-1,1)+PSmin; (smoothPS(delta:theta)); zeros(beta-theta,1)+PSmin];
y3=[zeros(theta-1,1)+PSmin; (smoothPS(theta:alpha)); zeros(beta-alpha,1)+PSmin];
y4=[zeros(alpha-1,1)+PSmin; (smoothPS(alpha:beta))];
a1=area(axis,1:beta,y1);
set(a1,'FaceColor','yellow')
hold on
a2=area(axis,1:beta,y2);
set(a2,'FaceColor','blue')
a3=area(axis,1:beta,y3);
set(a3,'FaceColor','green')
a4=area(axis,1:beta,y4);
set(a4,'FaceColor','red')
ADDED
And here is the function which calls the function above.
function [] = drawPowerSpectrum(axesContainer,dataContainer,fs)
size = length(axesContainer);
for l=1:size
powerSpectrumSmooth(axesContainer{l},dataContainer{l},fs)
set(axesContainer{l},'XTickLabel','')
set(axesContainer{l},'YTickLabel','')
uistack(axesContainer{l}, 'top');
end
ADDED 29th July
Here is a script which reproduces the error, so you can run it in your computer. Before running it again you might need to clear variables.
len = 9;
axesContainer = cell(len,1);
x = [0.1,0.4,0.7,0.1,0.4,0.7,0.1,0.4,0.7];
y = [0.1,0.1,0.1,0.4,0.4,0.4,0.7,0.7,0.7];
figure(1)
for i=1:len
axesContainer{i} = axes('Position',[x(i),y(i),0.2,0.2]);
end
dataContainer = cell(len,1);
N = 1500;
for i=1:len
dataContainer{i} = rand(1,N)*100;
end
for l=1:len
y1=[(dataContainer{l}(1:N/4)) zeros(1,3*N/4)];
y2=[zeros(1,N/4) (dataContainer{l}(N/4+1:(2*N/4))) zeros(1,2*N/4)];
y3=[zeros(1,2*N/4) (dataContainer{l}(2*N/4+1:3*N/4)) zeros(1,N/4)];
y4=[zeros(1,3*N/4) (dataContainer{l}(3*N/4+1:N))];
axes=axesContainer{l};
a1=area(axes,1:N,y1);
set(a1,'FaceColor','yellow')
hold on
a2=area(axes,1:N,y2);
set(a2,'FaceColor','blue')
hold on
a3=area(axes,1:N,y3);
set(a3,'FaceColor','green')
hold on
a4=area(axes,1:N,y4);
set(a4,'FaceColor','red')
set(axes,'XTickLabel','')
set(axes,'YTickLabel','')
end
My result of this script is plotted below:
Again only one picture contains all areas.
It looks like that every call to plot(axes,data) deletes whatever was written in axes.
Important note: Do not use a variable name the same as a function. Do not call something sin ,plot or axes!! I changed it to axs.
To solve the problem I just used the classic subplot instead of creating the axes as you did:
len = 9;
axesContainer = cell(len,1);
x = [0.1,0.4,0.7,0.1,0.4,0.7,0.1,0.4,0.7];
y = [0.1,0.1,0.1,0.4,0.4,0.4,0.7,0.7,0.7];
figure(1)
dataContainer = cell(len,1);
N = 1500;
for i=1:len
dataContainer{i} = rand(1,N)*100;
end
for l=1:len
y1=[(dataContainer{l}(1:N/4)) zeros(1,3*N/4)];
y2=[zeros(1,N/4) (dataContainer{l}(N/4+1:(2*N/4))) zeros(1,2*N/4)];
y3=[zeros(1,2*N/4) (dataContainer{l}(2*N/4+1:3*N/4)) zeros(1,N/4)];
y4=[zeros(1,3*N/4) (dataContainer{l}(3*N/4+1:N))];
axs=subplot(3,3,l);
a1=area(axs,1:N,y1);
set(a1,'FaceColor','yellow')
hold on
a2=area(axs,1:N,y2);
set(a2,'FaceColor','blue')
hold on
a3=area(axs,1:N,y3);
set(a3,'FaceColor','green')
hold on
a4=area(axs,1:N,y4);
set(a4,'FaceColor','red')
set(axs,'XTickLabel','')
set(axs,'YTickLabel','')
axis tight % this is to beautify it.
end
As far as I know, you can still save the axs variable in an axescontainer and then modify the properties you want (like location).
I found out how to do what I needed.
len = 8;
axesContainer = cell(len,1);
x = [0.1,0.4,0.7,0.1,0.4,0.7,0.1,0.4];
y = [0.1,0.1,0.1,0.4,0.4,0.4,0.7,0.7];
figure(1)
for i=1:len
axesContainer{i} = axes('Position',[x(i),y(i),0.2,0.2]);
end
dataContainer = cell(len,1);
N = 1500;
for i=1:len
dataContainer{i} = rand(1,N)*100;
end
for l=1:len
y1=[(dataContainer{l}(1:N/4)) zeros(1,3*N/4)];
y2=[zeros(1,N/4) (dataContainer{l}(N/4+1:(2*N/4))) zeros(1,2*N/4)];
y3=[zeros(1,2*N/4) (dataContainer{l}(2*N/4+1:3*N/4)) zeros(1,N/4)];
y4=[zeros(1,3*N/4) (dataContainer{l}(3*N/4+1:N))];
axes=axesContainer{l};
Y=[y1',y2',y3',y4'];
a=area(axes,Y);
set(axes,'XTickLabel','')
set(axes,'YTickLabel','')
end
The area is supposed to work with matrices like this. The tricky part is, that the signal in every next column is not plotted absolutely, but relatively to the data in previous column. That means, if at time 1 the data in first column has value 1 and data in second column has value 4, the second column data is ploted at value 5. Source: http://www.mathworks.com/help/matlab/ref/area.html

Rolling window for averaging using MATLAB

I have the following code, pasted below. I would like to change it to only average the 10 most recently filtered images and not the entire group of filtered images. The line I think I need to change is: Yout(k,p,q) = (Yout(k,p,q) + (y.^2))/2;, but how do I do it?
j=1;
K = 1:3600;
window = zeros(1,10);
Yout = zeros(10,column,row);
figure;
y = 0; %# Preallocate memory for output
%Load one image
for i = 1:length(K)
disp(i)
str = int2str(i);
str1 = strcat(str,'.mat');
load(str1);
D{i}(:,:) = A(:,:);
%Go through the columns and rows
for p = 1:column
for q = 1:row
if(mean2(D{i}(p,q))==0)
x = 0;
else
if(i == 1)
meanvalue = mean2(D{i}(p,q));
end
%Calculate the temporal mean value based on previous ones.
meanvalue = (meanvalue+D{i}(p,q))/2;
x = double(D{i}(p,q)/meanvalue);
end
%Filtering for 10 bands, based on the previous state
for k = 1:10
[y, ZState{k}] = filter(bCoeff{k},aCoeff{k},x,ZState{k});
Yout(k,p,q) = (Yout(k,p,q) + (y.^2))/2;
end
end
end
% for k = 2:10
% subplot(5,2,k)
% subimage(Yout(k)*5000, [0 100]);
% colormap jet
% end
% pause(0.01);
end
disp('Done Loading...')
The best way to do this (in my opinion) would be to use a circular-buffer to store your images. In a circular-, or ring-buffer, the oldest data element in the array is overwritten by the newest element pushed in to the array. The basics of making such a structure are described in the short Mathworks video Implementing a simple circular buffer.
For each iteration of you main loop that deals with a single image, just load a new image into the circular-buffer and then use MATLAB's built in mean function to take the average efficiently.
If you need to apply a window function to the data, then make a temporary copy of the frames multiplied by the window function and take the average of the copy at each iteration of the loop.
The line
Yout(k,p,q) = (Yout(k,p,q) + (y.^2))/2;
calculates a kind of Moving Average for each of the 10 bands over all your images.
This line calculates a moving average of meanvalue over your images:
meanvalue=(meanvalue+D{i}(p,q))/2;
For both you will want to add a buffer structure that keeps only the last 10 images.
To simplify it, you can also just keep all in memory. Here is an example for Yout:
Change this line: (Add one dimension)
Yout = zeros(3600,10,column,row);
And change this:
for q = 1:row
[...]
%filtering for 10 bands, based on the previous state
for k = 1:10
[y, ZState{k}] = filter(bCoeff{k},aCoeff{k},x,ZState{k});
Yout(i,k,p,q) = y.^2;
end
YoutAvg = zeros(10,column,row);
start = max(0, i-10+1);
for avgImg = start:i
YoutAvg(k,p,q) = (YoutAvg(k,p,q) + Yout(avgImg,k,p,q))/2;
end
end
Then to display use
subimage(Yout(k)*5000, [0 100]);
You would do sth. similar for meanvalue

Kmean plotting in matlab

I am on a project thumb recognition system on matlab. I implemented Kmean Algorithm and I got results as well. Actually now I want to plot the results like here they done. I am trying but couldn't be able to do so. I am using the following code.
load training.mat; % loaded just to get trainingData variable
labelData = zeros(200,1);
labelData(1:100,:) = 0;
labelData(101:200,:) = 1;
k=2;
[trainCtr, traina] = kmeans(trainingData,k);
trainingResult1=[];
for i=1:k
trainingResult1 = [trainingResult1 sum(trainCtr(1:100)==i)];
end
trainingResult2=[];
for i=1:k
trainingResult2 = [trainingResult2 sum(trainCtr(101:200)==i)];
end
load testing.mat; % loaded just to get testingData variable
c1 = zeros(k,1054);
c1 = traina;
cluster = zeros(200,1);
for j=1:200
testTemp = repmat(testingData(j,1:1054),k,1);
difference = sum((c1 - testTemp).^2, 2);
[value index] = min(difference);
cluster(j,1) = index;
end
testingResult1 = [];
for i=1:k
testingResult1 = [testingResult1 sum(cluster(1:100)==i)];
end
testingResult2 = [];
for i=1:k
testingResult2 = [testingResult2 sum(cluster(101:200)==i)];
end
in above code trainingData is matrix of 200 X 1054 in which 200 are images of thumbs and 1054 are columns. actually each image is of 25 X 42. I reshaped each image in to row matrix (1 X 1050) and 4 other (some features) columns so total of 1054 columns are in each image. Similarly testingData I made it in the similar manner as I made testingData It is also the order of 200 X 1054. Now my Problem is just to plot the results as they did in here.
After selecting 2 features, you can just follow the example. Start a figure, use hold on, and use plot or scatter to plot the centroids and the data points. E.g.
selectedFeatures = [42,43];
plot(trainingData(trainCtr==1,selectedFeatures(1)),
trainingData(trainCtr==1,selectedFeatures(2)),
'r.','MarkerSize',12)
Would plot the selected feature values of the data points in cluster 1.