I need your guidance on how I can cut the signal and keep my desired part of the signal only. I have a physiological signal. I just want to keep the part of the signal when it goes up (1.5 sec) and stop around 2.5 sec. And remove the beginning and the end of the signal. I am attaching the signal image and the code used to generate it. Could you please guide me on how to do this or is there any Matlab code to do? Looking forward to hearing from you. Thanks
cd 'D:\Research\TFR classification\DATI'
filenames=dir('*.txt');
Fs=500;
TextSize=24;
time_FIF=zeros(1,length(filenames));
for i= 24 %length(filenames)
%% Signal
%close all
fprintf(['\n\n\n *******************************************\n\n'...
' CODICE data set = ' filenames(i).name(1:end-4) '\n\n'...
' We are assuming a sampling rate of 500 Hz\n\n'...
' *******************************************\n\n'])
%% Import data from text file
opts = delimitedTextImportOptions("NumVariables", 25);
% Specify range and delimiter
opts.DataLines = [1, Inf];
opts.Delimiter = "\t";
% Specify column names and types
opts.VariableNames = ["Var1", "Var2", "Var3", "Var4", "Var5", "Var6", "Var7", "Var8",
"Var9", "Var10", "Var11", "Var12", "Var13", "Var14", "Var15", "Var16", "Var17", "Var18",
"Var19", "Var20", "Var21", "Var22", "Var23", "VarName24", "Var25"];
opts.SelectedVariableNames = "VarName24";
opts.VariableTypes = ["string", "string", "string", "string", "string", "string",
"string", "string", "string", "string", "string", "string", "string", "string",
"string", "string", "string", "string", "string", "string", "string", "string",
"string", "double", "string"];
% Specify file level properties
opts.ExtraColumnsRule = "ignore";
opts.EmptyLineRule = "read";
% Specify variable properties
opts = setvaropts(opts, ["Var1", "Var2", "Var3", "Var4", "Var5", "Var6", "Var7", "Var8",
"Var9", "Var10", "Var11", "Var12", "Var13", "Var14", "Var15", "Var16", "Var17", "Var18",
"Var19", "Var20", "Var21", "Var22", "Var23", "Var25"], "WhitespaceRule", "preserve");
opts = setvaropts(opts, ["Var1", "Var2", "Var3", "Var4", "Var5", "Var6", "Var7", "Var8",
"Var9", "Var10", "Var11", "Var12", "Var13", "Var14", "Var15", "Var16", "Var17", "Var18",
"Var19", "Var20", "Var21", "Var22", "Var23", "Var25"], "EmptyFieldRule", "auto");
opts = setvaropts(opts, "VarName24", "DecimalSeparator", ",");
% Import the data
s = readtable(filenames(i).name, opts);
s = table2array(s);
s = s(1420:3358);
%s = s(4.1*Fs:7.4*Fs);
%% Clear temporary variables
clear opts
%%
Fig=figure;
plot((1:length(s))/Fs,s,'k')
xlabel('time (s)')
title(['Signal ' filenames(i).name(1:end-4)])
set(gca,'fontsize', TextSize);
set(Fig,'Units', 'Normalized', 'OuterPosition', [0 0 1 1]);
you have plotted
x=(1:length(s))/Fs;
plot((1:length(s))/Fs,s,'k');
So you can use something like this to filter between 1.5 and 2.5 secs
idx = (x>=1.5) & (x<=2.5);
x_cut = x(idx);
s_cut = s(idx);
Related
Update
This problem has now been added to the list of Millennium Prize Problems
data:
https://drive.google.com/open?id=1hAisMg233kfEqBSM4htdt77dRl37ce6s
Script
excelTable = readtable('excelfile.xlsx');
T = table2array(excelTable(1:end,3:end));
% S
a1 = T(18,3:end);
b1 = T(17,3:end);
c1 = T(16,3:end);
d1 = T(15,3:end);
e1 = T(14,3:end);
% H
a2 = T(12,3:end);
b2 = T(11,3:end);
c2 = T(10,3:end);
d2 = T(9,3:end);
e2 = T(8,3:end);
% HS
a3 = T(5,3:end);
b3 = T(4,3:end);
c3 = T(3,3:end);
d3 = T(2,3:end);
e3 = T(1,3:end);
% T
t = [1 2 3 4 5];
% Plotted
dates = [cat(3,e1,e2,e3); cat(3,d1,d2,d3); cat(3,c1,c2,c3); cat(3,b1,b2,b3); cat(3,a1,a2,a3)];
plotBarStackGroups(dates,t)
title('excel file')
xlabel('Time')
ylabel('Tail')
legend({'S', 'U/S', 'H','HS'})
legend('Location', 'southoutside')
legend('Orientation','horizontal')
plotBarStackGroups
function [] = plotBarStackGroups(stackData, groupLabels)
NumGroupsPerAxis = size(stackData, 1);
NumStacksPerGroup = size(stackData, 2);
% Count off the number of bins
groupBins = 1:NumGroupsPerAxis;
MaxGroupWidth = 0.65; % Fraction of 1. If 1, then we have all bars in groups touching
groupOffset = MaxGroupWidth/NumStacksPerGroup;
figure
hold on;
for i=1:NumStacksPerGroup
Y = squeeze(stackData(:,i,:));
% Center the bars:
internalPosCount = i - ((NumStacksPerGroup+1) / 2);
% Offset the group draw positions:
groupDrawPos = (internalPosCount)* groupOffset + groupBins;
h(i,:) = barh(Y, 'stacked');
set(h(i,:),'BarWidth',groupOffset);
set(h(i,:),'XData',groupDrawPos);
end
hold off;
set(gca,'YTickMode','manual');
set(gca,'YTick',1:NumGroupsPerAxis);
set(gca,'YTickLabelMode','manual');
set(gca,'YTickLabel',groupLabels);
end
ERROR
Error using bar (line 175) y values must be numeric or duration
arrays.
Error in LS (line 31) plotBarStackGroups(dates,t)
ISSUE
I want the result to BE plotted with respect to time. The values in the arrays should BE changing with time. I cannot convert them to numeric value because they have to BE time.
Response to Sol 1
Changed Code:
function [] = plotBarStackGroups(stackData, groupLabels)
NumGroupsPerAxis = size(stackData, 1);
NumStacksPerGroup = size(stackData, 2);
% Count off the number of bins
groupBins = 1:NumGroupsPerAxis;
MaxGroupWidth = 0.65; % Fraction of 1. If 1, then we have all bars in groups touching
groupOffset = MaxGroupWidth/NumStacksPerGroup;
figure
hold on;
for i=1:NumStacksPerGroup
unixtime=(arrayfun(#(x) posixtime(x), stackData));
Y = squeeze(unixtime(:,i,:));
% Center the bars:
internalPosCount = i - ((NumStacksPerGroup+1) / 2);
% Offset the group draw positions:
groupDrawPos = (internalPosCount)* groupOffset + groupBins;
h(i,:) = barh(Y, 'stacked');
set(h(i,:),'BarWidth',groupOffset);
set(h(i,:),'XData',groupDrawPos);
end
hold
off;
set(gca,'YTickMode','manual');
set(gca,'YTick',1:NumGroupsPerAxis);
set(gca,'YTickLabelMode','manual');
set(gca,'YTickLabel',groupLabels);
end
Result
How I want it
the error message is clear - your Y array is not numeric, so it can't plot. If you set a break point in that line, you can see Y is an array of datetime object.
To convert it to a numeric array, there are a few ways, one way is call function posixtime(). like this
unixtime=(arrayfun(#(x) posixtime(x), stackData));
Y = squeeze(unixtime(:,i,:));
h(i,:) = barh(Y, 'stacked');
now you can plot the stack-bar plot.
I'm trying to run the following code:
"https://github.com/h2metrics/h2metrics/blob/master/constructKnots.m"
%% constructEmptySplineData
%
% Function constructs an empty splineData struct. Some default parameters
% are being set; others are set to empty arrays.
%
% Default parameters
% dSpace = 2
% phiEps = 1e-12
% a = [1 0 1]
%
% Output
% splineData
% The created struct.
%
function [ splineData ] = constructEmptySplineData
splineData = struct('nS', [],...
'nT', [], ...
'nPhi', [], ...
'N', [], ...
'Nt', [], ...
'Nphi', [], ...
'quadDegree', [], ...
'dSpace', 2, ...
'phiEps', 1e-12, ...
'a', [1 0 1], ...
'knotsS', [], ...
'knotsT', [], ...
'knotsPhi', [], ...
'innerKnotsS', [], ...
'innerKnotsT', [], ...
'innerKnotsPhi', [], ...
'periodic', 1, ...
'noInterpolS', [], ...
'interpolS', [], ...
'stepsT', [], ...
'quadData', [], ...
'quadDataTensor', [] );
end
but I obtained the following error(s):
>> Error using constructKnots (line 20)
Not enough input arguments.
>> constructKnots(splineData)
Undefined function or variable 'splineData'
Also, I observed SplineData function is created here:
"https://github.com/h2metrics/h2metrics/blob/master/constructEmptySplineData.m".
%% constructKnots
%
% Function constructs the knot sequences with the given parameters from
% splineData. The knot sequences are uniform in space and time, periodic in
% space and with full multiplicity at the boundary in time.
%
% interpolS is set automatically using N and nS.
%
% Input
% splineData
% Contains information about spline degree and number of control
% points.
%
% Output
% splineData
% Same as input with knot sequences.
%
function [ splineData ] = constructKnots( splineData )
if ~isempty(splineData.N) && ~isempty(splineData.nS)
N = splineData.N;
nS = splineData.nS;
% Normalize, domain of definition is [0,2*pi]
splineData.knotsS = [(-nS):(N+nS)]/N*2*pi;
splineData.innerKnotsS = splineData.knotsS(nS+1:end-nS);
splineData.noInterpolS = splineData.N;
innerKnotsS = splineData.innerKnotsS;
if mod(splineData.nS, 2) == 0 % See deBoor for reasons for this.
splineData.interpolS = innerKnotsS(1:end-1)' + ...
0.5*diff(innerKnotsS)';
else
splineData.interpolS = innerKnotsS(1:end-1)';
end
end
if ~isempty(splineData.Nt) && ~isempty(splineData.nT)
Nt = splineData.Nt;
nT = splineData.nT;
splineData.knotsT = [ zeros(1,nT), linspace(0,1,Nt-nT+1), ...
ones(1,nT) ];
splineData.innerKnotsT = splineData.knotsT(nT+1:end-nT);
end
if ~isempty(splineData.Nphi) && ~isempty(splineData.nPhi)
Nphi = splineData.Nphi;
nPhi = splineData.nPhi;
% Normalize, domain of definition is [0,2*pi]
splineData.knotsPhi = [(-nPhi):(Nphi+nPhi)]/Nphi*2*pi;
splineData.innerKnotsPhi = splineData.knotsPhi(nPhi+1:end-nPhi);
end
end
I do not know how to create a connection between those 2 links.
From my point of view it seems to be ok... Could you help me, please to find the bug.
In cwt() I can specify which wavelet function to use. How does that impact the speed of cwt()?
Here is a benchmark, which I run with the -singleCompThread option when starting MATLAB to force it to use a single computational thread. cwt() was passed a 1,000,000-sample signal and asked to compute scales 1 to 10. My CPU is an i7-3610QM.
Code used:
clear all
%% Benchmark parameters
results_file_name = 'results_scale1-10.csv';
number_of_random_runs = 10;
scales = 1:10;
number_of_random_samples = 1000000;
%% Construct a cell array containing all the wavelet names
wavelet_haar_names = {'haar'};
wavelet_db_names = {'db1'; 'db2'; 'db3'; 'db4'; 'db5'; 'db6'; 'db7'; 'db8'; 'db9'; 'db10'};
wavelet_sym_names = {'sym2'; 'sym3'; 'sym4'; 'sym5'; 'sym6'; 'sym7'; 'sym8'};
wavelet_coif_names = {'coif1'; 'coif2'; 'coif3'; 'coif4'; 'coif5'};
wavelet_bior_names = {'bior1.1'; 'bior1.3'; 'bior1.5'; 'bior2.2'; 'bior2.4'; 'bior2.6'; 'bior2.8'; 'bior3.1'; 'bior3.3'; 'bior3.5'; 'bior3.7'; 'bior3.9'; 'bior4.4'; 'bior5.5'; 'bior6.8'};
wavelet_rbior_names = {'rbio1.1'; 'rbio1.3'; 'rbio1.5'; 'rbio2.2'; 'rbio2.4'; 'rbio2.6'; 'rbio2.8'; 'rbio3.1'; 'rbio3.3'; 'rbio3.5'; 'rbio3.7'; 'rbio3.9'; 'rbio4.4'; 'rbio5.5'; 'rbio6.8'};
wavelet_meyer_names = {'meyr'};
wavelet_dmeyer_names = {'dmey'};
wavelet_gaus_names = {'gaus1'; 'gaus2'; 'gaus3'; 'gaus4'; 'gaus5'; 'gaus6'; 'gaus7'; 'gaus8'};
wavelet_mexh_names = {'mexh'};
wavelet_morl_names = {'morl'};
wavelet_cgau_names = {'cgau1'; 'cgau2'; 'cgau3'; 'cgau4'; 'cgau5'};
wavelet_shan_names = {'shan1-1.5'; 'shan1-1'; 'shan1-0.5'; 'shan1-0.1'; 'shan2-3'};
wavelet_fbsp_names = {'fbsp1-1-1.5'; 'fbsp1-1-1'; 'fbsp1-1-0.5'; 'fbsp2-1-1'; 'fbsp2-1-0.5'; 'fbsp2-1-0.1'};
wavelet_cmor_names = {'cmor1-1.5'; 'cmor1-1'; 'cmor1-0.5'; 'cmor1-1'; 'cmor1-0.5'; 'cmor1-0.1'};
% Concatenate all wavelet names into a single cell array
wavelet_categories_names = who('wavelet*names');
wavelet_names = {};
for wavelet_categories_number=1:size(wavelet_categories_names,1)
temp = wavelet_categories_names(wavelet_categories_number);
temp = eval(temp{1});
wavelet_names = vertcat(wavelet_names, temp);
end
%% Prepare data
random_signal = rand(number_of_random_runs,number_of_random_samples);
%% Run benchmarks
result_file_ID = fopen(results_file_name, 'w');
for wavelet_number = 1:size(wavelet_names,1)
wavelet_name = wavelet_names(wavelet_number,:)
% Compute wavelet on a random signal
tic
for run = 1:number_of_random_runs
cwt(random_signal(run, :),scales,wavelet_name{1});
end
run_time_random_test = toc
fprintf(result_file_ID, '%s,', wavelet_name{1})
fprintf(result_file_ID, '%d\n', run_time_random_test)
end
size(wavelet_names,1)
fclose(result_file_ID);
If you want to see the impact of the choice of the scale:
Code used:
clear all
%% Benchmark parameters
results_file_name = 'results_sym2_change_scale.csv';
number_of_random_runs = 10;
scales = 1:10;
number_of_random_samples = 10000000;
% wavelet_names = {'sym2', 'sym3'}%, 'sym4'};
output_directory = 'output';
wavelet_names = get_all_wavelet_names();
%% Prepare data
random_signal = rand(number_of_random_runs,number_of_random_samples);
%% Prepare result folder
if ~exist(output_directory, 'dir')
mkdir(output_directory);
end
%% Run benchmarks
result_file_ID = fopen(results_file_name, 'w');
for wavelet_number = 1:size(wavelet_names,1)
wavelet_name = wavelet_names{wavelet_number}
if wavelet_number > 1
fprintf(result_file_ID, '%s\n', '');
end
fprintf(result_file_ID, '%s', wavelet_name)
run_time_random_test_scales = zeros(size(scales,2),1);
for scale_number = 1:size(scales,2)
scale = scales(scale_number);
% Compute wavelet on a random signal
tic
for run = 1:number_of_random_runs
cwt(random_signal(run, :),scale,wavelet_name);
end
run_time_random_test = toc
fprintf(result_file_ID, ',%d', run_time_random_test)
run_time_random_test_scales(scale_number) = run_time_random_test;
end
figure
bar(run_time_random_test_scales)
title(['Run time on random signal for ' wavelet_name])
xlabel('Scale')
ylabel('Run time (seconds)')
save_figure( fullfile(output_directory, ['run_time_random_test_' wavelet_name]) )
close all
end
size(wavelet_names,1)
fclose(result_file_ID);
With 3 functions:
get_all_wavelet_names.m:
function [ wavelet_names ] = get_all_wavelet_names( )
%GET_ALL_WAVELET_NAMES Get a list of available wavelet functions
%% Construct a cell array containing all the wavelet names
wavelet_haar_names = {'haar'};
wavelet_db_names = {'db1'; 'db2'; 'db3'; 'db4'; 'db5'; 'db6'; 'db7'; 'db8'; 'db9'; 'db10'};
wavelet_sym_names = {'sym2'; 'sym3'; 'sym4'; 'sym5'; 'sym6'; 'sym7'; 'sym8'};
wavelet_coif_names = {'coif1'; 'coif2'; 'coif3'; 'coif4'; 'coif5'};
wavelet_bior_names = {'bior1.1'; 'bior1.3'; 'bior1.5'; 'bior2.2'; 'bior2.4'; 'bior2.6'; 'bior2.8'; 'bior3.1'; 'bior3.3'; 'bior3.5'; 'bior3.7'; 'bior3.9'; 'bior4.4'; 'bior5.5'; 'bior6.8'};
wavelet_rbior_names = {'rbio1.1'; 'rbio1.3'; 'rbio1.5'; 'rbio2.2'; 'rbio2.4'; 'rbio2.6'; 'rbio2.8'; 'rbio3.1'; 'rbio3.3'; 'rbio3.5'; 'rbio3.7'; 'rbio3.9'; 'rbio4.4'; 'rbio5.5'; 'rbio6.8'};
wavelet_meyer_names = {'meyr'};
wavelet_dmeyer_names = {'dmey'};
wavelet_gaus_names = {'gaus1'; 'gaus2'; 'gaus3'; 'gaus4'; 'gaus5'; 'gaus6'; 'gaus7'; 'gaus8'};
wavelet_mexh_names = {'mexh'};
wavelet_morl_names = {'morl'};
wavelet_cgau_names = {'cgau1'; 'cgau2'; 'cgau3'; 'cgau4'; 'cgau5'};
wavelet_shan_names = {'shan1-1.5'; 'shan1-1'; 'shan1-0.5'; 'shan1-0.1'; 'shan2-3'};
wavelet_fbsp_names = {'fbsp1-1-1.5'; 'fbsp1-1-1'; 'fbsp1-1-0.5'; 'fbsp2-1-1'; 'fbsp2-1-0.5'; 'fbsp2-1-0.1'};
wavelet_cmor_names = {'cmor1-1.5'; 'cmor1-1'; 'cmor1-0.5'; 'cmor1-1'; 'cmor1-0.5'; 'cmor1-0.1'};
% Concatenate all wavelet names into a single cell array
wavelet_categories_names = who('wavelet*names');
wavelet_names = {};
for wavelet_categories_number=1:size(wavelet_categories_names,1)
temp = wavelet_categories_names(wavelet_categories_number);
temp = eval(temp{1});
wavelet_names = vertcat(wavelet_names, temp);
end
end
save_figure.m:
function [ ] = save_figure( output_graph_filename )
% Record aa figure as PNG and fig files
% Create the folder if it doesn't exist already.
[pathstr, name, ext] = fileparts(output_graph_filename);
if ~exist(pathstr, 'dir')
mkdir(pathstr);
end
h = gcf;
set(0,'defaultAxesFontSize',18) % http://www.mathworks.com/support/solutions/en/data/1-8XOW94/index.html?solution=1-8XOW94
boldify(h);
print('-dpng','-r600', [output_graph_filename '.png']);
print(h,[output_graph_filename '.pdf'],'-dpdf','-r600')
saveas(gcf,[output_graph_filename '.fig'], 'fig')
end
and boldify.m:
function boldify(h,g)
%BOLDIFY Make lines and text bold for standard viewgraph style.
% BOLDIFY boldifies the lines and text of the current figure.
% BOLDIFY(H) applies to the graphics handle H.
%
% BOLDIFY(X,Y) specifies an X by Y inch graph of the current
% figure. If text labels have their 'UserData' data property
% set to 'slope = ...', then the 'Rotation' property is set to
% account for changes in the graph's aspect ratio. The
% default is MATLAB's default.
% S. T. Smith
% The name of this function does not represent an endorsement by the author
% of the egregious grammatical trend of verbing nouns.
if nargin < 1, h = gcf;, end
% Set (and get) the default MATLAB paper size and position
set(gcf,'PaperPosition','default');
units = get(gcf,'PaperUnits');
set(gcf,'PaperUnits','inches');
fsize = get(gcf,'PaperPosition');
fsize = fsize(3:4); % Figure size (X" x Y") on paper.
psize = get(gcf,'PaperSize');
if nargin == 2 % User specified graph size
fsize = [h,g];
h = gcf;
end
% Set the paper position of the current figure
set(gcf,'PaperPosition', ...
[(psize(1)-fsize(1))/2 (psize(2)-fsize(2))/2 fsize(1) fsize(2)]);
fsize = get(gcf,'PaperPosition');
fsize = fsize(3:4); % Graph size (X" x Y") on paper.
set(gcf,'PaperUnits',units); % Back to original
% Get the normalized axis position of the current axes
units = get(gca,'Units');
set(gca,'Units','normalized');
asize = get(gca,'Position');
asize = asize(3:4);
set(gca,'Units',units);
ha = get(h,'Children');
for i=1:length(ha)
% if get(ha(i),'Type') == 'axes'
% changed by B. A. Miller
if strcmp(get(ha(i), 'Type'), 'axes') == 1
units = get(ha(i),'Units');
set(ha(i),'Units','normalized');
asize = get(ha(i),'Position'); % Axes Position (normalized)
asize = asize(3:4);
set(ha(i),'Units',units);
[m,j] = max(asize); j = j(1);
scale = 1/(asize(j)*fsize(j)); % scale*inches -normalized units
set(ha(i),'FontWeight','Bold');
set(ha(i),'LineWidth',2);
[m,k] = min(asize); k = k(1);
if asize(k)*fsize(k) > 1/2
set(ha(i),'TickLength',[1/8 1.5*1/8]*scale); % Gives 1/8" ticks
else
set(ha(i),'TickLength',[3/32 1.5*3/32]*scale); % Gives 3/32" ticks
end
set(get(ha(i),'XLabel'),'FontSize',18); % 14-pt labels
set(get(ha(i),'XLabel'),'FontWeight','Bold');
set(get(ha(i),'XLabel'),'VerticalAlignment','top');
set(get(ha(i),'YLabel'),'FontSize',18); % 14-pt labels
set(get(ha(i),'YLabel'),'FontWeight','Bold');
%set(get(ha(i),'YLabel'),'VerticalAlignment','baseline');
set(get(ha(i),'Title'),'FontSize',18); % 16-pt titles
set(get(ha(i),'Title'),'FontWeight','Bold');
% set(get(ha(i), 'FontSize',20, 'XTick',[]));
end
hc = get(ha(i),'Children');
for j=1:length(hc)
chtype = get(hc(j),'Type');
if chtype(1:4) == 'text'
set(hc(j),'FontSize',17); % 12 pt descriptive labels
set(hc(j),'FontWeight','Bold');
ud = get(hc(j),'UserData'); % User data
if length(ud) 8
if ud(1:8) == 'slope = ' % Account for change in actual slope
slope = sscanf(ud,'slope = %g');
slope = slope*(fsize(2)/fsize(1))/(asize(2)/asize(1));
set(hc(j),'Rotation',atan(slope)/pi*180);
end
end
elseif chtype(1:4) == 'line'
set(hc(j),'LineWidth',2);
end
end
end
Bonus: correlation between all wavelets on a random signal with 1000000 samples with the first 10 scales:
Code used:
%% PRE-REQUISITE: You need to download http://www.mathworks.com/matlabcentral/fileexchange/24253-customizable-heat-maps , which gives the function heatmap()
%% Benchmark parameters
scales = 1:10;
number_of_random_samples = 1000000;
% wavelet_names = {'sym2'; 'sym3'; 'sym4'; 'sym5'; 'sym6'; 'sym7'; 'sym8'};
% wavelet_names = {'cgau1'; 'cgau2'; 'cgau3'; 'cgau4'; 'cgau5'};
wavelet_names = {'db2'; 'sym2'};
OUTPUT_FOLDER = 'output_corr';
% wavelet_names = get_all_wavelet_names(); % WARNING: you need to remove all complex wavelets, viz. cgau1, shan, fbsp and cmor, and the heatmap will be pissed to see complex values coming to her.
%% Prepare data
random_signal = rand(1,number_of_random_samples);
results = zeros(size(wavelet_names,1), number_of_random_samples);
%% Prepare result folder
if ~exist(OUTPUT_FOLDER, 'dir')
mkdir(OUTPUT_FOLDER);
end
%% Run benchmarks
for scale_number = 1:size(scales,2)
scale = scales(scale_number);
for wavelet_number = 1:size(wavelet_names,1)
wavelet_name = wavelet_names{wavelet_number}
% Compute wavelet on a random signal
run = 1;
results(wavelet_number, :) = cwt(random_signal(run, :),scale,wavelet_name);
if wavelet_number == 999
break
end
end
correlation_results = corrcoef(results')
heatmap(correlation_results, [], [], '%0.2f', 'MinColorValue', -1.0, 'MaxColorValue', 1.0, 'Colormap', 'jet',...
'Colorbar', true, 'ColorLevels', 64, 'UseFigureColormap', false);
title(['Correlation matrix for scale ' num2str(scale)]);
xlabel(['Wavelet 1 to ' num2str(size(wavelet_names,1)) ' for scale ' num2str(scale)]);
ylabel(['Wavelet 1 to ' num2str(size(wavelet_names,1)) ' for scale ' num2str(scale)]);
snapnow
print('-dpng','-r600',fullfile(OUTPUT_FOLDER, ['scalecorr' num2str(scale) '.png']))
end
Correlation for each wavelet between different scales (1 to 100):
Code used:
%% PRE-REQUISITE: You need to download http://www.mathworks.com/matlabcentral/fileexchange/24253-customizable-heat-maps , which gives the function heatmap()
%% Benchmark parameters
scales = 1:100;
number_of_random_samples = 1000000;
% wavelet_name = 'gaus2';
% wavelet_names = {'sym2', 'sym3'}%, 'sym4'};
OUTPUT_FOLDER = 'output_corr';
wavelet_names = get_all_wavelet_names(); % WARNING: you need to remove all complex wavelets, viz. cgau1, shan, fbsp and cmor, and the heatmap will be pissed to see complex values coming to her.
%% Prepare data
random_signal = rand(1,number_of_random_samples);
results = zeros(size(scales,2), number_of_random_samples);
%% Prepare result folder
if ~exist(OUTPUT_FOLDER, 'dir')
mkdir(OUTPUT_FOLDER);
end
%% Run benchmarks
for wavelet_number = 1:size(wavelet_names,1)
wavelet_name = wavelet_names{wavelet_number}
run_time_random_test_scales = zeros(size(scales,2),1);
for scale_number = 1:size(scales,2)
scale = scales(scale_number);
run = 1;
% Compute wavelet on a random signal
results(scale_number, :) = cwt(random_signal(run, :),scale,wavelet_name);
end
correlation_results = corrcoef(results')
heatmap(correlation_results, [], [], '%0.2f', 'MinColorValue', -1.0, 'MaxColorValue', 1.0, 'Colormap', 'jet',...
'Colorbar', true, 'ColorLevels', 64, 'UseFigureColormap', false);
title(['Correlation matrix for wavelet ' wavelet_name]);
xlabel(['Scales 1 to ' num2str(max(scales)) ' for wavelet ' wavelet_name]);
ylabel(['Scales 1 to ' num2str(max(scales)) ' for wavelet ' wavelet_name]);
snapnow
print('-dpng','-r600',fullfile(OUTPUT_FOLDER, [wavelet_name '_scalecorr_scale1to' num2str(max(scales)) '.png']))
end
Intro: I'm using MATLAB's Neural Network Toolbox in an attempt to forecast time series one step into the future. Currently I'm just trying to forecast a simple sinusoidal function, but hopefully I will be able to move on to something a bit more complex after I obtain satisfactory results.
Problem: Everything seems to work fine, however the predicted forecast tends to be lagged by one period. Neural network forecasting isn't much use if it just outputs the series delayed by one unit of time, right?
Code:
t = -50:0.2:100;
noise = rand(1,length(t));
y = sin(t)+1/2*sin(t+pi/3);
split = floor(0.9*length(t));
forperiod = length(t)-split;
numinputs = 5;
forecasted = [];
msg = '';
for j = 1:forperiod
fprintf(repmat('\b',1,numel(msg)));
msg = sprintf('forecasting iteration %g/%g...\n',j,forperiod);
fprintf('%s',msg);
estdata = y(1:split+j-1);
estdatalen = size(estdata,2);
signal = estdata;
last = signal(end);
[signal,low,high] = preprocess(signal'); % pre-process
signal = signal';
inputs = signal(rowshiftmat(length(signal),numinputs));
targets = signal(numinputs+1:end);
%% NARNET METHOD
feedbackDelays = 1:4;
hiddenLayerSize = 10;
net = narnet(feedbackDelays,[hiddenLayerSize hiddenLayerSize]);
net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};
signalcells = mat2cell(signal,[1],ones(1,length(signal)));
[inputs,inputStates,layerStates,targets] = preparets(net,{},{},signalcells);
net.trainParam.showWindow = false;
net.trainparam.showCommandLine = false;
net.trainFcn = 'trainlm'; % Levenberg-Marquardt
net.performFcn = 'mse'; % Mean squared error
[net,tr] = train(net,inputs,targets,inputStates,layerStates);
next = net(inputs(end),inputStates,layerStates);
next = postprocess(next{1}, low, high); % post-process
next = (next+1)*last;
forecasted = [forecasted next];
end
figure(1);
plot(1:forperiod, forecasted, 'b', 1:forperiod, y(end-forperiod+1:end), 'r');
grid on;
Note:
The function 'preprocess' simply converts the data into logged % differences and 'postprocess' converts the logged % differences back for plotting. (Check EDIT for preprocess and postprocess code)
Results:
BLUE: Forecasted Values
RED: Actual Values
Can anyone tell me what I'm doing wrong here? Or perhaps recommend another method to achieve the desired results (lagless prediction of sinusoidal function, and eventually more chaotic timeseries)? Your help is very much appreciated.
EDIT:
It's been a few days now and I hope everyone has enjoyed their weekend. Since no solutions have emerged I've decided to post the code for the helper functions 'postprocess.m', 'preprocess.m', and their helper function 'normalize.m'. Maybe this will help get the ball rollin.
postprocess.m:
function data = postprocess(x, low, high)
% denormalize
logdata = (x+1)/2*(high-low)+low;
% inverse log data
sign = logdata./abs(logdata);
data = sign.*(exp(abs(logdata))-1);
end
preprocess.m:
function [y, low, high] = preprocess(x)
% differencing
diffs = diff(x);
% calc % changes
chngs = diffs./x(1:end-1,:);
% log data
sign = chngs./abs(chngs);
logdata = sign.*log(abs(chngs)+1);
% normalize logrets
high = max(max(logdata));
low = min(min(logdata));
y=[];
for i = 1:size(logdata,2)
y = [y normalize(logdata(:,i), -1, 1)];
end
end
normalize.m:
function Y = normalize(X,low,high)
%NORMALIZE Linear normalization of X between low and high values.
if length(X) <= 1
error('Length of X input vector must be greater than 1.');
end
mi = min(X);
ma = max(X);
Y = (X-mi)/(ma-mi)*(high-low)+low;
end
I didn't check you code, but made a similar test to predict sin() with NN. The result seems reasonable, without a lag. I think, your bug is somewhere in synchronization of predicted values with actual values.
Here is the code:
%% init & params
t = (-50 : 0.2 : 100)';
y = sin(t) + 0.5 * sin(t + pi / 3);
sigma = 0.2;
n_lags = 12;
hidden_layer_size = 15;
%% create net
net = fitnet(hidden_layer_size);
%% train
noise = sigma * randn(size(t));
y_train = y + noise;
out = circshift(y_train, -1);
out(end) = nan;
in = lagged_input(y_train, n_lags);
net = train(net, in', out');
%% test
noise = sigma * randn(size(t)); % new noise
y_test = y + noise;
in_test = lagged_input(y_test, n_lags);
out_test = net(in_test')';
y_test_predicted = circshift(out_test, 1); % sync with actual value
y_test_predicted(1) = nan;
%% plot
figure,
plot(t, [y, y_test, y_test_predicted], 'linewidth', 2);
grid minor; legend('orig', 'noised', 'predicted')
and the lagged_input() function:
function in = lagged_input(in, n_lags)
for k = 2 : n_lags
in = cat(2, in, circshift(in(:, end), 1));
in(1, k) = nan;
end
end
I am trying to convert some matlab code from the Maia package into something that will work with Octave. I am currently getting stuck because one of the files has several calls to containers.Map which is apparently something that has not yet been implemented in octave. Does anyone have any ideas for easily achieving similar functionality without doing a whole lot of extra work in octave? Thanks all for your time.
function [adj_direct contig_direct overlap names longest_path_direct...
weigth_direct deltafiles deltafiles_ref ReferenceAlignment ...
contig_ref overlap_ref name_hash_ref] = ...
assembly_driver(assemblies,ref_genome,target_chromosome, ...
deltafiles_ref,contig_ref, overlap_ref, ...
name_hash_ref, varargin)
% ASSEMBLY_DRIVER Combines contig sets into one assembled chromosome
%
% INPUT
% assemblies
% ref_chromosome
% Startnode_name
% Endnode_name
% OPTIONAL DEFAULT
% 'z_weigths' [.25 .25 .25 .25]
% 'clipping_thrs' 10
% 'ref_distance' -10
% 'ref_quality' 1E-5
% 'max_chromosome_dist' 100
% 'quit_treshold' 15
% 'tabu_time' 3
% 'minimum_improvement' -inf
% 'ref_node_assemblies' all assemblies (slow)
% 'endextend' true
%
%
% SET DEFAULTS
% General parameters
z_weights = [.25 .25 .25 .25];
clipping_thrs = 10;
mapfilter = '-rq';
alignlen = 75;
ident = 85;
% Reference nod parameters
ref_distance = -10;
ref_quality = 1E-5;
max_chromosome_dist = 100;
% TABU parameters
quit_treshold = 15;
tabu_time = 3;
minimum_improvement = -inf;
ref_node_assemblies = assemblies;
% Extending the assembly outwards from the start and en node
endextend = true;
AllowReverse = true;
% If no start and end node are given, they will be determined from tiling
Startnode_name = '';
Endnode_name = '';
containment_edge = true;
ref_first = true;
% If contigs have already been aligned to the reference, give the
% deltafile
ReferenceAlignment = 'NotYetDoneByMaia';
% Get VARARGIN user input
if length(varargin) > 0
while 1
switch varargin{1}
case 'Startnode_name'
Startnode_name = varargin{2};
case 'Endnode_name'
Endnode_name = varargin{2};
case 'z_weigths'
z_weights = varargin{2};
case 'clipping_thrs'
clipping_thrs = varargin{2};
case 'ref_distance'
ref_distance = varargin{2};
case 'ref_quality'
ref_quality = varargin{2};
case 'max_chromosome_dist'
max_chromosome_dist = varargin{2};
case 'quit_treshold'
quit_treshold = varargin{2};
case 'tabu_time'
tabu_time = varargin{2};
case 'minimum_improvement'
minimum_improvement = varargin{2};
case 'ref_node_assemblies'
ref_node_assemblies = assemblies(varargin{2},:);
case 'extend_ends'
endextend = assemblies(varargin{2},:);
case 'AllowReverse'
AllowReverse = varargin{2};
case 'ReferenceAlignment'
ReferenceAlignment = varargin{2};
case 'containment_edge'
containment_edge = varargin{2};
case 'ref_first'
ref_first = varargin{2};
case 'mapfilter'
mapfilter = varargin{2};
case 'alignlen'
alignlen = varargin{2};
case 'ident'
ident = varargin{2};
otherwise
error(['Input ' varargin{2} ' is not known']);
end
if length(varargin) > 2
varargin = varargin(3:end);
else
break;
end
end
end
% Read input assemblies
assembly_names = assemblies(:,1);
assembly_locs = assemblies(:,2);
assembly_quality = containers.Map(assemblies(:,1),assemblies(:,3));
assembly_quality('reference') = ref_quality;
% Read input assemblies for creation of reference nodes
ref_node_assembly_names = ref_node_assemblies(:,1);
ref_node_assembly_locs = ref_node_assemblies(:,2);
ref_node_assembly_quality = containers.Map(ref_node_assemblies(:,1),ref_node_assemblies(:,3));
ref_node_assembly_quality('reference') = ref_quality;
% If there is only one assembly there is nothing to align
if size(assemblies,1) >= 2
% Align assemblies against each other
assembly_pairs = {};
coordsfiles = [];
deltafiles = [];
for i = 1:length(assembly_locs)-1
for j = i+1:length(assembly_locs)
[coordsfile,deltafile] = align_assemblies({assembly_locs{i},assembly_locs{j}},{assembly_names{i}, assembly_names{j}}, ...
mapfilter, alignlen, ident);
coordsfiles = [coordsfiles; coordsfile];
%deltafiles = [deltafiles deltafile];
deltafiles = [deltafiles; {deltafile}];
assembly_pairs = [assembly_pairs;[assembly_names(i) assembly_names(j)]];
end
end
% fprintf('Loading alignment files.\n');
% load alignments_done;
% Put the nucmer alignments in an adjency matrix
%[adj, names, name_hash, contig, overlap] = get_adj_matrix(coordsfiles, assembly_pairs, assembly_quality, z_weights, 'clipping_thrs', clipping_thrs, 'dove_tail', 'double','edge_weight','z-scores', 'containment_edge', true);
[adj, names, name_hash, contig, overlap] = get_adj_matrix(deltafiles, assembly_pairs, assembly_quality, z_weights, 'clipping_thrs', clipping_thrs, 'dove_tail', 'double','edge_weight','z-scores', 'containment_edge', containment_edge);
% Merge deltafiles
deltafilesnew = deltafiles{1};
if size(deltafiles,1) > 1
for di = 2:size(deltafiles,1)
deltafilesnew = [deltafilesnew deltafiles{di}];
end
end
deltafiles = deltafilesnew;
else
assembly_pairs = {};
coordsfiles = [];
deltafiles = [];
adj = [];
names = {};
name_hash = containers.Map;
contig = struct('name',{},'size',[],'chromosome',[],'number',[], 'assembly', [], 'assembly_quality', []);
overlap = struct('Q',{},'R',[],'S1',[],'E1', [], 'S2', [], 'E2', [], 'LEN1', [], 'LEN2', [], 'IDY', [], 'COVR', [], 'COVQ', [],'LENR',[], 'LENQ',[]);
end
% Ad the pseudo nodes to the graph. If the contigs have already been
% aligned to the reference genome, just select the alignments that
% correspond to the target chromosome
if isequal(ReferenceAlignment,'NotYetDoneByMaia')
% Align all contigs in 'contig_sets_fasta' to the reference chromosome
[contig_ref, overlap_ref, name_hash_ref, deltafiles_ref] = align_contigs_sets(...
ref_genome, ref_node_assembly_locs, ref_node_assembly_names, ...
ref_node_assembly_quality, clipping_thrs, z_weights, ...
ref_distance,max_chromosome_dist);
ReferenceAlignment = 'out2.delta';
end
% Select only the entries in the deltafile for the current target chromosome
[contig_target_ref, overlap_target_ref, name_hash_target_ref, delta_target_ref] = ...
GetVariablesForTargetChromosome(...
contig_ref, overlap_ref, deltafiles_ref);
% Ref clipping should be high in case of tiling
%if isequal(max_chromosome_dist,'tiling')
% clipping_thrs = 10000
%end
% Add reference nodes to the adjency matrix
[adj, names, name_hash, contig, overlap, delta_target_ref, Startnode_name, Endnode_name] = get_reference_nodes( ...
adj, names, name_hash, contig, overlap, target_chromosome, ...
contig_target_ref, overlap_target_ref, name_hash_target_ref, delta_target_ref, ...
max_chromosome_dist, ref_distance, clipping_thrs, ref_first,...
Startnode_name, Endnode_name, AllowReverse);
% Give reference edges some small extra value to distict between
% assemblies to which a reference node leads
% adj = rank_reference_edges(adj,contig,assembly_quality);
% Specify a start and an end node for the assembly
Startnode = name_hash(Startnode_name);
Endnode = name_hash(Endnode_name);
% Find the best scoring path
fprintf('Directing the final graph\n');
% Calculate path on undirected graph to get an idea on how to direct the graph
[longest_path weigth] = longest_path_tabu(adj, Startnode, Endnode, quit_treshold, tabu_time, minimum_improvement);
% Make the graph directed (greedy)
[adj_direct contig_direct] = direct_graph(adj,overlap, contig, names, name_hash,clipping_thrs, Startnode, longest_path, true, ref_first);
% Calcultate final layout-path
fprintf('Find highest scoring path\n');
[longest_path_direct weigth_direct] = longest_path_tabu(adj_direct, Startnode, Endnode, quit_treshold, tabu_time, minimum_improvement);
function [contig_target_ref, overlap_target_ref, name_hash_target_ref, delta_target_ref] = ...
GetVariablesForTargetChromosome(...
contig_ref, overlap_ref, deltafiles_ref)
% Select only the entries in the deltafile for the current target chromosome
delta_target_ref = deltafiles_ref;
for di = size(delta_target_ref,2):-1:1
if ~isequal(delta_target_ref(di).R,target_chromosome)
delta_target_ref(di) = [];
end
end
overlap_target_ref = overlap_ref;
for oi = size(overlap_target_ref,2):-1:1
if ~isequal(overlap_target_ref(oi).R,target_chromosome)
overlap_target_ref(oi) = [];
end
end
contig_target_ref = contig_ref;
for ci = size(contig_target_ref,1):-1:1
if isequal(contig_target_ref(ci).assembly, 'reference') && ~isequal(contig_target_ref(ci).name,target_chromosome)
contig_target_ref(ci) = [];
end
end
name_hash_target_ref = make_hash({contig_target_ref.name}');
end
end
There is no exact equivalent of containers.Map in Octave that I know of...
One option is to use the java package to create java.util.Hashtable. Using this example:
pkg load java
d = javaObject("java.util.Hashtable");
d.put('a',1)
d.put('b',2)
d.put('c',3)
d.get('b')
If you are willing to do a bit of rewriting, you can use the builtin struct as a rudimentary hash table with strings (valid variable names) as keys, and pretty much anything stored in values.
For example, given the following:
keys = {'Mon','Tue','Wed'}
values = {10, 20, 30}
you could replace this:
map = containers.Map(keys,values);
map('Mon')
by:
s = struct();
for i=1:numel(keys)
s.(keys{i}) = values{i};
end
s.('Mon')
You might need to use genvarname to produce valid keys, or maybe a proper hashing function that produces valid key strings.
Also look into struct-related functions: getfield, setfield, isfield, fieldnames, rmfield, etc..