What is the purpose of a sequence folding layer in matlab? - matlab

Question

When designing a CNN for 1D time series signal classification in MATLAB i get the error that the 2dconvolutional layer does not take sequences as input. From my understanding it is perfectly possible to convolve of an "array" with a 3x1 filter. To resolve this issue MATLAB suggests to use a "sequence folding layer". What would be the function of such a sequence folding layer and how would the architecture need to be changed?
I get the following error message:

Answer 1

What would be the function of such a sequence folding layer and how would the architecture need to be changed?
Simply speaking, you're parsing in a sequence (or as you called it - "an array") of images whereas you need to convert them into a batch of images before performing any convolutional operations.
Documentation about sequenceFoldingLayer:
A sequence folding layer converts a batch of image sequences to a batch of images. Use a sequence folding layer to perform convolution operations on time steps of image sequences independently.
Regarding the usage of a sequenceFoldingLayer (once again suggested to check out the documentation):
To use a sequence folding layer, you must connect the miniBatchSize output to the miniBatchSize input of the corresponding sequence unfolding layer. For an example, see Create Network for Video Classification.
On said website are also lots of examples on how to create a sequence folding layer - e.g. one with the name of "fold1":
layer = sequenceFoldingLayer('Name','fold1')
... as well as examples on how to properly implement it within your project.

Answer 2

What would be the function of such a sequence folding layer and how would the architecture need to be changed?
Using sequenceFolding/sequenceUnFoldingLayer works as follows experimentally, the results in the red box in the figure are consistent.
%% Testing the sequenceFoldingLayer/sequenceUnfoldingLayer internal flow
% The difference between using the sequence layer and not using the sequenceFolding/sequenceUnFoldingLayer to extract features
% Common parameters used below
%
% 测试sequenceFoldingLayer/sequenceUnfoldingLayer内部具体操作流程
% 用和不用sequence layer的区别，使用sequenceFolding/sequenceUnFoldingLayer提取出特征结果比较
% 下面使用的共有参数
inputSize = [28 28 1];
filterSize = 5;
numFilters = 20;
numHiddenUnits = 200;
numClasses = 10;
inputData = rand(28,28,1,30);% h*w*c*n
convL = convolution2dLayer(filterSize,numFilters,'Name','conv',...
'Weights',ones(filterSize,filterSize,1,numFilters),...
'Bias',ones(1,1,numFilters));
bnL= batchNormalizationLayer('Name','bn',...
'TrainedVariance',ones(1,1,numFilters),...
'TrainedMean',zeros(1,1,numFilters),...
'Offset',zeros(1,1,numFilters),...
'Scale',ones(1,1,numFilters));
%% first stage, use sequence layer
layers = [ ...
sequenceInputLayer(inputSize,'Name','input')
sequenceFoldingLayer('Name','fold')
convL
bnL
reluLayer('Name','relu')
sequenceUnfoldingLayer('Name','unfold')
flattenLayer('Name','flatten')
lstmLayer(numHiddenUnits,'OutputMode','last','Name','lstm',...
'InputWeights',ones(800,11520),...
'RecurrentWeights',ones(800,200),...
'Bias',ones(800,1));
fullyConnectedLayer(numClasses, 'Name','fc',...
'Weights',ones(numClasses,200),...
'Bias',ones(numClasses,1));
softmaxLayer('Name','softmax')
classificationLayer('Name','classification')];
lgraph = layerGraph(layers);
lgraph = connectLayers(lgraph,'fold/miniBatchSize','unfold/miniBatchSize');
assembleNet = assembleNetwork(lgraph);
%% second stage: not use sequence layer
% The sequenceFoldingLayer is essentially a per-frame graph acquisition feature as follows
layers2 = [ ...
imageInputLayer(inputSize,'Name','input','Normalization','none')
convL
bnL
reluLayer('Name','relu')
fullyConnectedLayer(numClasses, 'Name','fc',...
'Weights',ones(numClasses,11520),...
'Bias',ones(numClasses,1));
softmaxLayer('Name','softmax')
classificationLayer('Name','classification')];
lgraph2 = layerGraph(layers2);
assembleNet2 = assembleNetwork(lgraph2);
%% visualize
analyzeNetwork(lgraph)
analyzeNetwork(lgraph2)
%% comparison
out1 = activations(assembleNet,inputData,'unfold','OutputAs','channels');
out2 = activations(assembleNet2,inputData,'relu','OutputAs','channels');
t1 = out1{1};
t2 = out2;
%% result is same
t1(1:10)
t2(1:10)