Caffe, operations among batches - neural-network

Question

Since I have a classifier based on single patch scores, I would like to sum together the predictions a network produces for different images.
From
https://github.com/BVLC/caffe/blob/master/src/caffe/proto/caffe.proto, Reduction does not support operation over axis different than the last one.
Also a pooling operation would produce an average of its input but, obviously, without touching on the full batch.
I have implemented a python layer, but this is not fast enough for large scale experiments.
Is there a way to "sum" or, more generally, operate over the first axis with the tools already available?

Answer 1

Yes. You can. If you have N x p x q x r blob of prediction, the first use Slice (SliceLayer), creating N blobs, each of shape 1 x p x q x r. Then use these N blobs as N bottoms for the eltwise (EltwiseLayer) layer to produce a single top.

Answer 2

If your predictions have the dimensions: N x c (for mini-batch size of N and c channels), then you can splice this into c blobs with dimension N. You can feed these into a Reduction layer.
For example, you to write the following as a Jinja2 template:
layer {
name: "pred-slice"
type: "Slice"
bottom: "pred"
{%- for num in range(10) %}
top: "pred-{{ num }}-vector"
{%- endfor %}
slice_param {
slice_dim: 1
{%- for num in range(1, 10) %}
slice_point: {{ num }}
{%- endfor %}
}
include {
phase: TEST
}
}
{%- for num in range(10) %}
layer {
name: "pred-{{num}}"
type: "Reduction"
bottom: "pred-{{ num }}-vector"
top: "pred-{{ num }}"
include {
phase: TEST
}
reduction_param {
operation: MEAN
}
}
{%- endfor %}
which expands to:
layer {
name: "pred-slice"
type: "Slice"
bottom: "pred"
top: "pred-0-vector"
top: "pred-1-vector"
top: "pred-2-vector"
top: "pred-3-vector"
top: "pred-4-vector"
top: "pred-5-vector"
top: "pred-6-vector"
top: "pred-7-vector"
top: "pred-8-vector"
top: "pred-9-vector"
slice_param {
slice_dim: 1
slice_point: 1
slice_point: 2
slice_point: 3
slice_point: 4
slice_point: 5
slice_point: 6
slice_point: 7
slice_point: 8
slice_point: 9
}
include {
phase: TEST
}
}
layer {
name: "pred-0"
type: "Reduction"
bottom: "pred-0-vector"
top: "pred-0"
include {
phase: TEST
}
reduction_param {
operation: MEAN
}
}
layer {
name: "pred-1"
type: "Reduction"
bottom: "pred-1-vector"
top: "pred-1"
include {
phase: TEST
}
reduction_param {
operation: MEAN
}
}
layer {
name: "pred-2"
type: "Reduction"
bottom: "pred-2-vector"
top: "pred-2"
include {
phase: TEST
}
reduction_param {
operation: MEAN
}
}
layer {
name: "pred-3"
type: "Reduction"
bottom: "pred-3-vector"
top: "pred-3"
include {
phase: TEST
}
reduction_param {
operation: MEAN
}
}
layer {
name: "pred-4"
type: "Reduction"
bottom: "pred-4-vector"
top: "pred-4"
include {
phase: TEST
}
reduction_param {
operation: MEAN
}
}
layer {
name: "pred-5"
type: "Reduction"
bottom: "pred-5-vector"
top: "pred-5"
include {
phase: TEST
}
reduction_param {
operation: MEAN
}
}
layer {
name: "pred-6"
type: "Reduction"
bottom: "pred-6-vector"
top: "pred-6"
include {
phase: TEST
}
reduction_param {
operation: MEAN
}
}
layer {
name: "pred-7"
type: "Reduction"
bottom: "pred-7-vector"
top: "pred-7"
include {
phase: TEST
}
reduction_param {
operation: MEAN
}
}
layer {
name: "pred-8"
type: "Reduction"
bottom: "pred-8-vector"
top: "pred-8"
include {
phase: TEST
}
reduction_param {
operation: MEAN
}
}
layer {
name: "pred-9"
type: "Reduction"
bottom: "pred-9-vector"
top: "pred-9"
include {
phase: TEST
}
reduction_param {
operation: MEAN
}
}