caffe: A CNN with two loss layers and two different label sets - neural-network

Question

I am interested to have a deep CNN with two loss layers, each of them having its own separate label set. For example, imagine a 10 layer CNN, loss1 is located in 5th layer using label1, then loss2 in the last layer using label2.
Note: label1 (loss1) can be considered as pre-processing step, in order to ease the task of the CNN to solve label2 (loss2 i.e. real label).
Question: as far as i know, caffe loss layer automatically uses "label" (in my .lmdb dataset). how to make it to use "label1" in loss1 and "label2" in the loss2?
Update: so i took Shai's advice to use HDF5. here is my progress so far (giving only the input and output layers):
layer {
name: "data"
type: "HDF5Data"
top: "data"
top: "label1"
top: "label2"
include {
phase: TEST
}
hdf5_data_param {
source: "data/2048_1e3_0.00/2048_1e3_0.00_s_val_list.txt"
}
}
layer {
name: "data"
type: "HDF5Data"
top: "data"
top: "label1"
top: "label2"
include {
phase: TRAIN
}
hdf5_data_param {
source: "data/2048_1e3_0.00/2048_1e3_0.00_s_train_list.txt"
}
...
layer {
name: "loss1"
type: "EuclideanLoss"
bottom: "ampl"
bottom: "label1"
top: "loss1"
}
...
layer {
name: "loss2"
type: "EuclideanLoss"
bottom: "ampl"
bottom: "label2"
top: "loss2"
}
Am I in the right direction?

Answer 1

AFAIK Caffe stores binary Datums in lmdb. The way Datum is defined restricts it to a single integer label per input image. Therefore it can be very tricky to use "Data" layer with lmdb in your scenario.
On the other hand you can use "HDF5Data" layer instead: this layer is much more flexible and can easily provide with more than a single label per input image. There are several examples here at SO on how to use this input layer, see, e.g. this answer.