Keras convolution along single feature row - neural-network

Question

I have a multiclass classification problem. Say I have a feature matrix:
A B C D
1 -1 1 -6
2 0.5 0 11
7 3.7 1 1
4 -50 1 0
And labels:
LABEL
0
1
2
0
2
I want try to apply convolution kernels along each single feature row with Keras. Say nb_filter=2 and batch_size=3. So I expect input shape for convolution layer to be (3, 4) and output shape to be (3, 3) (as it is applied for AB, BC, CD).
Here what I tried with Keras (v1.2.1, Theano backend):
def CreateModel(input_dim, num_hidden_layers):
from keras.models import Sequential
from keras.layers import Dense, Dropout, Convolution1D, Flatten
model = Sequential()
model.add(Convolution1D(nb_filter=10, filter_length=1, input_shape=(1, input_dim), activation='relu'))
model.add(Dense(3, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
model.summary()
return model
def OneHotTransformation(y):
from keras.utils import np_utils
return np_utils.to_categorical(y)
X_train = X_train.values.reshape(X_train.shape[0], 1, X_train.shape[1])
X_test = X_test.values.reshape(X_test.shape[0], 1, X_test.shape[1]),
y_train = OneHotTransformation(y_train)
clf = KerasClassifier(build_fn=CreateModel, input_dim=X_train.shape[1], num_hidden_layers=1, nb_epoch=10, batch_size=500)
clf.fit(X_train, y_train)
Shapes:
print X_train.shape
print X_test.shape
print y_train.shape
Output:
(45561, 44)
(11391, 44)
(45561L,)
When I try to run this code I get and exception:
ValueError: Error when checking model target: expected dense_1 to have 3 dimensions, but got array with shape (45561L, 3L)
I tried to reshape y_train:
y_train = y_train.reshape(y_train.shape[0], 1, y_train.shape[1])
This gives me exception:
ValueError: Error when checking model target: expected dense_1 to have 3 dimensions, but got array with shape (136683L, 2L)
Is this approach with Convolution1D correct to achieve my goal?
If #1 is yes, how can I fix my code?
I've already read numerous github issues and some questions (1, 2) here, but it didn't really help.
Thanks.
UPDATE1:
According to Matias Valdenegro comment.
Here are shapes after reshaping 'X' and after onehot encoding for 'y':
print X_train.shape
print X_test.shape
print y_train.shape
Output:
(45561L, 1L, 44L)
(11391L, 1L, 44L)
(45561L, 3L)
UPDATE2: Thanks again to Matias Valdenegro. X reshaping is done after creating model for sure it was a copy paste issue. The code should look like:
def CreateModel(input_dim, num_hidden_layers):
from keras.models import Sequential
from keras.layers import Dense, Dropout, Convolution1D, Flatten
model = Sequential()
model.add(Convolution1D(nb_filter=10, filter_length=1, input_shape=(1, input_dim), activation='relu'))
model.add(Dense(3, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
model.summary()
return model
def OneHotTransformation(y):
from keras.utils import np_utils
return np_utils.to_categorical(y)
clf = KerasClassifier(build_fn=CreateModel, input_dim=X_train.shape[1], num_hidden_layers=1, nb_epoch=10, batch_size=500)
X_train = X_train.values.reshape(X_train.shape[0], 1, X_train.shape[1])
X_test = X_test.values.reshape(X_test.shape[0], 1, X_test.shape[1]),
y_train = OneHotTransformation(y_train)
clf.fit(X_train, y_train)

Answer 1

The input to a 1D convolution should have dimensions (num_samples, channels, width). So that means you need to reshape X_train and X_test, not y_train:
X_train = X_train.reshape(X_train.shape[0], 1, X_train.shape[1])
X_test = X_test.reshape(X_test.shape[0], 1, X_test.shape[1])