I have the following model:
sharedLSTM1 = LSTM((data.shape[1]), return_sequences=True)
sharedLSTM2 = LSTM(data.shape[1])
def createModel(dropoutRate=0.0, numNeurons=40, optimizer='adam'):
inputLayer = Input(shape=(timesteps, data.shape[1]))
sharedLSTM1Instance = sharedLSTM1(inputLayer)
sharedLSTM2Instance = sharedLSTM2(sharedLSTM1Instance)
dropoutLayer = Dropout(dropoutRate)(sharedLSTM2Instance)
denseLayer1 = Dense(numNeurons)(dropoutLayer)
denseLayer2 = Dense(numNeurons)(denseLayer1)
outputLayer = Dense(1, activation='sigmoid')(denseLayer2)
return (inputLayer, outputLayer)
inputLayer1, outputLayer1 = createModel()
inputLayer2, outputLayer2 = createModel()
model = Model(inputs=[inputLayer1, inputLayer2], outputs=[outputLayer1, outputLayer2])
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
What will be the behaviour of model.fit([data1, data2], [labels1, labels2]) in this model. Will it alternatively train the two NNs for each epoch? Or will it completely train one network, and then train the other? Or maybe some other way?
It will train the only existing network at once.
You don't have two models, you have one model only. This model will be trained.
Data1 and Data2 will be fed simultaneously.
The loss function will be applied to both outputs, and both will backpropagate.
Related
I have a LeNet-300-100 dense neural network for MNIST dataset where I want to freeze the first two layers having 300 and 100 hidden neurons in the first two hidden layers. I just want to train the output layer. The code I have to do this is as follows:
from tensorflow import keras
inner_model = keras.Sequential(
[
keras.Input(shape=(1024,)),
keras.layers.Dense(300, activation="relu", kernel_initializer = tf.initializers.GlorotNormal()),
keras.layers.Dense(100, activation="relu", kernel_initializer = tf.initializers.GlorotNormal()),
]
)
model_mnist = keras.Sequential(
[keras.Input(shape=(1024,)), inner_model, keras.layers.Dense(10, activation="softmax"),]
)
# model_mnist.trainable = True # Freeze the outer model
# Freeze the inner model-
inner_model.trainable = False
# Sanity check-
inner_model.trainable, model_mnist.trainable
# (False, True)
# Compile NN-
model_mnist.compile(
loss=tf.keras.losses.categorical_crossentropy,
# optimizer='adam',
optimizer=tf.keras.optimizers.Adam(lr = 0.0012),
metrics=['accuracy'])
However, this code doesn't seem to be freezing the first two hidden layers and they are also learning. What am I doing wrong?
Thanks!
Solution: Use 'trainable' parameter while defining neural network model to freeze the desired layers of the model as follows-
model = Sequential()
model.add(Dense(units = 300, activation="relu", kernel_initializer = tf.initializers.GlorotNormal(), trainable = False))
model.add(Dense(units = 100, activation = "relu", kernel_initializer = tf.initializer.GlorotNormal(), trainable = False))
model.add(Dense(units = 10, activation = "softmax"))
# Compile model as usual
I'm using pytorch to train part of the network. For example, I have a model structure
hidden1 = Layer1(x)
hidden2 = Layer2(hidden1)
out = Layer3(hidden2)
If I want to train Layer3 only, I can use
hidden1 = Layer1(x)
hidden2 = Layer2(hidden1).detach()
out = Layer3(hidden2)
However, this time I want to train Layer1 only. How can I achieve this? Thanks.
detach will not really "freeze" your layer.
If you don't want to train a layer, you should use requires_grad=False instead.
For example:
hidden2.weight.requires_grad = False
hidden2.bias.requires_grad = False
Then to unfreeze, you do the same with requires_grad=True.
I want to use the svm classifying whether an image contains car or not.
I trained svm classifier using HOG. Then I try to use the classifier, so I looked up certain Mathworks tutorial.
I could not fined any useful tutorial for using svm classifier.
I use the data set from http://cogcomp.org/Data/Car/
This is my code for svm classifier.
imgPos = imread(strrep(file, '*', int2str(0)));
[hog_4x4, vis4x4] = extractHOGFeatures(imgPos,'CellSize',[4 4]);
cellSize = [4 4];
hogFeatureSize = length(hog_4x4);
temp(1:500) = 1;
temp(501:1000) = 0;
trainingLabels = categorical(temp);
trainingFeatures = zeros(fileNum*2, hogFeatureSize, 'single');
for n = 1:500
posfile = strrep(posFile, "*", int2str(n-1));
imgPos = imread(posfile);
trainingFeatures(n, :) = extractHOGFeatures(imgPos, 'CellSize', cellSize);
negfile = strrep(negFile, "*", int2str(n-1));
imgNeg = imread(negfile);
trainingFeatures(n+500, :) = extractHOGFeatures(imgNeg, 'CellSize', cellSize);
end
classifier = fitcecoc(trainingFeatures, trainingLabels);
I want use the classifier to detect car objects.
If it's possible I want to surround each detected car object with frame.
Any help is appreciated.
Your looking for the predict method. Get your test data features and run the following:
predictions = predict(classifier, testFeatures);
Hi I am trying to make following scheme of neural net using either pytorch or keras but it i don't know how to do it, can any one help.
scheme:
Scheme
Here goes a Keras Implementation using the functional API
from keras.models import Model
from keras.layers import Dense, Input, concatenate
def createModel( inp_1_shape, inp_2_shape):
first_input = Input(shape = (inp_1_shape,))
first_dense = Dense(1, )(first_input)
second_input = Input(shape = (inp_2_shape,))
second_dense = Dense(1, )(second_input)
merge = concatenate([first_dense, second_dense])
merge = Dense(2, )(merge)
merge = Dense(3, )(merge)
merge = Dense(1, )(merge)
model = Model(inputs=[first_input, second_input], outputs=merge)
model.compile(optimizer='adam', loss='mse', metrics=['accuracy'])
return model
Just call this function and it will return a keras Model, you might need to double check the no of neurons in each layer, but other than that, you will be fine.
Happy Training
I'm new in machine learning (and to stackoverflow as well) and i want to make some classification tasks. I performed two group classifications on my data set (field of speech acoustics) with LIBSVM and Matlab's Pattern Recignition Tool from the Neural network toolbox to create a simple network with one hidden layer. In the hope of higher classification results i want to try Deep Neural Networks, and i found this code: http://www.mathworks.com/matlabcentral/fileexchange/42853-deep-neural-network
I have some difficulty understanding it.
My data is constructed of 127 samples of 19 parameters, so my input number is 19. I want to classify them in two groups: 0 and 1, so my output number is 1. The values in my data set are normalized between 0 and 1.
My code is the following:
clear all
clc
addpath('..');
load('data.mat')
inputdata = inputs;
outputdata = outputs;
datanum = 127;
outputnum = 1;
hiddennum = 3;
inputnum = 19;
% rbm = randRBM(inputnum, outputnum);
% rbm = pretrainRBM( rbm, inputdata );
dbn = randDBN([inputnum, hiddennum, outputnum]);
dbn = pretrainDBN( dbn, inputdata );
dbn = SetLinearMapping( dbn, inputdata, outputdata );
dbn = trainDBN( dbn, inputdata, outputdata );
estimate = v2h( dbn, inputdata )
[rmse AveErrNum] = CalcRmse(dbn, inputdata, outputdata)
The code runs. The rmse is 0.4183, the AveErrNum is 0.1969. What i need is the classification accuracy between my targets (stored in outputdata) and the networks predictions (Accuracy = data classified correctly / all data).
Where do i find the networks predictions after binarization?
Do I use the right type of network for my classification?
Don't I need to divide my data into Training, Validation and Testing samples (like in the case of a simple neural network with one hidden layer)?
Thanks in advance for any help!