I'm using pytorch to train part of the network. For example, I have a model structure
hidden1 = Layer1(x)
hidden2 = Layer2(hidden1)
out = Layer3(hidden2)
If I want to train Layer3 only, I can use
hidden1 = Layer1(x)
hidden2 = Layer2(hidden1).detach()
out = Layer3(hidden2)
However, this time I want to train Layer1 only. How can I achieve this? Thanks.
detach will not really "freeze" your layer.
If you don't want to train a layer, you should use requires_grad=False instead.
For example:
hidden2.weight.requires_grad = False
hidden2.bias.requires_grad = False
Then to unfreeze, you do the same with requires_grad=True.
Related
I trained a new network using EfficientNet for object detection, instead of using the original output layer of EfficientNet i change it to 5 output dense (5 labels from the original layer).
in the training procedure, i locked all the layers and only train my custom output layer
i managed to achieve a better result in my custom output layer labels
(in EfficientNet they achieve 84% i managed to get an average of 92% accuracy )
i would like to get the original 1000 label and also my own 5 output layer
TLDR
what I am trying to achieve is this :
let's say I have 2 neural networks: A, B they both contains N-1 identical layers and different output layer
A (output layer) dim is 5
B (output layer) dim is 1000
because they both identical up until the output layer they both will have the same output before the output layer
I would like to run model B then get the output of the dropout layer, run this output against model A output layer instead of running both of the models again
is it possible or I am facing an XY problem
?
what is the right way to create a neural network with 2 output layers?
I illustrated the problem :
I found how to do it just need to find the point of the split
copy the first layer and direct its input to the other model and then concatenate all layers together
efficientnet = efn.EfficientNetB0()
retrained_model = load_model(model_path, custom_objects=get_custom_objects())
top_conv = efficientnet.get_layer('top_conv')
top_conv._name = top_conv._name + str("ef0")
top_bn = efficientnet.get_layer("top_bn")
top_bn._name = top_bn._name + str("ef0")
top_activation = efficientnet.get_layer('top_activation')
top_activation._name = top_activation._name + str("ef0")
avg_pool = efficientnet.get_layer('avg_pool')
avg_pool._name = avg_pool._name + str("ef0")
top_dropout = efficientnet.get_layer('top_dropout')
top_dropout._name = top_dropout._name + str("ef0")
probs = efficientnet.get_layer('probs')
probs(top_dropout(
avg_pool(top_activation(top_bn(top_conv(retrained_model.get_layer('block7a_project_bn').output))))))
model = Model(inputs=retrained_model.input,
outputs=[retrained_model.output, probs.get_output_at(0)])
model.summary()
model.save("/home/naor/projects/efficientnetretrainedmodel/bin/model-2-pred.h5")
the new model created as model A input and as output an [A.output ,B.output]
I want to use the svm classifying whether an image contains car or not.
I trained svm classifier using HOG. Then I try to use the classifier, so I looked up certain Mathworks tutorial.
I could not fined any useful tutorial for using svm classifier.
I use the data set from http://cogcomp.org/Data/Car/
This is my code for svm classifier.
imgPos = imread(strrep(file, '*', int2str(0)));
[hog_4x4, vis4x4] = extractHOGFeatures(imgPos,'CellSize',[4 4]);
cellSize = [4 4];
hogFeatureSize = length(hog_4x4);
temp(1:500) = 1;
temp(501:1000) = 0;
trainingLabels = categorical(temp);
trainingFeatures = zeros(fileNum*2, hogFeatureSize, 'single');
for n = 1:500
posfile = strrep(posFile, "*", int2str(n-1));
imgPos = imread(posfile);
trainingFeatures(n, :) = extractHOGFeatures(imgPos, 'CellSize', cellSize);
negfile = strrep(negFile, "*", int2str(n-1));
imgNeg = imread(negfile);
trainingFeatures(n+500, :) = extractHOGFeatures(imgNeg, 'CellSize', cellSize);
end
classifier = fitcecoc(trainingFeatures, trainingLabels);
I want use the classifier to detect car objects.
If it's possible I want to surround each detected car object with frame.
Any help is appreciated.
Your looking for the predict method. Get your test data features and run the following:
predictions = predict(classifier, testFeatures);
Hi I am trying to make following scheme of neural net using either pytorch or keras but it i don't know how to do it, can any one help.
scheme:
Scheme
Here goes a Keras Implementation using the functional API
from keras.models import Model
from keras.layers import Dense, Input, concatenate
def createModel( inp_1_shape, inp_2_shape):
first_input = Input(shape = (inp_1_shape,))
first_dense = Dense(1, )(first_input)
second_input = Input(shape = (inp_2_shape,))
second_dense = Dense(1, )(second_input)
merge = concatenate([first_dense, second_dense])
merge = Dense(2, )(merge)
merge = Dense(3, )(merge)
merge = Dense(1, )(merge)
model = Model(inputs=[first_input, second_input], outputs=merge)
model.compile(optimizer='adam', loss='mse', metrics=['accuracy'])
return model
Just call this function and it will return a keras Model, you might need to double check the no of neurons in each layer, but other than that, you will be fine.
Happy Training
I have the following model:
sharedLSTM1 = LSTM((data.shape[1]), return_sequences=True)
sharedLSTM2 = LSTM(data.shape[1])
def createModel(dropoutRate=0.0, numNeurons=40, optimizer='adam'):
inputLayer = Input(shape=(timesteps, data.shape[1]))
sharedLSTM1Instance = sharedLSTM1(inputLayer)
sharedLSTM2Instance = sharedLSTM2(sharedLSTM1Instance)
dropoutLayer = Dropout(dropoutRate)(sharedLSTM2Instance)
denseLayer1 = Dense(numNeurons)(dropoutLayer)
denseLayer2 = Dense(numNeurons)(denseLayer1)
outputLayer = Dense(1, activation='sigmoid')(denseLayer2)
return (inputLayer, outputLayer)
inputLayer1, outputLayer1 = createModel()
inputLayer2, outputLayer2 = createModel()
model = Model(inputs=[inputLayer1, inputLayer2], outputs=[outputLayer1, outputLayer2])
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
What will be the behaviour of model.fit([data1, data2], [labels1, labels2]) in this model. Will it alternatively train the two NNs for each epoch? Or will it completely train one network, and then train the other? Or maybe some other way?
It will train the only existing network at once.
You don't have two models, you have one model only. This model will be trained.
Data1 and Data2 will be fed simultaneously.
The loss function will be applied to both outputs, and both will backpropagate.
I am trying to implement answer selection model in deep learning as shown below in keras based on this paper,
I understand implementing steps embedding, bi-LSTM and pooling in above flow.
But how do I implement the merge function to compute the cosine similarity and loss function in keras?
loss function is defined as,
L= max{0,M-cosine(q,a+)+cosine(q,a-)}
where,
M = constant margin
q = question
a+ = correct answer
a- = wrong answer
Update 1:
After going through few blogs, this is how I implemented.
#build model
input_question = Input(shape=(max_len, embedding_dim))
input_sentence = Input(shape=(max_len, embedding_dim))
question_lstm = Bidirectional(LSTM(64))
sentence_lstm = Bidirectional(LSTM(64))
encoded_question = question_lstm(input_question)
encoded_sentence = sentence_lstm(input_sentence)
cos_distance = merge([encoded_question, encoded_sentence], mode='cos', dot_axes=1)
cos_distance = Reshape((1,))(cos_distance)
cos_similarity = Lambda(lambda x: 1-x)(cos_distance)
predictions = Dense(1, activation='sigmoid')(cos_similarity)
model = Model([input_question, input_sentence], [predictions])
model.compile('adam', 'binary_crossentropy', metrics=['accuracy'])
With above implementation, I am still not able to figure out how to implement hinge loss. Please help