I am using a Keras network that uses an input image of 128x128 pixels, this network got an accuracy of more than 85% on the chars74K dataset. When I converted this network to a CoreML model the results are always 100% certain but always wrong, never the correct letter. The code for my Keras network can be found here: https://github.com/thijsheijden/chars74kCNN
The code I used to convert to a CoreMLModel is the following:
import coremltools
import h5py
import pandas
coreml_model = coremltools.converters.keras.convert(
"chars74kV4.0.h5", class_labels = "class_labels.txt", image_input_names= ['input'], input_names=['input'])
coreml_model.author = 'Thijs van der Heijden'
coreml_model.license = 'MIT'
coreml_model.description = 'A basic Deep Convolutional Neural Network to classify handwritten letters.'
coreml_model.input_description['input'] = 'A 128x128 pixel Image'
coreml_model.save('chars74k.mlmodel')
The code for my IOS App can be found here: https://github.com/thijsheijden/Visionary
I would greatly appreciate any help as I am really stuck on this one! Thanks in advance!
Related
I’m trying to import a DNN trained model from MATLAB to PyTorch.
I’ve found solutions for the opposite case (from PyTorch to MATLAB), but no proposed solutions on how to import a trained model from MATLAB to PyTorch.
Any ideas, please?
You can first export your model to ONNX format, and then load it using ONNX; prerequisites are:
pip install onnx onnxruntime
Then,
onnx.load('model.onnx')
# Check that the IR is well formed
onnx.checker.check_model(model)
Until this point, you still don't have a PyTorch model. This can be done through various ways since it's not natively supported.
A workaround (by loading only the model parameters)
import onnx
onnx_model = onnx.load('model.onnx')
graph = onnx_model.graph
initalizers = dict()
for init in graph.initializer:
initalizers[init.name] = numpy_helper.to_array(init)
for name, p in model.named_parameters():
p.data = (torch.from_numpy(initalizers[name])).data
Using onnx2pytorch
import onnx
from onnx2pytorch import ConvertModel
onnx_model = onnx.load('model.onnx')
pytorch_model = ConvertModel(onnx_model)
Note: Time Consuming
Using onnx2keras, then MMdnn to convert from Keras to PyTorch (Examples)
I just created a model that does a binary classification and has a dense layer of 1 unit at the end. I used Sigmoid activation. However, I get this error now when I wanna convert it to CoreML.
I tried to change the number of units to 2 and activation to softmax but still didn't work.
import coremltools as ct
#1. define input size
image_input = ct.ImageType(scale=1/255)
#2. give classifier
classifier_config = coremltools.ClassifierConfig(class_labels=[0, 1]) #ERROR here
#3. convert the model
coreml_model = coremltools.convert("mask_detection_model_surgical_mask.h5",
inputs=[image_input], classifier_config=classifier_config)
#4. load and resize an example image
example_image = Image.open("Unknown3.jpg").resize((256, 256))
# Make a prediction using Core ML
out_dict = coreml_model.predict({mymodel.input_names[0]: example_image})
print(out_dict["classLabels"])
# save to disk
#coreml_model.save("FINALLY.mlmodel")
I found the answer to my question.
Use Softmax activation and 2 Dense units as the final layer with either loss='binary_crossentropy' or `loss='categorical_crossentropy'
Good luck to hundreds of people who posted a similar question but received no answer.
I'm trying to train a hybrid model with GP on top of pre-trained CNN (Densenet, VGG and Resnet) with CIFAR10 data, mimic the ex2 function in the gpflow document. But the testing result is always between 0.1~0.2, which generally means random guess (Wilson+2016 paper shows hybrid model for CIFAR10 data should get accuracy of 0.7). Could anyone give me a hint of what could be wrong?
I've tried same code with simpler cnn models (2 conv layer or 4 conv layer) and both have reasonable results. I've tried to use different Keras applications: Densenet121, VGG16, ResNet50, neither works. I've tried to freeze the weights in the pre-trained models still not working.
def cnn_dn(output_dim):
base_model = DenseNet121(weights='imagenet', include_top=False, input_shape=(32,32,3))
bout = base_model.output
fcl = GlobalAveragePooling2D()(bout)
#for layer in base_model.layers:
# layer.trainable = False
output=Dense(output_dim, activation='relu')(fcl)
md=Model(inputs=base_model.input, outputs=output)
return md
#add gp on top, reference:ex2() function in
#https://nbviewer.jupyter.org/github/GPflow/GPflow/blob/develop/doc/source/notebooks/tailor/gp_nn.ipynb
#needs to slightly change build graph part because keras variable #sharing is not the same as tensorflow
#......
## build graph
with tf.variable_scope('cnn'):
md=cnn_dn(gp_dim)
f_X = tf.cast(md(X), dtype=float_type)
f_Xtest = tf.cast(md(Xtest), dtype=float_type)
#......
## predict
res=np.argmax(sess.run(my, feed_dict={Xtest:xts}),1).reshape(yts.shape)
correct = res == yts.astype(int)
print(np.average(correct.astype(float)))
I finally figure out that the solution is training larger iterations. In the original code, I just use 50 iterations as used in the ex2() function for MNIST data and it is not enough for more complicated network and CIFAR10 data. Adjusting some hyper-parameter (e.g. learning rate and activation function) also helps.
I have this AlexNet model in MATLAB:
net = alexnet;
layers = net.Layers;
layers(end-2) = fullyConnectedLayer(numClasses);
layers(end) = classificationLayer;
I'm using it to learn features from sequencies of frames from videos of different classes. So i need to extract learned features from the 'fc7' layer of this model to save these features as a vector and pass it to an LSTM layer.
The training process of this model for transfer learning its ok, all right.
I divided my data set in a x_train and a x_test sets using splitEachLabel() in my imageDatastore(), and using the function augmentedImageSource() to resize all the images for the network. Everything ok!
But when i try yo use this snippet of code shown bellow to resize images from my imageDatastore to be readed by the function activations(), to save the features as a vector, i'm getting an error:
imageSize = [227 227 3];
auimds = augmentedImageSource(imageSize, imds, 'ColorPreprocessing', 'gray2rgb');
Function activations:
layer = 'fc7';
fclayer = activations(mynet, auimds, layer,'OutputAs','columns');
The error:
Error using SeriesNetwork>iDataDispatcher (line 1113)
For an image input layer, the input data for predict must be a single image, a 4D array of images, or an imageDatastore with the correct size.
Error in SeriesNetwork/activations (line 791)
dispatcher = iDataDispatcher( X, miniBatchSize, precision, ...
Someone help me, please!
Thanks for the support!
Did you check the input size of that layer? The error you are getting is related with the input size of the current layer. Can you check your mynet structure and its fc7 layer input's size in your workspace in Matlab?
I am using Keras (version 2.0.0) and I'd like to make use of pretrained models like e.g. VGG16.
In order to get started, I ran the example of the [Keras documentation site ][https://keras.io/applications/] for extracting features with VGG16:
from keras.applications.vgg16 import VGG16
from keras.preprocessing import image
from keras.applications.vgg16 import preprocess_input
import numpy as np
model = VGG16(weights='imagenet', include_top=False)
img_path = 'elephant.jpg'
img = image.load_img(img_path, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
features = model.predict(x)
The used preprocess_input() function bothers me
(the function does Zero-centering by mean pixel what can be seen by looking at the source code).
Do I really have to preprocess input data (validation/test data) before using a trained model?
a)
If yes, one can conclude that you always have to be aware of what preprocessing steps have been performed during training phase?!
b)
If no: Does preprocessing of validation/test data cause a bias?
I appreciate your help.
Yes you should use the preprocessing step. You can retrain the model without it but the first layers will learn to center your datas so this is a waste of parameters.
If you do not recenter your performances will suffer.
Great thread on reddit : https://www.reddit.com/r/MachineLearning/comments/3q7pjc/why_is_removing_the_mean_pixel_value_from_each/