I'm creating a text classification model for sentiment analysis, I would like to know what machine learning algorithm is used by createML here?
You can set the algorithm used by the classifier or specify the language which you would like to classify in the parameters of the initializer. See the developer documentation in Xcode under CreateML > MLTextClassifier > MLTextClassifier.ModelParameters
init(validation: MLTextClassifier.ModelParameters.ValidationData, algorithm: MLTextClassifier.ModelAlgorithmType, language: NLLanguage?)
under the parameter algorithm set the algorithm you would like to use.
In developer documentation under the enum MLTextClassifier.ModelAlgorithmType you will find what is available. For example: case crf(revision: Int?) is a conditional random field model algorithm.
Related
newbie here in deep learning. My question is:
I have an already trained object detection (yolov5) model for 3 classes [0,1,2]. Now, my next step is to classify one class , e.g. class [0] as anomaly or not. In other words, I need an additional classifier to further classify it into to sub-classes , i.e., anomalous or non-anomalous through the use of classifier or anomaly detection model. Can you give me an advice on how can I proceed with this? I will use GANs as anomaly detection model. This would be a great help. Thank you in advance.
one of the ways to solve your problems is through One-Class Learning (OCL). In OCL, the algorithm learns from only one user-defined interest class (in your case, non-anomalous objects of class 0) and classifies a new example as belonging to this interest class or not. Thus, you can adapt OCL algorithms to your problem. One of the ways is to use labeled examples of class 0 non-anomalous and use these examples in learning the algorithm. Finally, your algorithm will answer whether new instances of class 0 are non-anomalous (class of interest) or anomalous (non-interest class). Examples of OCL algorithms can be found at: https://scikit-learn.org/stable/modules/outlier_detection.html. It is noteworthy that these are traditional OCL algorithms. On the other hand, there are already versions of OCL algorithms based on deep learning.
In addition, I'll provide you with an example of anomaly detection using the One-Class Support Vector Machine (OCSVM), one of the most traditional famous OCL algorithms.
from sklearn.svm import OneClassSVM as OCSVM
normal_data_of_class_0 = [[],[],[],[], ......, []] #class 0 normal examples
ocsvm = OCSVM()
ocsvm.fit(normal_data_of_class_0)
data_of_class_0 = [[],[],[],[], ......, []] #class 0 new examples
y_pred = ocsvm.predict(data_of_class_0) # +1 == normal data (interest class) || -1 == abnormal data (outliers)
I know Matlab has the function TrainAutoencoder(input, settings) to create and train an autoencoder. The result is capable of running the two functions of "Encode" and "Decode".
But this is only applicable to the case of normal autoencoders. What if you want to have a denoising autoencoder? I searched and found some sample codes, where they used the "Network" function to convert the autoencoder to a normal network and then Train(network, noisyInput, smoothOutput)like a denoising autoencoder.
But there are multiple missing parts:
How to use this new network object to "encode" new data points? it doesn't support the encode().
How to get the "latent" variables to the features, out of this "network'?
I appreciate if anyone could help me resolve this issue.
Thanks,
-Moein
At present (2019a), MATALAB does not permit users to add layers manually in autoencoder. If you want to build up your own, you will have start from the scratch by using layers provided by MATLAB;
In order to to use TrainNetwork(...) to train your model, you will have you find out a way to insert your data into an object called imDatastore. The difficulty for autoencoder's data is that there is NO label, which is required by imDatastore, hence you will have to find out a smart way to avoid it--essentially you are to deal with a so-called OCC (One Class Classification) problem.
https://www.mathworks.com/help/matlab/ref/matlab.io.datastore.imagedatastore.html
Use activations(...) to dump outputs from intermediate (hidden) layers
https://www.mathworks.com/help/deeplearning/ref/activations.html?searchHighlight=activations&s_tid=doc_srchtitle
I swang between using MATLAB and Python (Keras) for deep learning for a couple of weeks, eventually I chose the latter, albeit I am a long-term and loyal user to MATLAB and a rookie to Python. My two cents are that there are too many restrictions in the former regarding deep learning.
Good luck.:-)
If you 'simulation' means prediction/inference, simply use activations(...) to dump outputs from any intermediate (hidden) layers as I mentioned earlier so that you can check them.
Another way is that you construct an identical network but with the encoding part only, copy your trained parameters into it, and feed your simulated signals.
Different teams use different libraries to train and run neural networks (caffe, torch, theano...). This makes sharing difficult: each library has its own format to store networks and you have to install a new library each time you want to test other teams' work.
I am looking for solutions to make this less tedious:
- Is there a preferred (shared?) format to store neural networks?
- Is there a service or library that can help handle different types of networks / or transform one type into another?
Thank you!
Is there a preferred (shared?) format to store neural networks?
Each library / framework has its own serialization, e.g. Caffe uses Protocol Buffers, Torch has a built-in serialization scheme and Theano objects can be serialized with pickle.
In some cases like OverFeat or darknet the weights and biases are stored on-disk in binary format via plain fwrite-s of the corresponding float(or double) contiguous arrays (see this answer for more details). Note that this does not cover the architecture of the network / model which has to be known or represented separately (like declared explicitly at load time).
Also: a library like libccv stores the structure and the weights in a SQLite database.
Is there a service or library that can help handle different types of networks / or transform one type into another?
I don't think there is a single (meta) library that claims to do so. But it exists distinct projects that provide convenient converters.
Some examples (non exhaustive):
Caffe -> Torch: https://github.com/szagoruyko/loadcaffe
Torch -> Caffe: https://github.com/facebook/fb-caffe-exts
Caffe -> TensorFlow: https://github.com/ethereon/caffe-tensorflow
--
UPDATE (2017-09): two noticeable initiatives are:
(1) the ONNX format (a.k.a. Open Neural Network Exchange):
[...] a standard for representing deep learning models that enables models to be transferred between frameworks
See these blog posts.
(2) the CoreML format introduced by Apple:
[...] a public file format (.mlmodel) for a broad set of ML methods [...] Models in this format can be directly integrated into apps through Xcode.
ONNX (Open Neural Network Exchange)
ONNX is an open-source AI ecosystem that provides a common format for neural networks.
It helps converting a deep learning model to another.
Generally, model conversion takes weeks/months without ONNX:
ONNX offers simpler and faster conversion process:
For all supported conversions, see here.
It makes deployment easier, models stored in a much preferable way:
In the above image, ONNX acts as a data ingestion layer, transforms each input model to the same format. Otherwise, all models would be like a bunch of puzzle pieces that do not fit each other.
How to Use ONNX - Keras Conversion Example
Let's say you have your Keras model and you want to transform it to ONNX:
model = load_model("keras_model.hdf5") # h5 is also OK!
onnx_model = keras2onnx.convert_keras(model, model.name)
onnx_model_file = 'output_model.onnx'
onnx.save_model(onnx_model, onnx_model_file)
Then load & run saved model :
onnx_model = onnx.load_model('output_model.onnx')
content = onnx_model.SerializeToString()
sess = onnxruntime.InferenceSession(content)
x = x if isinstance(x, list) else [x]
feed = dict([(input.name, x[n]) for n, input in enumerate(sess.get_inputs())])
# Do inference
pred_onnx = sess.run(None, feed)
This example uses keras2onnx to convert the Keras model and onnxruntime to do inference.
Note: There are also lots of pre-trained models in the ONNX format. Check this out!
References:
1. https://towardsdatascience.com/onnx-made-easy-957e60d16e94
2. https://blog.codecentric.de/en/2019/08/portability-deep-learning-frameworks-onnx/
3. http://on-demand.gputechconf.com/gtc/2018/presentation/s8818-onnx-interoperable-deep-learning-presented-by-facebook.pdf
4. https://devblogs.nvidia.com/tensorrt-3-faster-tensorflow-inference/
I am implementing a ANFIS model with hydrid training method. I have 13 inputs that maps to one output. I framed rules with respect to my system. When I train data, i get the below error:
Number of output MF's is not equal to number of rules
Any clues about where I am going wrong?
Quote from the Fuzzy Logic Toolbox User's Guide:
Constraints of anfis:
anfis is much more complex than the fuzzy
inference systems discussed so far, and is not available for all of
the fuzzy inference system options. Specifically, anfis only supports
Sugeno-type systems, and these must have the following properties:
* Have no rule sharing. Different rules cannot share the same output
membership function, namely the number of output membership functions
must be equal to the number of rules.
The ANFIS model structure should therefore look like this:
FYI the error comes from the following code snippet:
/* output MF no. must be the same as rule no. */
if (fis->output[0]->mf_n != fis->rule_n) {
fisFreeFisNode(fis);
PRINTF("Number of output MF's is not equal to number of rules -->\n");
fisError("Parameter sharing in FIS is not allowed!");
}
What is Conditional Random Field?
How does exactly Conditional Random Field identify proper names as person, organization, or place in a structured or unstructured text?
For example: This product is ordered by StackOverFlow Inc.
What does Conditional Random Field do to identify StackOverFlow Inc. as an organization?
A CRF is a discriminative, batch, tagging model, in the same general family as a Maximum Entropy Markov model.
A full explanation is book-length.
A short explanation is as follows:
Humans annotate 200-500K words of text, marking the entities.
Humans select a set of features that they hope indicate entities. Things like capitalization, or whether the word was seen in the training set with a tag.
A training procedure counts all the occurrences of the features.
The meat of the CRF algorithm search the space of all possible models that fit the counts to find a pretty good one.
At runtime, a decoder (probably a Viterbi decoder) looks at a sentence and decides what tag to assign to each word.
The hard parts of this are feature selection and the search algorithm in step 4.
Well to understand that you got to study a lot of things.
For start
Understand the basic of markov and bayesian networks.
Online course available in coursera by daphne coller
https://class.coursera.org/pgm/lecture/index
CRF is a special type of markov network where we have observation and hidden states.
The objective is to find the best State Assignment to the unobserved variables also known as MAP problem.
Be Prepared for a lot of probability and Optimization. :-)