Our current flow: Conversation of tf2->onnx->TensorRT (all16 and 32 and 8 bits options)
Is there an existing tool like https://github.com/lutzroeder/netron (or any other way) to see the output model of TensorRT?
You can now visualize tensorrt engine graphs using https://github.com/NVIDIA/TensorRT/tree/main/tools/experimental/trt-engine-explorer
It also offers other info about the engine.
There is no way to do this, because the TensorRT model (engine) is optimized for certain hardware (A specific NVIDIA GPU architecture). It is already "compiled" (similar to coreML mlmodel's are compiled to mlmodelc, where as TensorFlow (or TFLite) models are not "compiled" for specific hardware.
From the NVIDIA forums: https://forums.developer.nvidia.com/t/visualizing-tensorrt-engine/69032
I assume you are asking for something analogous to tensorboard or netron? Unfortunately, I’m not aware a way for TRT.
From the creator of Netron (lutzroeder):
Unless this format is documented not sure much can be done here. Netron has a browser-only version so even if a Python or C++ API existed for supporting non-inference scenarios it wouldn't be useful.
You can still of course visualize the ONNX model right before you create the TensorRT one. TensorRT just needs an optimized model, so I don't expect it to be different.
Is the goal to visualize it after the compression from onnx/TensorRT? Or just to visualize it in general? You can always just use the keras tool.
I believe you can use a TensorRT model as well with netron, based on this youtube video.
Related
I am new to rasa and started creating a very domain-specific chatbot. As part of it, I understand its better to use supervised embeddings as part of nlu pipeline, since my use case is domain-specific.
I have an example intent in my nlu.md
## create_system_and_config
- create a [VM](system) of [12 GB](config)
If I try to use a supervised featurizer, it might be working fine with my domain-specific entities, but my concern here is, by using only supervised learning, won't we lose the advantage of pre-trained models? For example, in a query such as add a (some_system) of (some_config). add and create are very closely related. pre-trained models will be able to pick such verbs easily. Is it possible to have a combination of pre-trained model and then do some supervised learning on top of it in our nlu pipeline, something like transfer learning?
If you're creating domain-specific chatbot, it's always better to use supervised embedding instead of pre-trained
For example, in general English, the word “balance” is closely related
to “symmetry”, but very different to the word “cash”. In a banking
domain, “balance” and “cash” are closely related and you’d like your
model to capture that.
In your case also
your model needs to capture that words VM and Virtual Machine are same. Pretrained featurizers are not trained to capture this and they are more generic.
The advantage of using pre-trained word embeddings in your pipeline is
that if you have a training example like: “I want to buy apples”, and
Rasa is asked to predict the intent for “get pears”, your model
already knows that the words “apples” and “pears” are very similar.
This is especially useful if you don’t have enough training data
For more details you can refer Rasa document
I've generated a PySpark Word2Vec model like so:
from pyspark.ml.feature import Word2Vec
w2v = Word2Vec(vectorSize=100, minCount=1, inputCol='words', outputCol = 'vector')
model = w2v.fit(df)
(The data that I used to train the model on isn't relevant, what's important is that its all in the right format and successfully yields a pyspark.ml.feature.Word2VecModel object.)
Now I need to convert this model to a Gensim Word2Vec model. How would I go about this?
If you still have the training data, re-training the gensim Word2Vec model may be the most straightforward approach.
If you only need the word-vectors, perhaps PySpark's model can export them in the word2vec.c format that gensim can load with .load_word2vec_format().
The only reason to port the model would be to continue training. Such incremental training, while possible, involves considering a lot of tradeoffs in balancing the influence of the older and later training to get good results.
If you are in fact wanting to do this conversion in order to do more training in such a manner, it again suggests that using the original training to reproduce a similar model could be plausible.
But, if you have to convert the model, the general approach would be to study the source code and internal data structures of the two models, to discover how they alternatively represent each of the key aspects of the model:
the known word-vectors (model.wv.vectors in gensim)
the known-vocabulary of words, including stats about word-frequencies and the position of individual words (model.wv.vocab in gensim)
the hidden-to-output weights of the model (`model.trainables' and its properties in gensim)
other model properties describing the model's modes & metaparameters
A reasonable interactive approach could be:
Write some acceptance tests that take models of both types, and test whether they are truly 'equivalent' for your purposes. (This is relatively easy for just checking if the vectors for individual words are present and identical, but nearly as hard as the conversion itself for verifying other ready-to-be-trained-more behaviors.)
Then, in an interactive notebook, load the source model, and also create a dummy gensim model with the same vocabulary size. Consulting the source code, write Python statements to iteratively copy/transform key properties over from the source into the target, repeatedly testing if they verify as equivalent.
When they do, take those steps you did manually and combine them into a utility method to do the conversion. Again verify its operation then try using the converted model however you'd hoped – perhaps discovering overlooked info or discovering other bugs in the process, and then improving the verification method and conversion method.
It's possible that the PySpark model will be missing things the gensim model expects, which might require synthesizing workable replacement values.
Good luck! (But re-train the gensim model from the original data if you want things to just be straightforward and work.)
Assuming I model a complete system correctly, according to the Modelica syntax, are the compilers 'mature' enough to handle it?
I need to model a system with at least 15 connected components, each components is relatively simple, mathematically speaking, only algebric equations. Modelica is very appealing to me but I am a complete beginner and this project is important to me so I'm a little bit afraid to commit to Modelica.
I understand that compilers can't fully simulate all of the standard library examples and models, how can I know what are the exact limitations?
Thanks.
Well, it depends quite a bit on what tool you choose of course. I can tell you from personal experience that over 10 years ago I used Dymola in a project at Ford Motor Company where we modeled an engine (combustion), transmission (mechanisms and hydraulics) and chassis (multi-body representation). The resulting system had 250,000 equations and certainly hundreds if not thousands of components and connections. You can find more information about the project in a paper I wrote.
Of course, it depends on other things besides the size of your models. Most Modelica tools do not really support variable structure (DAEs with variable index) and others have limitations with respect to some of the language constructs that they fully support (which therefore means some libraries are not fully supported).
Unfortunately, at the moment there is not a comprehensive way to qualify the support from different tools but this is something that the Modelica Association recognizes is a problem and they are working on it.
But overall, Modelica is quite mature and is used in many, many industrial projects. You can find the proceedings from the previous 8 Modelica Conferences at http://www.modelica.org/ and you will see that many big name companies (Ford, BMW, GM, Toyota, Airbus, etc) have published material there.
In a project that I'm working on I have a decision problem and none of my attempts have given satisfying results: using domain specifik knowledge, trying to generate statistics and creating a statistical model etc etc.
I have basic knowledge about neural networks, now I want to try if that approach might yield some good results. I have a lot of data so all I want is basically to set up a simple NN, train it and see what I get.
Do you know about any decent Perl modules that might use for this purpose? I've found a few but I'd hate to waste my time trying all of them.
There are several, but I'd start with AI::NeuralNet::Simple. On CPAN, the AI namespace has a whole bunch of others, some are wrappers for C libraries, others do fancy neural net things, but this is a simple base to start with. And it is well documented, which is not the case for some of the others
Caveats: none of the NN modules is that well maintained or widely tested. This one explicitly states that it is alpha code not for production use. For a proof-of-concept, it should be fine.
I'd use AI::FANN , you can find it on CPAN as well.
I'm currently looking into unifying the model of my application. At the moment I'm using a different model on the mac/iphone. Missing classes (NSAttributedString) and missing technologies (Bindings) made me go with this decision.
With the first restriction gone in SDK 3.2 and my plan to also create an optimized iPad version I'm reconsidering my decision. As I also need to store NSPoints/CGPoints, NSRect/CGRects, NSColor/UIColor and NSImage/UIImage structs/objects in my model I'm not sure what the best way to handle them would be.
Writing my own MNColor object that encapsulates NSColor and UIColor depending on architecture?
Writing my own rect-functions that call the appropriate function depending on arch? Or use CGRect in the model on the mac?
Any input would be very appreciated!
CorePlot is an excellent Cocoa plotting framework for iPhone and Mac OS X.
It shares common code across both platforms - Maybe you can get some ideas by browsing their source.
For some cross-plattform issues, CP uses platform specific defines in separate header files to get an "agnostic" image class.
The Xcode project for each platform includes one of the according headers:
MacOnly/CPPlatformSpecificDefines.h
iPhoneOnly/CPPlatformSpecificDefines.h
They also have a custom color class CPColor. (Based on CGColorRef)
For NSPoint and NSRect, I would use the Core Graphics structs in the model and convert them using NSRectFromCGRect and NSPointFromCGPoint where needed. (Mac OS 10.5+)
A recent CIMGF article also deals with iPhone/Mac incompatibilities:
Creating a NSManagedObject that is Cross Platform
I might be misunderstanding your setup and question but it sounds like you have a insufficiently abstracted data model.
Strictly speaking, "NSPoints/CGPoints, NSRect/CGRects, NSColor/UIColor and NSImage/UIImage structs/objects" are all implementation/UI elements that have nothing to do with the data model. Granted, the API makes it easy to archive these but this lures you into the problem you have now. You're saving objects/structs that are attached to specific hardware and specific implementations and now you can port/reuse them easily.
The better way is to create an abstracted data model that knows nothing about the hardware or the rest of the API. It should store all the NSPoints/CGPoints, NSRect/CGRects as strings or numbers. It should store the colors as numbers, strings or raw data. The images should be stored as raw data.
This way the core of your application i.e. the data it actual manipulates is generic. To display the information, you just need your controller to request the raw data and let the controller convert it the hardware/API specific struct/object.
Core data provides a good example of an abstracted data model. It only stores strings, numbers, dates, booleans etc yet it can store any information of arbitrary complexity for any platform that supports core data.
Even if you don't use Core Data, that is the type of data model you should shoot for.
#"Writing my own rect-functions that call the appropriate function depending on arch" -
This will be good.
#"Writing my own MNColor object that encapsulates NSColor and UIColor" -
Will be good provided your design of wrapper class is capable of handling MNColor object in cross-platform situations. i.e. The database from mac if imported to iPhone should now be capable of somehow providing UIColor object through your wrapper instead of NSColor.
It depends on your usage but I discourage storing images in a database. They're better off on the filesystem with (perhaps) the paths to the images being stored in the database.
The one case where I can see any gains by having the images stored in the database is if you want one filesystem unit that you can move around that moves everything around. Though with the iPhone this isn't a likely use-case.