we would like to make sure that the MLFLOW experiment management platform fits our needs and workflow.
We work with image processing CNNs like Yolo, UNET, and RetinaNet based on an NVIDIA TAO framework.
What we actually need is a tool that concentrates on one place (in a nice and representative way comfortable for comparison) at least the three following things for each experiment:
a- chosen by user typical meta parameters that were used to train a network (such as batches, subdivisions, max batches, etc)
b- a link to the dataset the network was trained on, located on our cloud storage (such as one-drive, google drive or google cloud) or a list of filenames or a link to a file storage cloud or online drive suggested by MLFLOW service if there is such a thing.
c- a result of running the trained network - the number of detected objects
Thus the question is:
Does the MLFLOW fit our needs?
If not ill be glad if anyone could suggest a relevant alternative.
Thank you
I use Comet.ml and it addresses all 3 of your points.
With Comet, parameter tracking is as easy as calling the experiment.log_parameter function. You can also use the diff tool to compare two experiments by their hyper parameters and even group experiments by hyper-paremeters!
Comet has the concept of artifacts. You can upload your dataset as an artifact and version it. You can also have remote artifacts!
Comet has a feature call the Image Panel. This allows users to visualize their model performance on the data across different experiment runs. For the object detection use case, use experiment.log_image to log your images in which you have you drawn your model predicted bounding box on! You will then see in the Image Panel different experiments and how they each one of them are drawing their predictions side by side
Related
i'm using anylogic in an offline environment that cannot be connected to the internet.
in my laptop i can use the Anylogic GIS maps placing agents and using it to calculate distances etc.
is there a way to download a specific (or not specific ) gis map and then load it into Anylogic ?
and also - will it have functionality ?
Yes. Run the model from a non-offline environment and make it us all GIS features that you need, i.e. display all areas, zooms...
The model will cache the data locally in a subfolder.
If you copy that into your offline model's folder, the model will work as if it would be online.
Obvious caveat: you must make sure to force-cache everything you need from the GIS map when online
I was tasked to find the best way to create a facial recognition feature on an app with machine learning. This feature will be used to clock employees into the app. The feature will support...
multiple users per device.
continuous training (so when the mlmodel recognizes someone, it will send new images to the model on the back-end and train the model with the new recently taken images)
updates new classes (when a new user comes along and wants to use the app, the app will take pictures of them, send those images to the model training program on the back-end which will train the mlmodel to recognize the new user)
sends the newly updated models to other devices in the same store so they will recognize the employees as well
What I've tried.
I've tinkered with on-device training and Knn. But from what I understand on-device training will not work for this, because the on-device training models can only have up to 10 classes and knn isn't giving very accurate results...at all
Manual training and re-training with the createML. This is when I...
train a model with createML on my mac
download the model to the app with URLSession
add a new user with the app or take updated pictures of old users
send the images of the new user/updated old user to createML on my mac
create a whole new model with all the images I've ever taken of all the users
repeat steps 2-5 forever
This works just fine but is unbelievably expensive, time-consuming, and unfeasible for the number of eventual users the app will have, to do over and over again.
I'm still very new to machine learning and I feel like I'm going about this the wrong way. I would just like to see if anyone knows of a better/more efficient method of continuous learning so the model will remember what it's learned previously and I can just add new classes or images to it with createML... or if someone could point me in the right direction.
Take a look at Turi Create -- also from Apple: https://github.com/apple/turicreate
It can do everything Create ML does, but is in python and programmable, so you could automate the whole process on your backend. If you know how to do it in CreateML, you will find Turi Create easy to pick up.
To have an accurate result you should look into more powerful machine learning models. Here is an example of a really powerful face recognition model: https://github.com/davidsandberg/facenet.
Now, the next question becomes how would you integrate your app with this new model. This is really up to you but I would recommend you to checkout a few backend alternatives like leveraging AWS Services (EC2 compute servers , Sagemaker, API Gateway, etc) to run and coordinate the inferences. A couple of benefits to doing this is that your app would just be mainly front-end thus making it light and also scalable across different and older IOS platforms and devices. But more importantly, it gives you extra leg-space to do more sophisticated things in the future, where as using CoreML you will be mainly limited to on-device computational power and also the swift-based language.
However, leveraging cloud services would also have other cons attached like the learning curve (learning AWS Services) and potentially privacy issues.
This is just one of the ways, there are many other similar cloud providers like Google , IBM and Azure. Without knowing further your timeline, budget, technical expertise, I can only give you these options and the rest of the choice is yours to make
Can I run a computer vision pipeline with Kubeflow? Is it a good idea, would it run efficiently?
Let's say the steps of the pipeline would need to be image segmentation, some filtering and what not (gpu enabled opencv until now) and maybe a tensorflow serving for a CNN at the end.
Any useful resources?
Thanks,
The kubeflow pipelines would make a good fit with your specific use case.
The idea is that you containerize all the individual steps that you want to have decoupled, something like: 1/ preprocessing, 2/ training, 3/ serving. Each container is designed so it can take the relevant arguments that you want to modify over time to run different versions of the pipeline.
For the preprocessing image I would suggest to start from a GPU image with opencv installed that drops the output on Google Cloud Storage.
For the training you could leverage the google/cloud-sdk:latest image that comes with gcloud command so you just copy over your code and run the ml engine command.
For serving, you could use ml engine to deploy the model and thus start building the image from again google/cloud-sdk:latest, alternatively you could use the TF serving images that are available off the shelf and you only need to specify the bucket where your saved model is stored and the model name see instructions.
This blog posts describes how to build a similar pipeline.
Is there an offline Map layer available for Leaflet? I don't need in detail, but basic geometry would be sufficient.
For sure you can set up your own offline map (raster tiles and/or vector shapes). The difficulty or out-of-the-box availability depends on what kind of information and level of details you want.
GeoJSON:
The easiest is if you need just world countries borders with little detail, just to get the outline. In that case, you can find GeoJSON files on Internet that contain that data for a few hundreds kB (the weight of a single normal big image), e.g. https://github.com/johan/world.geo.json
Then simply use L.geoJson(geoJsonData).addTo(map) to have it shown.
Demo: http://plnkr.co/edit/aB6p7IC2cF7xW41Ju8m7?p=preview
Downloaded tiles:
A more complex (technically and contractually) but still manageable situation is if you want raster tiles (like the OSM website for example). You can download tiles (which are just normal images) from an online server, then use them offline. This is called "tiles scraping" or "tiles bulk downloading".
As for the technical side (you may have to download thousands of individual images, depending on to which zoom level / details you want to use offline), several software are available (have a look at the above OSM Wiki link).
As for the contractual side, many tile servers (including OSM for instance) specifically require in their Terms of Use not to perform bulk downloading (as it generates high demand and uses high bandwidth on their servers). So you should look for a service that accepts this usage.
Render tiles locally:
A perfectly authorized solution (but the most technically complex) is to download the raw OSM data, and to use it through software to generate your map (whether raster tiles or vector shapes).
You can probably find services on Internet that offer to download simplified OSM data (the full database for the entire planet is ~30 GB compressed…) or for a small geographic area (see the above OSM Wiki link).
Regarding the software, the link provided by chrki in the question comment (http://wiki.openstreetmap.org/wiki/Rendering) should get you started.
In particular, you can very well generate raster tiles once, save them and get rid of the rendering software, so you can use those tiles as if you had scraped them.
You can build your own local osm server. It is quite easy using Ubuntu, because there is special packages for it. Installation process is described on this great resource: switch2osm.org
Here is an example using NeDB and modifications to this project/examples https://github.com/tbicr/OfflineMap: https://github.com/KD0NKS/APRSViewJS/blob/master/js/techpire/LayerManager.js. This only caches what the user has already viewed and avoids bulk downloading from osm servers.
There is also a Leaflet Plugin: https://github.com/MazeMap/Leaflet.TileLayer.PouchDBCached
I'd like to have a program call a web service that performs object recognition using a neural network. Is there a web service that can do this?
There is the "Google Prediction API" available as a web service - although there is little information on what models they use.
Google has a service called "Search By Image". It's not just a tineye clone, but rather will give a suggestion for the type of object in the image you upload or provide URL for. It's not clear whether or not they use a neural network although Google researchers have been known to work on machine learning of course (like the recent stacked autoencoder that classified cats, human faces, bodies etc).
http://www.google.com/insidesearch/features/images/searchbyimage.html
Their description from help:
How it works
Google uses computer vision techniques to match your image to other images in the Google Images index and additional image collections. From those matches, we try to generate an accurate "best guess" text description of your image, as well as find other images that have the same content as your search image. Your search results page can show results for that text description as well as related images."
Here's an example where it correctly classified this image as "1 cat"
Check out Strong Steam:
StrongSteam is an AppStore of artificial intelligence and
data mining APIs to let you pull interesting information
out of images, video and audio.