Is there any good way to save and then run a tensorflow model without utilizing Tensorflow Serving. My model is pretty simple and requires only sequential serving for inference on my PC. So I think there should be a better way to avoid using too rich and complicated Tensorflow Serving system.
What I am doing now is saving all weights and the session after training; and, for inference, I define the graph again, restore session and load weights. Is that a reasonable way to do?
You could use Flask (http://flask.pocoo.org/) . In one of the routes you define you query your model. Would this help you?
Related
I was wondering if in a Federated Learning approach I need to split the local dataset in a number of batches equal to the number of communication rounds.
Otherwise I need to update locally on the whole local dataset each round.
Building a federated learning model
it depends on what you want to do. Federated learning is not a fixed method but a flexible approach that changes from solution to another and architecture to another i will try to make it clear by giving examples.
In Google keyboard for example they collect data in real-time so in each round there will be new data so in this case they are probably using the whole data for the update.
In another use-case you may have a huge local dataset and it takes ages to retrain the model locally so in this case you can train a subset in each round to reduce computation power and time needed to retrain the model.
Finally Federated leaning still have lot of challenges use it when it is really an obligation otherwise just adopt the normal centralized approach to train your model :)
I have a generic question around the best practises on usage of Neptune DB as a network database and its ability to scale up for complex computing. I want to develop a user recommendation system where incoming users on the platform are prompted other users they can likely follow in order to grow the network.
For implementing a simple technique like Triadic Closure, should I use gremlin queries on the Network DB(AWS Neptune in my case) for generating the recommendations? I believe in this case I would have to create python scripts that parallelise queries for multiple nodes and generate recommendation for each node at scale.
OR is it a more common practise to store the network data in the form of nodes, edges and their properties into a relational database, and then perform computations on the same by running SQL queries to load the network data into python, and then using packages like NetworkX on top of that. In this case I won't have to worry about batch computations since a relational database like Redshift would take care of it. However I would be writing python logics to implement techniques such as triadic closure.
Additionallly in the future I may want to use more complex graph computational techniques like graph clustering, partitioning, calculation of different kinds of centralities. Are all/any of these possible within the framework of Neptune+Gremlin.
With the above context below are the questions I am seeking answers for:
Whats is the commonly used tech stack by a data science team working with graph data to build solutions such as user recommendations? By data-science tech stack I mean technologies that help query, analyse, visualise, compute and serve.
Can Neptune + Gremlin replace python packages such as NetworkX for network analysis and centrality measurement?
Is Neptune DB ideal only as a data store OR can it also support complex network analysis and recommendation serving?
Any insight/resources on this would be really helpful!
It is definitely possible to do triadic closure in Gremlin. I have also seen data scientists use both NetworkX and Gremlin together by running the gremlin-python client in a Jupyter Notebook. As this question is quite specific to Amazon Neptune you may want to post to the Neptune support forum at [1]. There are also some useful Gremlin Recipes at [2]
If you post to the support forum I am sure someone will respond.
[1] https://forums.aws.amazon.com/forum.jspa?forumID=253&start=0
[2] http://tinkerpop.apache.org/docs/current/recipes/
My company does online consumer behavior analysis and we do realtime predictions using the data we collected from various websites (with our java script embedded).
We have been using AWS ML for real time prediction but now we are experimenting with AWS SageMaker it occurred to us that the realtime data processing is a problem compared to AWS ML. For example we have some string variables that AWS ML can convert to numerics and use them for real time prediction in AWS ML automatically. But it does not look like SageMaker can do it.
Does anyone have any experience with real time data processing and prediction in AWS SageMaker?
It sounds like you're only familiar with the training component of SageMaker. SageMaker has several different components:
Jupyter Notebooks
Labeling
Training
Inference
You're most likely dealing with #3 and #4. There are a few ways to work with SageMaker here. You can use one of the built-in algorithms which provide both training and inference containers that can be launched on SageMaker. To use these you can work entirely from the console and just point at your data in S3, similar to AWS ML. If you're not using the built-in algos then you can use the sagemaker-python-sdk to create both training and prediction containers if you're using a common framework like tensorflow, mxnet, pytorch, or others. Finally, if you're using a super custom algorithm (which you weren't if you're porting from AWS ML) then you can bring your own docker container for training and for inference.
To create an inference endpoint you can go to the console under the inference section and click around to build your endpoint. See the gif here for an example:
Beyond that if you want to use code to invoke the endpoint in real time you can use any of the AWS SDKs, I'll demonstrate with the python SDK boto3 here:
import boto3
sagemaker = boto3.client("runtime.sagemaker")
response = sagemaker.invoke_endpoint(EndpointName="herpderp", Body="some content")
In this code if you needed to convert the incoming string values to numerical values then you could easily do that with the code.
Yes it can! you have to create a Pipeline (Preprocess + model + Postprocess) and deploy it as endpoint for real time inference. you can double check the inference example in sagemaker github site. it's using sagemaker-python-sdk to train and deploy.
1: This is for small data sklearn model.
https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python-sdk/scikit_learn_inference_pipeline
2: it also support big data (spark ML Pipeline serving container), you can also find the example in its official github.
AWS SageMaker is a robust machine learning service in AWS that manages every major aspect of machine learning implementation, including data preparation, model construction, training and fine-tuning, and deployment.
Preparation
SageMaker uses a range of resources to make it simple to prepare data for machine learning models, even though it comes from many sources or is in a variety of formats.
It's simple to mark data, including video, images, and text, that's automatically processed into usable data, with SageMaker Ground Truth. GroundWork will process and merge this data using auto-segmentation and a suite of tools to create a single data label that can be used in machine learning models. AWS, in conjunction with SageMaker Data Wrangler and SageMaker Processing, reduces a data preparation phase that may take weeks or months to a matter of days, if not hours.
Build
SageMaker Studio Notebooks centralize everything relevant to your machine learning models, allowing them to be conveniently shared along with their associated data. You can choose from a variety of built-in, open-source algorithms to start processing your data with SageMaker JumpStart, or you can build custom parameters for your machine learning model.
Once you've chosen a model, SageMaker starts processing data automatically and offers a simple, easy-to-understand interface for tracking your model's progress and performance.
Training
SageMaker provides a range of tools for training your model from the data you've prepared, including a built-in debugger for detecting possible errors.
Machine Learning
The training job's results are saved in an Amazon S3 bucket, where they can be viewed using other AWS services including AWS Quicksight.
Deployment
It's pointless to have strong machine learning models if they can't be easily deployed to your hosting infrastructure. Fortunately, SageMaker allows deploying machine learning models to your current services and applications as easy as a single click.
SageMaker allows for real-time data processing and prediction after installation. This has far-reaching consequences in a variety of areas, including finance and health. Businesses operating in the stock market, for example, may make real-time financial decisions about stock and make more attractive acquisitions by pinpointing the best time to buy.
Incorporation with Amazon Comprehend, allows for natural language processing, transforming human speech into usable data to train better models, or provide a chatbot to customers through Amazon Lex.
In conclusion…
Machine Learning is no longer a niche technological curiosity; it now plays a critical role in the decision-making processes of thousands of companies around the world. There has never been a better time to start your Machine Learning journey than now, with virtually unlimited frameworks and simple integration into the AWS system.
In this case, you will need to preprocess your data before feeding it into the InvokeEndpoint request body. If you use python, you can use int('your_integer_string') or float('your_float_string') to convert a string to an integer or float. If you use java, you can use Integer.parseInt("yourIntegerString") or Long.parseLong("yourLongString") or Double.parseDouble("yourDoubleString") or Float.parseFloat("yourFloatString").
Hope this helps!
-Han
I wanted to check the source code of the distributed training feature of tensorflow and its overall structure. Worker-PS relations, etc. However I am lost in tensorflow's repository. Can someone guide me through the repository and point the source code I am looking for?
Unfortunately, not all tensorflow code (especially the part related to distributed computation) is open source. To quote Aurélien Géron from Hands-On Machine Learning with Scikit-Learn and TensorFlow:
The TensorFlow whitepaper presents a friendly dynamic placer algorithm that auto-magically distributes operations across all available devices, taking into account things like the measured computation time in previous runs of the graph, estimations of the size of the input and output tensors to each operation, the amount of RAM available in each device, communication delay when transferring data in and out of devices, hints and constraints from the user, and more. Unfortunately, this sophisticated algorithm is internal to Google; it was not released in the open source version of TensorFlow.
But here are the main entry points of TF distributed in the public repo:
Cluster in tensorflow/python/grappler/cluster.py
Server and ClusterSpec in tensorflow/python/training/server_lib.py
worker_service.proto in tensorflow/core/protobuf/worker_service.proto
To dive deep you'll need to enter native C++ code in tensorflow/core/distributed_runtime package, e.g., here's gRPC server implementation.
I'm looking to use WEKA to train and predict from data in MongoDB. Specifically, I intend to use Weka API to analyse data (e.g. build a recommendation engine). But I have no idea how to proceed, because the data in MongoDB is stored in the BSON format, while WEKA uses the ARFF format. I would like to use the WEKA API to read data from MongoDB, analyse it, and provide recommendations to the user in real-time. I can not find a bridge beween WEKA and MongoDB.
Is this even possible or should I try another approach?
Before I begin, I should say that WEKA isn't the best tool for working with Big Data. If you really have Big Data, you will likely want to use Spark and the Hadoop family as they are more suited to analysis.
To answer your question as written, I would advise doing the training manually (i.e. creating a training file using any programmatic tools available to you) and pretraining a model. These models can then be saved and integrated into a program accordingly.
For testing, you can follow the official instructions, but I usually take a bit of a shortcut: I usually preprocess my data into a CSV-like format (as if it was going into an ARFF file) and just prepend a valid ARFF header (the same one as your training file uses). From there, it is very easy to test the instances. In my experience, this greatly simplifies the process of writing code that actually makes novel predictions.