Posts

Showing posts with the label python

Deploying Pre-trained Keras Models Using Tensorflow 2 on Amazon SageMaker

How does one go about deploying a model on Amazon SageMaker from Keras, TensorFlow, or TensorFlow Hub without first doing training? There are countless articles and blog posts discussing how to train a machine learning model using TensorFlow or Keras, and then deploy that model right away to Amazon SageMaker.  But what if you're already starting from a SavedModel, or just want to serve up a model trained on plain vanilla ImageNet from within your own AWS account?  You might end up wading through tons of confusing, outdated information that will misguide you, causing you to go down rabbit holes that will make things seem unnecessarily complex.  For instance, you might be inclined to use compatibility libraries to find low-level attributes of modern classes to leverage older deprecated function calls, or to build totally unnecessary infrastructure along the side that makes a Docker container for your model, which will be inevitably broken because you don't know how to invok...

Extensible Database Tester in Python, with Permutations

Here is some Python code that allows you to generate all combinations of options by choosing one item from an arbitrary number of different lists.  Not only that, but each option is actually a data structure that also includes lots of metadata about the ramifications of the option, in addition to the option value itself. Let's say you're looking to run performance testing on different types of tables and underlying data arrangements in Amazon Redshift Spectrum.  You can devise a template SQL query that can be configurable in multiple ways.  For instance, you could choose between multiple tables or views from which to query data (say, if comparing Redshift to Redshift Spectrum performance), and you also want to see the performance of count(1) vs. count(*) to see if one uses more data or works quicker on Redshift vs. Spectrum.  Thus, you already have two lists of options, and need to come up with the four permutations of these options. The Groundwork Basically...

Validating Pre-Made Tensorflow Estimators Mid-Stream

In Francois Chollet’s book Deep Learning with Python , he stresses the importance of utilizing a separate validation set of data while training a machine learning model in order to test periodically (say after every epoch) that the accuracy on something else besides strictly the training data (e.g. this validation set) is in fact improving. Machine learning models are subject to learn relationships that have nothing to do with the problem at hand.   For instance, a model tasked with trying to determine which way a military tank is facing might end up making assumptions based on whether it is day or night.   This is often a result of trying to eke out the model’s maximum performance, say by optimizing for the smallest value of a loss function.   However, what ends up happening is that the model overfits on the training data, which means it loses its generalization — its ability to predict the correct outcome of new samples or examples that we as humans would inten...

Making a new Transaction Family on Hyperledger Sawtooth with Docker

(Important Note: Conversational capabilities have been disabled on this article.  If you need to ask a question or comment, please find me elsewhere.) Are you excited to see what all the buzz is about regarding Hyperledger Sawtooth ?  Are you a fan of using Docker containers to conveniently test Sawtooth in an isolated and easily-deployable environment?  Then it probably won't be long before you want to run your own smart contract code beyond the basic examples provided to you. At the time of this writing, the Docker containers for Sawtooth only provide examples of smart contracts written in Python 3 -- no JavaScript nor Go containers are available from Sawtooth Lake.  As far as transaction processors , which are the entities that actually run the smart contract code, the only ones available as Docker containers from Sawtooth Lake are: sawtooth-settings-tp sawtooth-intkey-tp-python sawtooth-xo-tp-python "Settings" is required in any Sawtooth dep...