Posts

Showing posts with the label programming

How I Stopped Worrying and Learned to Give Myself Any AWS Permissions I Wanted

I'm happy to have some ideas worth blogging about at last!  Here's a really big one I stumbled across if you're in an organization that wants to keep permissions relatively locked down on AWS (in terms of performing actions on resources), but still wants to enable individual users the ability to interact with their own stacks as needed without waiting on admins. The TL/DR Of It Using AWS CloudFormation, you can deploy a stack consisting of a single resource.  This resource is of type AWS::IAM::ManagedPolicy .  Attach all the policy statements you want, then add Users consisting of any IAM users on the AWS account.  Voila, instant permissions to do whatever. More About the Process Let's say you need to tweak various aspects of your AWS setup.  Maybe you have a DynamoDB entry that needs to be manually tweaked, a Lambda function where the concurrency needs to be adjusted, an SQS queue where the maximum age of a message needs to be adjusted, or files sitting around...

Unit Test Database Mocking in Golang is Killing Me!

For some reason, writing a particular set of unit tests in Golang proved to be an extreme hassle.  In this case, I was trying to add a column to a table and then write a unit test to verify this new functionality.   No matter how much I looked around for where to correctly specify how to build this table with the new column, the Cassandra database kept complaining the column didn’t exist.   Imagine how frustrating it is to specify a relational database schema that seems to be ignored by your unit tests. Background Our system consists of a Cassandra database plus a Go backend server.   The Go unit tests require Cassandra to be running locally on a Docker container, but do not seem to actually utilize the existing databases, opting to make its own that is out of reach from anything I can see from TablePlus.   The Go code itself utilizes these objects through some level of indirection by a manager, a harness, and a couple modules dealing in the ac...

Angular Noob: An Observable On An Observable Observing a Promise

With the reusability and extensibility of modern Web components, I do not look back on the days of jQuery with much fondness.  However,  I haven't paid much attention to Angular since Angular 1.  Since its syntax didn't really appeal to me, I opted to learn Polymer instead.  Well now, given a new opportunity, I am diving into a much more modern Angular and TypeScript.  Unfortunately, I am finding that a lot of articles people write on Angular, when you're diving into a well-established code base, are about as dense as reading toward the end of a book on quantum mechanics.  It's English alright, but the jargon is applied thickly.  (And this is coming from someone who has even impressed some of Google's Tensorflow engineers with their machine learning skillz.) The problem at hand is fairly straightforward.  We want to notify something in the UI upon the outcome of a RESTful request we make to an external resource so that it can display useful info...

Validating Pre-Made Tensorflow Estimators Mid-Stream

In Francois Chollet’s book Deep Learning with Python , he stresses the importance of utilizing a separate validation set of data while training a machine learning model in order to test periodically (say after every epoch) that the accuracy on something else besides strictly the training data (e.g. this validation set) is in fact improving. Machine learning models are subject to learn relationships that have nothing to do with the problem at hand.   For instance, a model tasked with trying to determine which way a military tank is facing might end up making assumptions based on whether it is day or night.   This is often a result of trying to eke out the model’s maximum performance, say by optimizing for the smallest value of a loss function.   However, what ends up happening is that the model overfits on the training data, which means it loses its generalization — its ability to predict the correct outcome of new samples or examples that we as humans would inten...