Top Tips for Google Cloud Platform – Deployment Manager

April 26, 2019
Alex Simpson

Up until I came on board with Datisan at the end of last year

I had been working as an AWS consultant for a number of years. Now, as a data engineer, I’m working exclusively with Google Cloud Platform (GCP) helping Datisan’s clients to gain insights into their marketing data to ultimately improve customer experience.

Data engineering isn’t new to me, as a few of my previous clients wanted data lakes and ingestion pipelines setup on AWS, however learning about GCP has been a real eye-opener.

The biggest learning curve for me (and the focus of this post) has been how to get started with deployment manager (DM), GCP’s infrastructure as code (IAC) service. Hopefully, with these insights, you and your IT team may be able to avoid some of the pitfalls I’ve fallen into.

First things first, have a look through the GCP documentation so you can get a better handle on the foundational concepts:

Now that you’ve wrapped your head around the official documentation, here are my:

Top Tips for Google Cloud Platform – Deployment Manager

1 – Python over jinja

Deployment manager templates have to be written in one of two languages, jinja or Python, my advice would be to stick with Python. I started with jinja, coming from a YAML centric AWS background I thought it would be easier, but this turned out to be wrong. Python gives you “orders of magnitude” more control over your infrastructure/deployments, with the added ability to import any library you choose, create helper-functions and most importantly, Python has the most significant source of examples on the internet and questions on StackOverflow. If you try to search for jinja detailed examples, you are going to have a tough time finding anything relevant. Python is broadly supported and also easy to get started with; for this reason, it is my pick out of the two deployment manager languages.

2 – Understand YAML’s role in your stack

Any deployment built with DM consists of two types of files, a YAML configuration file and a set of template files (as discussed earlier in either Python or jinja). Google docs refer to the YAML file as your “configuration” and any jinja or Python files as your “templates” so be aware of this when looking through the documentation.

3 – Understand the deployment process
gcloud deployment-manager deployments create {someStackName} --config {someConfigFile.yaml}

The “configuration” (YAML) files mentioned earlier are what you use in combination with the above gcloud command to deploy your DM stack, simply change {someStackName} to you desired stack name, and point {someConfigFile.yaml} towards your configuration file.

gcloud deployment-manager deployments delete {someStackName}

The gcloud cli (Command Line Interface) reads your configuration file and pulls in all the template files referenced in the imports section, it then transfers all these files to the deployment manager service where it runs the code inside the configuration and the templates.

To remove a stack you have just deployed the following command will ensure everything is cleaned up (or error out).

4 – Useful gcloud deployment flags

When deploying a DM stack, it is very handy for DM to rollback the stack if anything throws an error. This removes the need for you to delete a failed deployment and speeds up your turnaround time. Append the following command any time you are creating a stack to enable DM to rollback the changes. I append this every time I create a stack.

 --delete-policy abandon

If you change a component of a stack, manually via console or cli, the stack may fail to delete – leaving orphaned resources. This is because the resource DM is trying to delete, is not the same as when it was deployed. Appending the following command to any delete command will ensure the stack will delete and leave any resources it cannot delete, you will then have to go and cleanup the resources manually.

5 – Where to find the different supported resources

As soon as you branch away from the quickstart guide, you’ll quickly want to deploy infrastructure that means something to you, a customised deployment, with all the bespoke features deserving of a DM stack. First port of call should be The Supported Resource Types list from Google.

These are the resources that are officially supported by GCP. If the resource you are looking for is not in the Supported Resource Types you can look at the list of GCP provider Types:

These are currently in beta and as such can change without warning, but you’ll find they support a much wider variety of resources. Don’t be afraid about the beta tag, Google is really good with their betas, and they usually hit production without too many changes.


6 – The Google samples repository

I started with DM by assembling pieces of other peoples deployments, the easiest way that I found to do this is to have a browse through the Google deployment manager samples repository:

It has lots of really great examples for you to play with. It is also super handy to see working examples of what is possible, and what is less possible. To get started, I suggest you clone a version of the samples repository, find the DM resource you want to deploy (from the links in tip 5) and search the files in the sample repository for your desired resource type. You’ll quickly find a working DM stack containing the resource you want to deploy.

Hopefully, the tips above allow you to get off to a roaring start with your deployment manager stack creation, if you have any questions or want to get in touch with the team at Datisan hit us up here.