Automate Infrastructure with CloudFormation and Webhooks
Cost savings, improved reliability and testability, faster updates: these are just a few of the benefits when you automate infrastructure creation.
CloudFormation templates and Webhooks are two paths to automation. CloudFormation templates are ideal if you are getting started in a new effort and need to take a greenfield approach. (See our earlier post for details on using CloudFormation to perform a rolling deployment on AWS.) Webhooks are useful if you must incorporate existing solutions for creating infrastructure.
In this post, I'll show you how cloudtamer.io Cloud Rules can help you bring even more automation to both of these approaches.
The Automation Benefits of Cloud Rules
A cloudtamer.io Cloud Rule is a collection of cloud-specific resources (think AWS IAM Policies, AWS Service Catalog Portfolios, AWS CloudFormation templates, Azure Role Definitions) that can be applied to cloud accounts in a managed way. Cloud Rules are hierarchical and inheritable, giving you even more opportunities for automation. If you apply a Cloud Rule at the top of your cloud organizational structure, all the child Projects (i.e., accounts) below will inherit the Cloud Rule. Cloud Rules can also be applied locally to a specific project and users can request exemptions from a Cloud Rule. Cloud Rules can be applied at any time.
Using Cloud Rules with CloudFormation Templates
Let’s take a look at how you can launch infrastructure in an account by attaching a CloudFormation template to a Cloud Rule in cloudtamer.io.
From the Cloud Management tab, select AWS CloudFormation templates and then Create CloudFormation template.
The template used in this tutorial launches a single EC2 instance, and the CloudFormation template does not accept any Parameters. If you have a template that accepts Parameters, ensure they are formatted correctly. Check out the AWS DevOps blog for more information on formatting your CloudFormation Stack Parameters.
Select the appropriate region where this CloudFormation stack will be created, and select the appropriate owner(s) for management of the CloudFormation template in cloudtamer.io.
Once you have successfully created your CloudFormation Template, you can associate this with a Cloud Rule.
Under Cloud Management, navigate to Cloud Rules and click on the + icon.
Give the Cloud Rule a descriptive name, select the appropriate owner(s), and select the AWS CloudFormation template we just created.
After creating this Cloud Rule, we’ll add the Rule at the project level. After applying the Rule, the CloudFormation stack will be launched in the AWS account within this project.
After selecting the project, navigate to the Cloud Management tab and select Add Existing Cloud Rule. A modal will display. Select the Cloud Rule we created previously, and click Confirm selection.
You’ll see the Cloud Rule displayed, along with an indicator of its origin (in this case, local) and the status.
If you federate into the account from the Accounts tab, you’ll see that your CloudFormation template has been created.
That’s all it takes to automate infrastructure via Cloud Rules and CloudFormation templates. For cleanup, you can simply remove the Cloud Rule from the project, and this will delete the CloudFormation stack in the AWS Account.
Using Cloud Rules with Webhooks
So, you’ve seen how to create infrastructure using a greenfield approach. But what if you have existing tools that you’re using in your environment to create your infrastructure? Combining Cloud Rules and Webhooks can address this scenario.
Webhooks allow you to send data to a web endpoint in JSON format so you can act on an AWS account and extend the functionality of cloudtamer.io. This allows you to integrate your existing services and tools with cloudtamer.io to perform actions that are difficult or not available via IAM policies and CloudFormation templates.
In this next example, we’ll create a webhook that will trigger a Jenkins pipeline to create infrastructure in an AWS account. Navigate to the Cloud Management tab, select Webhooks, and click Create Webhook.
Enter a name and description for your webhook. Select the Send Access Keys checkbox; this allows cloudtamer.io to send temporary AWS API keys for the respective AWS Account. For Callout URL enter the URL to invoke your Jenkins job. The URL should follow the format in the screenshot below. Refer to the Jenkins wiki to get webhooks set up in your Jenkins environment.
Once your Callout URL has been set, click Test webhook. If everything is successful, you’ll get a success message.
Click Create Webhook when you're finished testing.
Next, we’ll create a new Cloud Rule from the Cloud Management tab and associate this with our webhook.
Name your Cloud Rule, set the appropriate ownership, and set the Pre-Rule Webhook field with the webhook you created. Your Cloud Rule should look something like this:
After creating your Cloud Rule we can follow the steps from the first tutorial to apply your Cloud Rule at the project level. Navigate to an existing Project, and select the Cloud Management tab. Select Cloud Rules and click Add Existing Cloud Rule. From the drop-down menu select the Cloud Rule created in our previous step and click Confirm Selection.
Navigate to your Jenkins and you should see your pipeline running. In this demo we are using an existing pipeline that handles deleting the default VPC and creating a new VPC in the AWS Account.
That’s all it takes to automate infrastructure with CloudFormation templates, Webhooks, and cloudtamer.io. For additional information on these components – and sample webhook code – customers can check out the Guides in our Support Center.
Cameron is a DevOps Engineer at cloudtamer.io.