Deploying code to Amazon OpsWorks using Codeship

Here, at Novo IT, we love using Amazon OpsWorks for deploying our internal projects. With OpsWorks, we can easily segregate our development environments in Stacks and control how each project gets built via Chef recipes. OpsWorks binds directly with your code repository of choice. When you initiate a new build, it will pull in the latest changes and build them for you.

One task, that is not immediately obvious how to solve, is triggering an OpsWorks build remotely from the command line, or from a build server. This article will explain how we do exactly this, using the excellent Codeship Continuous Integration service.

Setting up a new IAM user

We will make use of the AWS Identity and Access Management (IAM) feature to create a new user for our deployments. This is preferable to using, say your root account, as the new account will only have just enough permissions to carry out OpsWorks deployments.

First, we go to IAM: IAM Icon, click on Groups and then Create New Group. This brings up the following screen:

IAM Create Group

This group will provide all its members with access to OpsWorks. A name similar to "opsworks-users" makes sense. Once you've picked a name, click on Continue and we are presented with a Policy selection screen. At this point, we can select from the list of available policies or create a fully custom policy. In our case, the existing "AWS OpsWorks Full Access" policy is just what we need, so let's select it:

IAM Group Policy

Now that we have an IAM group in place, let's create a user we will use for deployment. On the main IAM screen, click on Users and Create New Users, this brings up the following screen:

IAM Create User

Choose a sensible name and generate your new user. It is very important to download the security credentials for your new user. We will need these for deployment, namely the Access Key ID and the Secret Access Key.

Finally, add the new user to our opsworks-users group, which will give the account access to our OpsWorks deployments. This can be done by first clicking on the user, then going to the Groups tab and clicking on the Add User to Groups button.

Gather information from your AWS account

There are some additional details we will need before we can get to deployment.

First we need to record the Stack ID for the OpsWorks Stack we will be deploying to. First we need to go to our Stack and click on the Stack Settings button:

Stack Settings button

On the next screen, the Stack ID will be shown under the label OpsWorks ID, let's record it:

Stack ID

After that we need the Application ID for the application we will be deploying. This can be found on the OpsWorks Application page:

Application ID

Finally, we go to the OpsWorks Instance that we will be deploying to and record it's OpsWorks ID:

Instance ID

At this point we should have our:

  • AWS Access Key
  • Secret Access Key
  • OpsWorks Stack ID
  • OpsWorks Application ID
  • OpsWorks Instance ID

Deploying from the command line

Before we jump into deploying from Codeship, let's try to trigger a remote OpsWorks deployment locally from our command line.

First thing we need to do is install the AWS Command Line Interface As the page mentions, we just need to run the following:

pip install awscli

Once the command line interface is installed, we need to initialise the AWS_ACCESS_KEY_ID environment variable with our AWS Access Key, and the AWS_SECRET_ACCESS_KEY variable with our Secret Access Key. In Linux we do as follows:

export AWS_ACCESS_KEY_ID=[Our AWS Access Key]
export AWS_SECRET_ACCESS_KEY=[Our Secret Access Key]

This will make sure that once we run AWS CLI commands, the tool will be able to log us into our AWS account automatically. Finally, we can run our deployment as follows:

aws --region='us-east-1' opsworks create-deployment --stack-id='[Our Stack ID]' --app-id='[Our Application ID]' --instance-ids='["[Our Instance ID]"]' --command='{"Name": "deploy"}'

If we go to our stack's deployment section after running this command, we should see a new build has just started.

Deploying from Codeship

Once we get this far, deploying from Codeship is surprisingly easy. First we need to go to the Environment section of our Codeship Project and provide our two environment variables there:

AWS_ACCESS_KEY_ID=[Our AWS Access Key]
AWS_SECRET_ACCESS_KEY=[Our Secret Access Key]

After that we go to the Deployment tab, create a new script deployment method and add the following two lines to it:

pip install awscli
aws --region='us-east-1' opsworks create-deployment --stack-id='[Our Stack ID]' --app-id='[Our Application ID]' --instance-ids='["[Our Instance ID]"]' --command='{"Name": "deploy"}'

Codeship does not have the AWS CLI installed by default, so the first line will install it on our build server (it only takes a few seconds). The second line will trigger a new deployment on our OpsWorks Stack.

I hope this article clarifies some details on Codeship + OpsWorks deployment process. If you have any feedback or clarifications, please email me at edgars@novoit.eu

Many Thanks,

Edgars