Building AWS Infra with Terraform 1

Part 1: Infrastructure as Code using Terraform

This is a tutorial that teachers how to build an AWS infrastructure using Terraform, starting with building a core network infrastructure as the infrastructure concern (or layer) and then building a web application as the web application concern.

The Concerns as Modules

We’ll implement this using a modular approach with Terraform modules, with each concern as a module.

The infrastructure will have two sub-modules: network and security, while the web application will have two sub-modules: application and database.

Infrastructure Concern

The core infrastructure will use the following AWS resources:

• Virtual Private Cloud (VPC)
• VPC Internet Gateway
• VPC Subnets (private and public)
• VPC Routing Tables
• VPC Security Groups

At the end of this section, you should have a basic grasp on how to create a infrastructure that supports a secure private network and a public facing network.

Web Application Concern

The web application layer will use the following AWS resources:

• Elastic Compute Cloud (EC2) instance for a web service
• Relational Database Service (RDS) for the DB used by the web service
• RDS Database Subnet Groups

This application will serve to demonstrate the infrastructure, but also gain knowledge on how to create basic instance with database support. Will will also learn how to import the existing infrastructure output for use with your projects.

The Tool Requirements

You’ll need essentially two tools:

• AWS CLI: https://aws.amazon.com/cli/
• Terraform: https://www.terraform.io/

Note: The instructions oriented toward using the GNU Bash Shell.

Installing AWS CLI

If you have a Python Environment, such as one installed with pyenv, you can simply run:

pip install awscli

Further information on installation on AWS:

• AWS CLI Installation Guide: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html

I wrote a previous article on creating a Python environment using pyenv:

Installing Pythons with PyEnv

Configuring AWS CLI

First follow the installation and configuration instructions from AWS docs:

• AWS CLI Configuration Guide: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html

Note: If not obvious, you need to have an AWS account with permissions to create resources, and copy of AWS access key id and AWS secret access key.

I recommend using AWS profiles, as it is easy to switch between professional work and learning home accounts.

aws configure --profile learning

After, you will have a ~/.aws/credentials file that looks something similar to this, but with valid credentials.

[default] 
aws_access_key_id = REDACTED
aws_secret_access_key = REDACTED
[learning]
aws_access_key_id = REDACTED
aws_secret_access_key = REDACTED

And you will want to have a ~/.aws/config file that looks something similar to this, with the desired region specified:

[default]
region = us-west-2
output = json
[profile learning]
region = us-east-2
output = json

Note: Pay attention to added word of profile in the config, which is different that the credentials file.

Test the profile:

export AWS_DEFAULT_PROFILE=learning
export AWS_PROFILE=learning
aws sts get-caller-identity
aws ec2 describe-instances

Note: Originally, AWS CLI used AWS_DEFAULT_PROFILE, but recent versions of AWS SDK only support AWS_PROFILE. Set both to be safe.

About Terraform

Terraform, for all intents and purposes, is a change configuration tool for configuring cloud resources and other resources that can be managed through a web interface (RESTful API). When configuring cloud resources, the popular term is cloud provisioning.

Terraform purposefully does not configure system resources, as there are popular CAPS (Chef, Ansible, Puppet, or Salt Stack) tools for this already.

On macOS, if you have Homebrew installed, you can install terraform using this:

brew install terraform

On Windows 10, if you have Chocolatey installed, you can install terraform using this in either command shell or PowerShell run with administrative privileges:

choco install terraform

Otherwise, you can download and install Terraform manually:

• https://www.terraform.io/downloads.html

Getting Started

At this point, we should have both terraform and AWS CLI tools installed and available in the path, and we also have AWS environment configured with a profile called learning, and set this to the default.

We are going to create two separate projects areas to support SoC (Separation of Concerns). This allows us to be modular, as well as not blow up our infrastructure while making changes to the web application.

Organizational Structure

To get started, let’s create the following organizational structure (output of tree -F):

.
└── tf-projects/
    ├── infra/
    │   ├── aws.tf
    │   ├── net/
    │   └── sec/
    └── webapp/
        ├── app/
        ├── aws.tf
        └── db/

We can create this in the GNU Bash Shell with the following commands:

mkdir -p ~/tf-projects/{infra/{net,sec},webapp/{app,db}}
touch ~/tf-projects/{infra,webapp}/aws.tf

AWS Provider

We need to use AWS provider we can interact with AWS.

Let’s update our blank aws.tf files:

cat <<-'AWSPROFILE' > ~/tf-projects/infra/aws.tf
variable "region" {}
provider "aws" {
  region     = "${var.region}"
}
AWSPROFILE
cp ~/tf-projects/infra/aws.tf ~/tf-projects/webapp/aws.tf

Terraform can pick up our credentials, but not the region, so we need tell Terraform about our default region:

export TF_VAR_region=$(
  awk -F'= ' '/region/{print $2}' <(
    grep -A1 "\[.*$AWS_PROFILE\]" ~/.aws/config)
)

With that we have our region setup and valid credentials configured, we need to download the Terraform AWS plugin:

pushd ~/tf-projects/infra/ && terraform init && popd
pushd ~/tf-projects/webapp/ && terraform init && popd

AWS Key Pairs

When deploying systems we need to generate AWS Key Pair, which will be used to access the remote system using a private key.

You can use these steps to create a key pair:

KEYPATH=".sekrets"
KEYNAME="deploy-aws"
openssl genrsa -out "$KEYPATH/aws.pem" 4096
openssl rsa -in "$KEYPATH/aws.pem" -pubout > "$KEYPATH/aws.pub"
chmod 400 "$KEYPATH/aws.pem"
aws ec2 import-key-pair \
  --key-name $KEYNAME \
  --public-key-material \
        "$(grep -v PUBLIC $KEYPATH/aws.pub | 
           tr -d '\n')"
cp $KEYPATH/aws.pem $HOME/.ssh/$KEYNAME.pem
cp $KEYPATH/aws.pub $HOME/.ssh/$KEYNAME.pub

Verify you key pair is installed

KEYNAME="deploy-aws"
aws ec2 describe-key-pairs \
  --query 'KeyPairs[*].[KeyName]' \
  --output text | grep $KEYNAME

Afterward, when logging into a system created by Terraform, you can use:

KEYNAME="deploy-aws"
ssh -i ~/.ssh/$KEYNAME.pem $AWS_HOST_IP

Conclusion: Until Next Time

This is the first step to configure and setup a AWS Terraform environment (aws and terraform tools), and follow up articles, I’ll walk through the first concern of AWS core infrastructure and then two articles on the web application concern, one the app and one for the database, both organized into modules that the code can be reused.

Building AWS Infra with Terraform 2

References

This tutorial was inspired by web console version tutorial, and does the equivalent resources using infrastructure as code with Terraform.

• Building Your First Amazon Virtual Private Cloud (VPC) from QwikLabs