Building AKS with Terraform

Provision an AKS Kubernetes Cluster with Terraform

This article covers provisioning a basic high-availability Kubernetes cluster with AKS (Amazon Kubernetes Service) using Terraform with the provider.

Previously, I covered how to do this using a shell script wrapper around Azure CLI. For this article, all cloud resources (Azure and Kubernetes) will be created with .

NOTE: Medium code blocks have been challenging (nearly impossible) to copy text, so to help readers, I have been moving code snippets to gists.

Requirements

The Tools

These are the baseline tools needed to work with Azure and Kubernetes.

  • Azure CLI tool (): command line tool that interacts with Azure API
  • Kubernetes client tool (): command line tool that interacts with Kubernetes API
  • Terraform (): command tool to provision Azure cloud resources and deploy Kubernetes applications.

Project Setup

The following file structure will be used:

You can create this with the following commands:

For the rest of the article, commands will be executed from the directory.

Provision Azure resources

This process will provision an AKS cluster, where Azure will then create a new resource group and provision cloud resources needed for the AKS cluster.

For a basic HA AKS cluster, two modules will be used: group and aks. These will be fetched from the blog source code repository.

Create the Terraform scripts

Create the file with the following contents:

Create the file with the following contents:

Create the variable definition

Create with the following content:

These are some example values for this project that you should change as appropriate. If you have an existing resource group that you wish to use, change to false, otherwise, Terraform will attempt to create it.

Provision AKS with Terraform

When ready, run the following commands:

NOTE: Currently there is no direct method to create dependencies between modules in Terraform. Thus if you need a resource created before other modules are executed, then you will have to manually orchestrate this with the command-line argument.

Interact with the Kubernetes cluster

In order to interact with the cluster, we need to setup a . Run these commands to populate a configuration with credentials and point environment variable to that file.

After this run the command to see the components installed on a vanilla cluster:

This should show something like the following:

Exploring Kubernetes networking: Kubenet

Run the following command to see the and and their corresponding IP addresses:

This should show something like the following:

This output shows both and privileged running on the Azure VNET created with the cluster, while other are on an overlay networks created by network plugin.

Exploring Kubernetes networking: Routes

The network plugin will need external routes configured. You can see this in action by looking are the Azure virtual network traffic routing with the following command:

This should show something like the following:

Demo: hello-kubernetes

For this demo application, will be used, which will display the names of the pods and nodes.

Create the provider script

For the provider, you need to specify where to access credentials, such as credentials. As we created the cluster with Azure, we can use the provider to fetch the credentials.

Create the file with the following contents:

Create the main script

The main part of the script will deploy two Kubernetes resources: and .

Create the file with the following contents:

Create the variable definition

Create with the following content:

Deploy

When ready, run the following commands to deploy demo and verify the results.

NOTE: Embedding is currently not working. Refer to this link:

This should show something like the following:

You can access the one of the pods using :

Afterward, you should see something like this with :

Cleanup

Kubernetes: hello-kubernetes

If you would like to just delete application, you can do the following:

AKS Cluster

If you just want to delete the Kubernetes cluster and associated resources, e.g. load balancer, managed identity, VMSS, and NSG, then run this command:

Resources

These are resources that I have discovered along the way when using Terraform and AKS.

Blog Source Code

Tutorials

Conclusion

This is a very basic tutorial covering how to get started with Terraform for Azure, AKS, and Kubernetes. This tutorial doesn’t go deep into the intricacies of provisioning AKS, as this will be covered in the source code supplement, should you want to delve further into this topic.

The Takeaways

Main takeaways:

Extra takeaways are exposure to:

Where to go next?

For Azure and AKS, there are more advance scenarios when configure AKS to work with Azure DNS, Azure Container Registry, Azure Key Vault, Azure Blob Storage, and other resources. These require setting up identities or service principals, which are created from Azure Active Directory , which is Kerberos + LDAP under the hood.

On the Kubernetes side, Terraform is becoming popular for deploying applications and other configurations. For advance scenarios where you need to use templates, Terraform is not very robust in this area.

As an alternative you can use Helm charts or orchestrate charts with Helmfile. These can be used within Terraform using Helm provider and Helmfile providers.

Thanks for following along.

Linux NinjaPants Automation Engineering Mutant — exploring DevOps, o11y, k8s, progressive deployment (ci/cd), cloud native infra, infra as code

Linux NinjaPants Automation Engineering Mutant — exploring DevOps, o11y, k8s, progressive deployment (ci/cd), cloud native infra, infra as code