Terraform: ExternalDNS on AKS

Deploy ExternalDNS (AzureDNS) with Terraform on AKS

External DNS is a Kubernetes addon that will automate updating DNS records when you deploy a Kubernetes application using an external load balancer ( or ).

This article demonstrates how to set all of this up using Terraform (). In a previous article, I covered how to set this up with Azure CLI () and Helmfile ().

What’s covered…

In this article, the following components will be created:

Requirements

These are the required and optional tools for this tutorial.

The Tools

These are the baseline tools needed to work with Azure and Kubernetes.

  • Azure CLI tool (): command line tool that interacts with Azure API
  • Kubernetes client tool (): command line tool that interacts with Kubernetes API
  • Terraform (): command tool to provision Azure cloud resources and deploy Kubernetes applications.

Optional Tools

Some optional tools that may be useful are:

  • Helm () to interact with deployed helm charts
  • POSIX shell () such as GNU Bash () or Zsh (): these scripts in this guide were tested using either of these shells on macOS and Ubuntu Linux.

Project Setup

The following file structure will be used:

You can create this with the following commands in either bash or zsh:

For the rest of the article, commands will be executed from the directory.

Provision Azure resources

This process will provision an AKS cluster, where Azure will then create a new resource group and provision cloud resources needed for the AKS cluster.

Create the Terraform scripts

Create the file with the following contents:

Create the file with the following contents:

Create the file with the following contents:

Create the variable definition

Create with the following content:

Provision Azure Resources

When ready, run the following commands:

Verify the AKS cluster

Run these commands to access the Kubernetes cluster and verify the components:

The results should look something like this following:

Verify Azure DNS zone

Run these commands to verify the zone is created and some records exist:

The results should look something like this following:

Verify role bindings

We can verify the role bindings to demonstrate that AKS nodes have access to the Azure DNS zone. This is useful for troubleshooting or to satiate curiosity:

The results should look something like this following:

Deploy Kubernetes Components

This script will leverage from existing infrastructure. If we wish to keep these separate, we can use alternatively, data sources to fetch information about the cluster we deployed.

Create the Terraform scripts

The module used to deploy AKS will provide the credentials needed to deploy things on Kubernetes. These will be passed to the provider, where will using to create the helm values and that will be passed to the Helm chart.

Create the file with the following contents:

Create the file with the following contents:

When ready, run the following commands:

Verify Deployment

You can verify the components were install with this command:

The results should look something like this following:

Verify configuration

As a sanity check, you can check the External DNS configuration saved in with the following command:

This should look something like the following:

NOTE: In case it is not obvious, and have been obfuscated.

Demo: hello-kubernetes

For this demo application, will be used, which will display the names of the pods and nodes.

This code will not leverage from earlier code that handles the infrastructure layer and the platform layer. In order to implement separation of concerns, the application layer will have applications in their own areas. These will reference the AKS cluster using a data source.

Create the provider script

Create the file with the following contents:

Create the main script

Create the file with the following contents:

Create the variable definitions

Create :

An external load balancer and a public IP address will be used with this demo. If the Azure DNS zone is public, then you will be able to view it with your DNS name, for example, if the domain was set to .

Deploy

Verify Deployment

This should look something like the following:

Verfy hello records

You can check to see if hello was added with the following command:

This should now show something like the following:

To verify a specific record entries, this can be done with a JMESPath query:

This should show something like the following:

Verify Hello Kubernetes works

After a few moments, you can check the results (substituting for your domain).

NOTE: If a private domain is being used, such as , then this obviously will not resolve. You will need to make an entry into local , run a local DNS server (such as dnsmasq), or remotely access the network through VPN or other means.

Cleanup Application

If you wish to remove only to remove the application and still use the Kubernetes cluster, you can run the following:

Cleanup the Project

If you wish to destroy everything, which is the AKS cluster and Azure DNS zone, you can remove everything with the following command:

Resources

Blog Source Code

Related Stories

  • AKS with External DNS: covers creating resources with Azure CLI and Helmfile. Access to Azure DNS is secured using kubelet identity, the user assigned managed identity that is created when provisioning AKS.
  • AKS with AAD Pod Idenity: access to Azure is secured using pod identity so that only add-ons External DNS and Cert Manager have access to Azure DNS. This using Azure CLI and Helmfile to create the Kubernetes cluster with Azure CNI, Calico for network polices and the ingress-nginx controller.

Conclusion

The biggest takeaway for Terraform is how create things with both the cloud provider and Kubernetes resources.

On a pure Azure side of things, this is an introduction to have to managed identities, specifically using kubelet identity, which will allow all pods in the cluster to access a particular. For more granular control (for implementing PoLP), you may want to explore AAD Pod Identity, to allow only specific pod or pods to access a resource.

For Kubernetes, the take away is how to automate creating DNS records as part of deploying applications. This is especially useful in test or stage environments were infrastructure will churn more frequently.

I hope this is useful in any of the above mentioned takeaways as well as a few listed below that you may have absorbed from the article::

  • using the provider to create Kubernetes objects like and
  • using the provider provider to deploy a public Helm charts.
  • using to dynamically compose helm values to the Helm chart
  • fetching credentials either either the module or data source required for and providers.
  • organizing, creating and using Terraform modules
  • applying separate concerns with infrastructure and platform as one concern and the application as another concern.

Thank you for reading.

Linux NinjaPants Automation Engineering Mutant — exploring DevOps, o11y, k8s, progressive deployment (ci/cd), cloud native infra, infra as code