GKE with CertManager

Using cert-manager add-on with GKE

📔 NOTE: This was tested on following below and may not work if versions are significantly different.* Kubernetes API v1.22
* kubectl v1.22
* gcloud 394.0.0
* ExternalDNS v0.12.2
* CertManager v1.9.1
* Dgraph v21.03.2

Requirements

Accounts

  • To follow this article, you will need to have a registered domain and forward DNS queries to the Cloud DNS name servers. Consult documentation from the registrar for your domain. This tutorial will use example.com as an example domain.
  • Google Cloud account a billing account and project setup. Google is offering a 90-day $300 free trial (Aug 2022) that is sufficient for this article. See https://cloud.google.com/free.

Knowledge

  • Basic knowledge of using Google Cloud SDK to configure access, setup a project, and provision resources.
  • Basic shell scripting knowledge including things like setting up environment variables
  • Basic Kubernetes using kubectl command to deploy applications and setup configuration with the KUBECONFIG environment variable.
  • Basic networking knowledge of TCP (Layer 4 and Layer 7), HTTP/1.1 protocol, and exposure to SSL/TLS certificates.
  • Understanding of load balancers and reverse proxies.

Tools (Required)

  • Google Cloud SDK (gcloud command) to interact with Google Cloud
  • Kubernetes client (kubectl command) to interact with Kubernetes
  • Helm (helm command) to install Kubernetes packages
  • helm-diff plugin to see differences about what will be deployed.
  • helmfile (helmfile command) to automate installing many helm charts

Tools (Recommended)

  • POSIX shell (sh) such as GNU Bash (bash) or Zsh (zsh): these scripts in this guide were tested using either of these shells on macOS and Ubuntu Linux.
  • GNU stream-editor (sed) and GNU grep (grep): scripts were tested with these tools and the macOS or BSD equivalents may not work.
  • curl (curl): tool to interact with web services from the command line.
  • jq (jq): a JSON processor tool that can transform and extract objects from JSON, as well as providing colorized JSON output greater readability.

Project Setup

Directory structure

~/projects/ingress-gce
├── dgraph
│ └── helmfile.yaml
├── kube-addons
│ ├── helmfile.yaml
│ └── issuers.yaml
└── ratel
└── helmfile.yaml
mkdir -p ~/projects/ingress-gce/{dgraph,ratel,kube-addons} 
cd ~/projects/ingress-gce
touch {dgraph,kube-addons,ratel}/helmfile.yaml \
kube-addons/issuers.yaml

Environment variables

Google project setup

Provision Cloud Resources

Cloud DNS

ns-cloud-d1.googledomains.com.
ns-cloud-d2.googledomains.com.
ns-cloud-d3.googledomains.com.
ns-cloud-d4.googledomains.com.

Google Kubernetes Engine

⚠️ NOTE: Though this Kubernetes cluster is secure in as far as principal of least privilege with identity principals (Google service accounts) for securing access to cloud resources, the master nodes and worker nodes are  accessible from the public Internet.For further security, such as  production environments, you may  consider using private masters nodes, which require some form of jump host or VPN to access them, as also having worker nodes on a private and public networks (subnets), so that you have further control of what endpoints should be explicitly exposed to the public Internet.Additionally, using a CNI plugin that supports network policies, such as Calico, allows you to restrict traffic from both external and internal networks.

Grant Access to Cloud DNS

Kubernetes Addons: CertManager and ExternalDNS

source env.sh
helmfile --file
kube-addons/helmfile.yaml
source env.sh
helmfile --file
kube-addons/issuers.yaml

Example Application: Dgraph

source env.sh
helmfile --file dgraph/helmfile.yaml
kubectl get ing --namespace dgraph
source env.sh
curl
https://dgraph.${DNS_DOMAIN}/health | jq
curl https://dgraph.${DNS_DOMAIN}/state | jq

Example Application: Ratel

source env.sh
helmfile --file ratel/helmfile.yaml

Fun with Dgraph

Dataset

Schema

Query

Cleanup

Kubernetes Resources

Google cloud resources

Resources

Helmfile

GKE default ingress (ingress-gce)

ExternalDNS

CertManager

Dgraph

Conclusion

The default GKE ingress

  1. Dgraph exposed to local private networks
  2. Each ingress allocates a new IP address

Finally

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Joaquín Menchaca (智裕)

Linux NinjaPants Automation Engineering Mutant — exploring DevOps, o11y, k8s, progressive deployment (ci/cd), cloud native infra, infra as code