I have been steadfast developing a small project to explore different technologies in areas of Infrastructure As Code, classic Desired State change configuration tools, and later Immutable Production.
In order to capture a large sampling of use cases that one finds in infrastructure automation, I needed a service that was a cluster (not a single system) and had dependencies on other infrastructural pieces. A great choice for this would be an ElasticSearch cluster that is purposefully dependent on a local Java repository to make installs faster. This forces one to consider these common use cases:
- order of systems provisioned (out of order, installs fail)
- configuring configuration state (list of private IP addresses) based on infrastructure state (are nodes running).
Currently, I am using Chef for configuring the service, and for the infrastructure on either AWS or Google Cloud, I will be configure this separately using command-line tools, Terraform, or Ansible. Down the road, I will move this Kubernetes specifically, but in the mean time, there’s a lot to learn.
In a small exercise with a colleague that is interested in developing mind maps, I developed this to map out my current activities:
The colors represent areas I have actively completed, while grey bubbles are ones I have not touched yet, and may not touch, depending on what is hot in the market, what is practical, or just interesting.
This is just part one of a series to increase capability and add more value to my future organization (or clients if I am consulting). This is a long term draft plan of the layers.
* Simple Cluster (ElasticSearch)
* Infrastructure: Terraform, Ansible
* Change Config: Chef
* Containerization: Docker
* HA Cluster (autoscale and auto-heal): ASG, MIG
* Secure Cluster (adv. VPC topologies)
* Orchestration: Kubernetes
This is a long term plan, each layer is cumulative, where one can grow in sampling different technologies, or down the stack to areas in increasing complexity. The emphasis is on use cases, on real world requirements that I discover by conferring with colleagues or businesses.
This is only one path of many: implementing and deploying a cluster. There’s a ton of other paths and technologies: queuing, streaming, persistence (sql db, memory db, key/value db, graph db), caching, and the whole classic monolith vs. request-based microservice (reverse-proxy routing) vs. async event microservice (kafka), scaling horizontally/vertically, lambda, continuous integration/delivery pipeline, testing, etc. The list is endless.
Deploying a cluster cross cuts on the requirements of implementing some of these solutions.