DevOps Concepts: Bake vs Fry 2

Part 2: Configuration Methods during the Cloud Age

In the last article we introduced the concept of system configuration using baking and frying, and we covered ways this was applied during the Iron Age.

We covered:

Now we’ll show how this concept is applied in the Cloud Age.

Previous Article

The Cloud Age

With the arrival of the Cloud Age, the entire data center was virtualized, not just the virtual machine instances, but storage and network infrastructure as well.

These new virtual resources are accessible through a web interface from the IaaS platforms, like AWS (2006), Google Cloud (2008), and Azure (2010), or OpenStack. Suddenly now with a few mouse clicks or a small script, you can create the whole infrastructure of your organization.

As greater number of applications are released as web services in the cloud, whether service (SaaS) or platform as a service (PaaS), focus shifted toward maintaining fleets of instances and deploying web applications to those instances.

Provisioning Instances (Frying)

Configuring systems with virtualization on your own hardware or cloud virtualization are virtually the same. The one exception is that you can no longer use network booting for early stage provisioning. Instead, with a new virtual machine instances, you select a pre-baked image, supplied by the cloud provider, such as Debian, RedHat, or Ubuntu.

With your operating system image selected, you configure an SSH key (so that you can access the systems later), and optionally use a startup script solution, such as cloud-init, to configure the system at launch time. From here, you can provision the whole system at launch or hand off to a change configuration solution, like Puppet (2005), Chef (2009), or more recent arrivals Salt Stack (2011) and Ansible (2012).

Frying Components on Cloud Instances

The above illustration shows the approaches to configuring systems:

Making Images (Baking)

Instead of the selection of images from the cloud provider, you can decide to bake your own images with tools like:

Building images with such automation is far more efficient than before, especially with Packer, where you can use previously popular tools like Debian Preseed and Kickstart for building the system, and then use Packer Provisioners to run change configuration tools like Chef, Ansible, Puppet, and Salt Stack to finish baking the image.

Despite efficiency in building images, there’s still challenges the Golden Image problem where curating a library of images of various combinations becomes a maintenance nightmare.

To Bake or Not to Bake?

Despite higher costs to build and maintain baked images, the costs for deployment and remediation (repairing systems) goes down dramatically when using images. The systems are disposed and replaced when they need to be upgraded or repaired, a pattern called immutable infrastructure.

Additionally with baked images, you can have a single image encapsulate the applications, dependencies, and configuration into a single deployable artifact that can be versioned.

In contrast with frying with incremental changes on a mutable system, you have multiple artifacts that need to be rigorously maintained, synchronized, and tested. Systems are upgraded and repaired incrementally, sometimes without automation. This leads to higher complexity for deploying systems.

The illustration shows:

Self Healing Managed Instances

Wouldn’t it be great if we don’t have to configure the systems each time and every time? Maybe we just set them up once, and afterward, they managed themselves? Or if they fail or crash, just replace themselves?

This is actually possible, for lack of a better term, cloud managed groups:

You declare in a configuration template some attributes, such as how many identical systems in the fleet, health checks used to determine if the instance is functional, and rules for scaling up or down the number of instances depending on usage.

Cloud Managed Groups

The above illustration is a generalized overview of the process. When you apply your configuration template, the cloud provider will launch a fleet of instances and park them all behind a load balancer.

The cloud provider will use what can be described as a reconciliation loop (to borrow a term from Kubernetes), where it continually monitors the health of your fleet of instances, leveraging from health check mechanism from the load balancer and used in conjunction with monitoring and alert system.

As apart of fleet management process, any failed instances will be removed from the load balancer set, and replaced with a new instances. As part of dynamic scaling process, new instances will be added or removed, as needed when the load on the existing instances increases or decreases.

When launching a new instances that is part of a fleet, they either need to be fully baked for their role, or need to be provisioned (fried) at launch time for their role (with optional hand-off to change configuration solution). The cloud provider will have some mechanism to provision the system at launch:

Cloud Provisioning (Frying)

We discussed configuring systems through baking or frying, but what about configuring the cloud itself?

Change Configuration platforms are good at configuring system resources: files, packages, services, users, groups, etc. What about cloud resources: virtual machine instances, virtual networks, virtual storage, auto-scale groups?

Configuring cloud resources is also sometimes called cloud provisioning, and then combined with configuring operating systems, you can have end-to-end infrastructure as code.

In my experience, these are the three most popular platforms for cloud provisioning are:

Below is more comprehensive list of other solutions that is sorted in the order of their release:

Service Discovery (Frying)

The previous article introduced service discovery for cluster membership or external services during the Iron Age. The application itself can query a discovery services to find other members in its set as well as external services, such as a database or message queue.

General Discovery Solutions (any cloud)

I am repeating the list from the last article. These can be implemented on any cloud platform.

AWS Discovery Solutions

These are two built for only AWS.

Cloud Metadata (Frying)

As an alternative to using service discovery, you can use cloud metadata.

The Cloud Age adds a new dimension in that services will include cloud resources exposed through web interface. Each cloud provider provides features that support metadata that can be used as a form of service discovery.

Google Cloud Metadata

Azure Metadata

AWS Metadata

Disposability Patterns

With all these technologies and methods, there comes the question to how to go about managing systems, from single systems to fleets of systems?

Before we dive into this, I wanted to take a small moment to talk about two related metaphors: pets vs. cattle and snowflake servers vs. pheonix servers.

Pets vs. Cattle

In the pets vs cattle centers around instances that are managed group (cattle) or managed as single instances (pets). When managed as a group, the instances are replaceable, but when managed individually, they are irreplaceable.

A while back I wrote an earlier article that expanded on this topic:

Snowflake Server vs. Pheonix Server

In snowflake server vs pheonix server concept centers are around configuration drift and disposibility of instances.

Instances become snowflakes as their configuration will gradually diverge or drifts form the desired state. In order to remedy this, you need to continually monitor the system and incrementally configure it back to the desired state.

As an alternative, you can dispose of the snowflake server completely, and replace it with a newly provisioned instance. The instance rises from the ashes of disposed instance, hence the term pheonix server.

A pheonix server can be created using frying (provisioning or change configuration) or baking (imaging). The later is also called an immutable server.

Combining the Concepts

The illustration below shows the two concepts combined and illustrates whether baking or frying are applicable.

These are the categories of tools you might use to support such systems:

Types of Tools include the following:


In the Cloud Age, we saw further virtualization of all resources besides instances of virtual servers, and new opportunities for automation, using infrastructure as code to not only configure components in the operating system, but for components in the cloud, such as networks and storage.

In the next article, I will discuss containers and orchestration and scheduling solutions for containers, similar to cloud managed groups, and show how baking and frying relate to these topics.


Immutable Infrastructure

Infrastructure as Code (IaC)

Pheonix vs. Snowflake

Google Cloud Topics

AWS Topics

Linux NinjaPants Automation Engineering Mutant — exploring DevOps, o11y, k8s, progressive deployment (ci/cd), cloud native infra, infra as code