Cooking with Chef on Vagrant

Streamlining development using Chef Zero provisioning

Chef, acquired by Progress in 2020, is a widely-used configuration management platform, similar to Puppet and Ansible, designed to automate the management of large-scale infrastructures, whether they’re running on hundreds or thousands of systems. At its core, Chef uses a centralized server to manage and apply configurations to systems running the Chef Client agent.

In this tutorial, we’ll explore how to get started with Chef using Vagrant, a powerful tool for managing virtual machines. Together, Vagrant and Chef simplify the process of launching virtual environments, such as Ubuntu or Rocky Linux, and provisioning (configuring) them using Chef, specifically using the Chef Zero Provisioner (chef_zero).

Chef Zero is a lightweight, in-memory version of the Chef server, providing full support for features like roles, environments, and data bags — without the complexity of setting up a full Chef Server. When combined with Vagrant, it offers a streamlined, user-friendly environment ideal for developing and testing infrastructure-as-code locally before moving to production.

Previous Article

In 2018, I wrote an article on this topic, which may have some useful explanations, but the versions used are no longer supported on current operating systems.

Vagrant Provisioning with Chef

Requirements

These are the tools required for this tutorial.

• Vagrant [vagrant]: virtualization management tool
• Virtualbox [vboxmanage]: virtualization solution that is used by Vagrant
• POSIX-compliant shell [bash or zsh]: tutorial uses commands compatible with one of these shells. On Windows, you can run these shells with either MSYS2 or Cygwin environments. On macOS or Linux (including WSL), one of these shells will be available.

Versions

These versions were used in testing the solution in this tutorial.

* **Host**: 
  * macOS "Monterey" 12.2.1
    * Vagrant 2.4.1
    * Virtualbox 7.0.18r162988
* **Guests**:
  * Ubuntu 22.04.3 "Jammy Jellyfish": `generic/ubuntu2204`
    * Chef Infra Client: 18.5.0
  * Rocky Linux 9.3 "Blue Onyx": `generic/rocky9`
    * Chef Infra Client: 18.5.0

Project Setup

You can create the project directory structure and files with the following commands using bash or zsh:

PROJ_HOME=~/vagrant-chef

# craete directory structure
mkdir -p \
  $PROJ_HOME/cookbooks/hello_web/{attributes,files/default,recipes} \
  $PROJ_HOME/{ubuntu2204,rocky9}/nodes

cd $PROJ_HOME

touch \
 ./cookbooks/hello_web/{attributes,recipes}/default.rb \
 ./cookbooks/hello_web/files/default/index.html \
 ./{ubuntu2204,rocky9}/Vagrantfile

Afterward, the following structure should be created:

Vagrant Configuration

In this tutorial, we’ll use VirtualBox as the default provider, which works consistently on Intel-based machines running Windows, macOS, and Linux. We’ll be configuring recent versions of Ubuntu and Rocky Linux.

Vagrant also supports other providers besides VirtualBox. Although this tutorial focuses on VirtualBox, feel free to explore these alternatives based on your system:

• Windows Professional: Hyper-V using the hyperv provider
• macOS: Hypervisor.framework using the vagrant-qemu plugin
• Linux: KVM using the vagrant-libvirt plugin

Ubuntu 22.04 guest virtual machine

Update the file $HOME/vagrant-chef/ubuntu2204/Vagrantfile with the following:

Vagrant.configure("2") do |config|
  config.vm.box = "generic/ubuntu2204"
  config.vm.network "forwarded_port", guest: 80, host: 8085

  config.vm.provision "chef_zero" do |chef|
    chef.cookbooks_path = "../cookbooks"
    chef.add_recipe "hello_web"
    chef.nodes_path = "nodes"

    chef.log_level = "debug"
    chef.arguments = "--chef-license accept-silent"
  end
end

You can download this image and start the virtual machine with the following command:

pushd ~/vagrant-chef/ubuntu2204 && vagrant up --no-provision && popd

Rocket Linux 9 guest virtual machine

Update the file $HOME/vagrant-chef/rocky9/Vagrantfile with the following:

Vagrant.configure("2") do |config|
  config.vm.box = "generic/rocky9"
  config.vm.network "forwarded_port", guest: 80, host: 8083

  config.vm.provision "chef_zero" do |chef|
    chef.cookbooks_path = "../cookbooks"
    chef.add_recipe "hello_web"
    chef.nodes_path = "nodes"

    #### Override Attributes ####
    chef.json = {
      'hello_web' => {
        'package' => 'httpd',
        'service' => 'httpd',
        'docroot' => '/var/www/html'
      }
    }

    chef.log_level = "debug"
    chef.arguments = "--chef-license accept-silent"
  end
end

You can download this image and start the virtual machine with the following command:

pushd ~/vagrant-chef/rocky9 && vagrant up --no-provision && popd

Chef Configuration

This guide will walk you through setting up all the Infrastructure-as-Code (IaC) scripts used in this project. Chef consists of several key components:

• Recipes: Written in Ruby, recipes define the desired state of a system, specifying how resources should be configured.
• Cookbooks: Collections of recipes, templates, files, and other configuration elements organized to manage system configurations.
• Ohai: A tool that gathers system information (facts), such as a node’s IP address, operating system, and memory. This data can be used by recipes to dynamically adjust configurations.
• Roles: Define a set of recipes or configurations that apply to a group of nodes.
• Attributes: Provide customization based on node-specific information, allowing for flexibility in configurations.
• Run List: A prioritized list of roles and recipes applied to a node to orchestrate its configuration.

These components work together to automate infrastructure management, ensuring consistency and scalability across your systems.

The hello_web cookbook

This is a simple cookbook to install Apache HTTP Server and a default doc-root that has the hello world web page.

Update $HOME/vagrant-chef/cookbooks/hello_web/files/default/index.html with the following:

<html>
  <body>
    <h1>Hello World!</h1>
  </body>
</html>

Update $HOME/vagrant-chef/cookbooks/hello_web/recipes/default.rb with the following:

apt_update 'Update the apt cache daily' do
  frequency 86_400
  action :periodic
end

package node['hello_web']['package']

cookbook_file "#{node['hello_web']['docroot']}/index.html" do
  source 'index.html'
  action :create
end

service node['hello_web']['service'] do
  supports status: true, restart: true, reload: true
  action %i(enable start)
end

Update $HOME/vagrant-chef/cookbooks/hello_web/attributes/default.rb with the following:

default['hello_web']['package'] = 'apache2'
default['hello_web']['service'] = 'apache2'
default['hello_web']['docroot'] = '/var/www/html'

📓 NOTE: Attributes are analogous to arguments that you would pass to a function or properties of an object that can be modified. This is the main entry point into a cookbook. These are the defaults, that can be overridden by injecting run-list details when running chef-client.

Provisioning Automatically

You can now install the Chef client and provision the systems with the commands below.

Ubuntu 22.04

On the Ubuntu 22.04 virtual machine guest, you can install Chef and provision the system with the following command:

pushd ~/vagrant-chef/ubuntu2204 && vagrant provision && popd

Rocky 9

On the Rocky 9 virtual machine guest, you can install Chef and provision the system with the following command:

pushd ~/vagrant-chef/rocky9 && vagrant provision && popd

Testing the Solution

You can use curl with the command line option to include headers in the response for testing purposes.

Ubuntu 22.04

The Ubuntu guest system, by default, does not have a firewall, making it accessible from outside the guest system. This setup maps port 80 to port 8085. To test this, you can use the following command from the host system:

curl --include http://localhost:8085

This should show something like the following below:

Rocky 9

By default, the Rocky Linux guest system has a firewall enabled, restricting access from outside the guest system. To allow traffic, you can open the necessary ports using the following command:

cd ~/vagrant-chef/rocky9

vagrant ssh --command \
 'sudo /usr/bin/firewall-cmd --zone=public --add-port=80/tcp'

This vagrant setup maps port 80 to port 8083. To test this, you can use the following command from the host system:

curl --include http://localhost:8083

This should show something like the following below:

Cleanup

These systems can be cleaned up by using the vagrant destroy command. This will halt the system and delete the disk image used by the guest VM.

pushd vm_guests/ubuntu2204 && vagrant destroy && popd
pushd vm_guests/rocky9 && vagrant destroy && popd

Addendum: Provisioning Manually

Vagrant automates provisioning the system, but for learning Chef, you may want to run these commands yourself. Here’s how you can do this on the virtual guests.

First, you need to log into the system with vagrant ssh.

So for example, on the Ubuntu 22.04 guest system, you would run this:

cd ~/vagrant-chef/ubuntu2204
vagrant ssh

And for Rocky Linux 9 for example, you would run this:

cd ~/vagrant-chef/rocky9
vagrant ssh

Once on the virtual guest system, run these commands below to setup the client configuration as well as a run list with optional override attributes.

#############
# Find mounted directories for Chef Cookbooks and Nodes
##########################
COOKBOOK_PATH=$(mount | grep -o /tmp.*/cookbooks | uniq)
NODE_PATH=$(mount | grep -o /tmp.*/node | uniq)

#############
# Construct client configuration
##########################
cat << EOF > /tmp/client.rb
cookbook_path ["$COOKBOOK_PATH"]
role_path []
log_level :debug
verbose_logging false
enable_reporting false
encrypted_data_bag_secret nil
data_bag_path []
chef_zero.enabled true
local_mode true
node_path ["$NODE_PATH"]
EOF

#############
# Construct override attributes with run list
##########################
if grep -q rocky /etc/os-release; then
  # configure using override attributes for Rocky Linux
  cat << 'EOF' > /tmp/dna.json
{
  "hello_web": {
    "package": "httpd",
    "service": "httpd",
    "docroot": "/var/www/html"
  },
  "run_list": [
    "recipe[hello_web]"
  ]
}
EOF
else
  # confgiure using default attributes
  cat << 'EOF' > /tmp/dna.json
{
  "run_list": [
    "recipe[hello_web]"
  ]
}
EOF
fi

#############
# Provision the system
##########################
sudo chef-client \
  --config /tmp/client.rb \
  --json-attributes /tmp/dna.json \
  --local-mode \
  --force-formatter \
  --chef-license "accept-silent"

#############
# Logout when finished
##########################
logout

From here, you can run through the tests described in the previous section titled Testing the Solution.

Conclusion

This tutorial helps you get started with creating a local development and a testing environment for Chef (or similar solutions). The provisioning process for Chef will automatically download and install the Chef client, set up the necessary configuration — such as override attributes and a run list — and run the Chef client to converge the system configuration to the desired state, as expressed in your Chef recipes.

Vagrant supports several Chef-related provisioners:

• chef_zero (used in this tutorial): An in-memory Chef server, suitable for testing cookbooks with full support for roles, environments, and data bags.
• chef_solo: A simple provisioner for testing cookbooks without needing a Chef server, but it lacks features like roles and environments.
• chef_client: Requires a Chef server for configuration management.
• chef_apply: A minimal provisioner for testing small recipes, but unsuitable for cookbooks with dependencies.

Industry Usage

I’ve worked with various flavors of Chef in SaaS solutions across industries like legal, e-commerce, video streaming, and application monitoring. These implementations were typically monolithic web applications built with frameworks such as Rails, Laravel, Flask, Play, and Spring, using Postgres or MySQL as the backend databases.

Back in 2014, there was significant demand for Chef expertise, particularly in cloud environments like AWS. However, since 2020, the market has shifted towards cloud-native infrastructure tools such as Terraform and platforms like Kubernetes, leading to fewer Chef-related opportunities.

Final Thoughts

Chef remains a robust platform with extensive community support and a rich ecosystem of open-source tools. Some of the key solutions that complement Chef include:

• Test Kitchen: A tool that launches virtual machines, integrates with Vagrant, provisions systems, and tests them using system test frameworks like ServerSpec, Inspec, Goss, or TestInfra. Test Kitchen has numerous plugins, such as kitchen-dokken for Docker integration, or Kitchen-Terraform to test Terraform modules.
• Knife: A tool for bootstrapping and provisioning systems with Chef. Variants like knife-solo and knife-zero support chef-solo and chef-zero, while knife-ec2, knife-google, and knife-azure extend Chef to cloud environments.
• Ohai: Chef’s tool for gathering system configuration data, with many community plugins. For instance, ohai-plugin-consul integrates with HashiCorp Consul.
• Supermarket: An online repository for Chef cookbooks, providing a vast range of reusable community solutions with notable contributors like sous-chefs.
• Berkshelf to manage cookbooks and cookbook dependencies. Recently, deprecated in favor of Policyfiles.
• ChefSpec is used to write RSpec unit tests to test chef recipes
• Foodcritic is a lint tool for Chef cookbooks.
• CINC: An open-source alternative to Chef, often used as a free version of the platform.
• Goiardi: A lightweight implementation of the Chef server written in Go (Golang).

Chef remains a powerful option for configuration management, particularly for those looking for a mature platform with strong community support. Its ecosystem continues to grow, offering new tools and plugins that expand its capabilities. Exploring Chef, especially in combination with its vibrant community, can provide an effective solution for managing configuration changes.

Resources

Tutorials

• Creating your First Chef Cookbook Updated by Elle Krout on December 3, 2019
• Configuration Management 101: Writing Chef Recipes by Erika Heidi on March 12, 2020
• A Beginner’s Guide to Chef by Elle Krout on March 4, 2021