Vagrant Provisioning with Puppet

Provisioning Virtual System using Puppet Apply

Vagrant has two provisioners for Puppet, one that requires a puppet server, called puppet_server, and one that doesn’t, called puppet. This tutorial will use the later. In this brief tutorial, we’ll walk through how to setup a minimal Vagrant and Puppet environment and creating a Puppet module for a Apache web server. This will show how to use the puppet provisioner, which uses puppet apply on the command line.

About Puppet

The configuration scripts used in Puppet are called manifests, and these can be organized into a component called a module. Puppet manifest use an intuitive proprietary DSL language oriented configuring resources, where resource is a “fundamental unit for modeling system configurations” and where “each resource describes some aspect of a system, like a specific service or package” (reference on resources).

The traditional starting point in Puppet is the site manifest, where you create node definitions that match on hostnames and define a list of classes that you wish to run on the matching system. For this tutorial, we’ll use the default node, which matches all systems.

Prerequisites

I have published some previous guides on installing Vagrant and Virtualboxon macOS, Windows, and Linux operating systems.

Windows 8.1

macOS High Sierra 10.3

Fedora 28

Part I: Puppet Module with Intelligent Defaults

Create Staging Area

mkdir -p ~/vagrant-puppet/{site,manifests}
cd ~/vagrant-puppet
mkdir -p site/hello_web/{files,manifests}
touch Vagrantfile bootstrap.sh manifests/default.pp \
site/hello_web/manifests/init.pp
cat <<-'HTML' > site/hello_web/files/index.html
<html>
<body>
<h1>Hello World!</h1>
</body>
</html>
HTML

This will create the following structure:

~/vagrant-puppet
├── Vagrantfile
├── bootstrap.sh
├── manifests
│ └── default.pp
└── site
└── hello_web
├── files
│ └── index.html
└── manifests
└── init.pp

Vagrant configuration

Vagrant.configure("2") do |config|
config.vm.box = "bento/ubuntu-16.04"
config.vm.network "forwarded_port", guest: 80, host: 8084
####### Install Puppet Agent #######
config.vm.provision "shell", path: "./bootstrap.sh"
####### Provision #######
config.vm.provision "puppet" do |puppet|
puppet.module_path = "./site"
puppet.options = "--verbose --debug"
end
end

Vagrant does not currently bootstrap, or rather install puppet agent. Thus, in order to use the Puppet Provisioner, we need to use a Vagrant box that already has puppet agent pre-installed (not advisable as this gets into the golden image anti-pattern), or we have to install it. For this tutorial, we’re going to use the shell provisioner to install the Puppet agent.

For the Puppet Provisioner, we specify where to source our modules, which is from a local directory ./site/. We’ll use the default site manifest location, which is ./manifests/default.pp.

We also turn on verbosity and debug so we can peak at what puppet is doing.

Bootstrap Puppet Agent

Site Manifest

node default {
class { 'hello_web': }
}

The term class here represents the module. Alternative to this syntax, if you are not passing parameters into the class, you could simply do this:

node default {
include hello_web
}

Module Class

In the module under site/hello_web, update the manifests/init.pp:

class hello_web (
$package_name = 'apache2',
$service_name = 'apache2',
$doc_root = '/var/www/html'
) {
package { $package_name:
ensure => present,
}
service { $service_name:
ensure => running,
enable => true,
}
file { "$doc_root/index.html":
source => "puppet:///modules/hello_web/index.html",
}
}

Some background on defaulting variables. Puppet doesn’t actually have variables, they are really constants, but Puppet calls them variables. If you want to have something that acts like a real variable, where it can be overriden, you use parameters in Puppet. In the parameter list, you can set them to a desired default. For this reason, many modules may have rather large parameter lists.

For the resources being configured, Puppet’s DSL follows this pattern:

type { 'title':
attribute => value
}

The service and package resources are pretty straight forward, but the the file resource in Puppet is something special. It does many operations, one of them is to copy files from the files directory in the Puppet module to the desired destination. To reference this, puppet uses the syntax:

"puppet://modules/<your_module_name>/<your_file_name>"

This will map to the following below, where $MODULE_PATH is ./site/ in our case.

$MODULE_PATH/<your_module_name>/files/<your_file_name>

Test the Solution

vagrant up
curl -i http://127.0.0.1:8084

This will give us something like this:

HTTP/1.1 200 OK
Date
: Sun, 12 Aug 2018 14:04:06 GMT
Server: Apache/2.4.18 (Ubuntu)
Last-Modified: Sun, 12 Aug 2018 14:02:40 GMT
ETag: "3c-5733d6e1783d6"
Accept-Ranges: bytes
Content-Length: 60
Content-Type: text/html
<html>
<body>
<h1>Hello World!</h1>
</body>
</html>

Part II: Pass Parameters to Puppet Module

Update Vagrant Configuration

Vagrant.configure("2") do |config|
config.vm.box = "bento/centos-7.5"
config.vm.network "forwarded_port", guest: 80, host: 8084
####### Install Puppet Agent #######
config.vm.provision "shell", path: "./bootstrap.sh"
####### Provision #######
config.vm.provision "puppet" do |puppet|
puppet.module_path = "./site"
puppet.options = "--verbose --debug"
end
end

Update Site Manifest

Update the manifests/default.pp to match the following:

node default {
class { 'hello_web':
package_name => 'httpd',
service_name => 'httpd',
doc_root => '/var/www/html',
}
}

Note: For the this is tutorial, it is fine to manually rejigger the values in the node definition, but out there in the real world, this is an anti-pattern and something we should never do, as humans are required for the automation to constantly change values, and thus won’t be automation anymore.

Test the Update

vagrant destroy --force
vagrant up
curl -i http://127.0.0.1:8084

This should give us something like:

HTTP/1.1 200 OK
Date
: Sun, 12 Aug 2018 14:56:32 GMT
Server: Apache/2.4.6 (CentOS)
Last-Modified: Sun, 12 Aug 2018 14:56:14 GMT
ETag: "3c-5733e2d9ffceb"
Accept-Ranges: bytes
Content-Length: 60
Content-Type: text/html; charset=UTF-8
<html>
<body>
<h1>Hello World!</h1>
</body>
</html>

Alternative Strategy (Intermediate)

As alternatives, there are at least two strategies, we can classify our nodes or have smarter parameters..

Node Classification with Node Definitions

Vagrant.configure("2") do |config|
config.vm.box = "bento/centos-7.5"
config.vm.hostname = "dk-ord-web01"
config.vm.network "forwarded_port", guest: 80, host: 8082
####### Install Puppet Agent #######
config.vm.provision "shell", path: "./bootstrap.sh"
####### Provision #######
config.vm.provision "puppet" do |puppet|
puppet.module_path = "./site"
puppet.options = "--verbose --debug"
end
end

To use this, you at minimum would need to do vagrant reload to assign the hostname, but better would be to remove it with vagrant destroy, so we can start with a clean virtual guest with vagrant up --no-provision.

Now that we have a desired hostname assigned to the virtual guest, we can then reference this in our manifest below.

Here’s two node definitions, borrowing some creative imagination, one definition for CentOS web servers in Chicago data center, and another for Ubuntu web servers in Las Vegas data center:

node dk-ord-web* {
class { 'hello_web':
package_name => 'httpd',
service_name => 'httpd',
doc_root => '/var/www/html',
}
}
node dk-las-web* {
class { 'hello_web':
package_name => 'apache2',
service_name => 'apache2',
doc_root => '/var/www/html',
}
}

Smart Parameters with inherited class

To get started with this, create a ./site/hello_web/manifests/params.pp file that can self configure based on, for example, the OS family:

class hello_web::params {
case $::osfamily {
'Debian': {
$package_name = "apache2"
$service_name = "apache2"
$doc_root = "/var/www/html"
}
'RedHat': {
$package_name = "httpd"
$service_name = "httpd"
$doc_root = "/var/www/html"
}
}
}

Once this is in place, we can refer to them in our modules class init.pp:

class hello_web(
$package_name = $hello_web::params::package_name,
$service_name = $hello_web::params::service_name,
$doc_root = $hello_web::params::doc_root
) inherits hello_web::params {
package { $package_name:
ensure => present,
}
service { $service_name:
ensure => running,
enable => true,
}
file { "$doc_root/index.html":
source => "puppet:///modules/hello_web/index.html",
}
}

This will inherit the params class so that we can access the values, and then copy them from whatever the params configures, and installed them as defaults to our main class.

To demonstrate this working, revert the site manifest ./manifests/default.pp to the basics:

node default {
include hello_web
}

Now using either CentOS (bento/centos-7.5) or Ubuntu (bento/ubuntu-16.04) for the VM box set in the Vagrantfile, bring up a new system:

vagrant destroy --force
vagrant up
curl -i 127.0.0.1:8084

Final Thoughts

This is just the basic surface level of what is possible with Puppet. Puppet can be extended to support alternative mechanism to classify nodes beyond a Node manifest. This is done through the ENC (External Node Classifier) mechanism, where you can create your own solution, or use some of the popular ones.

In the Puppet community, currently systems are configured with an external classifier called Hiera, that uses a hierarchical structure to classify what classes are configured on a node. Hiera supports multiple backends, such as YAML, JSON, and databases.

This rich environment using Hiera is now the preferred method of configuration. I did not want to jump into Hiera just yet, as the idea of this tutorial is not to blow up newcomers to smithereens for an introductory tutorial.