Image for post
Image for post

VirtualBox and Friends on macOS

VirtualBox, Vagrant, Test Kitchen, Docker Machine, Minikube

Introduction

Across macOS, Windows, and Linux, you’ll find these two popular tools of awesomeness: Vagrant and VirtualBox.

Image for post
Image for post

Vagrant allows you to automate creation and provisioning of disposable systems and infrastructure with ease, using a variety of backend virtualization solutions, with the free open source VirtualBox.

You can provision a system using CFEngine, CAPS (Chef, Ansible, Puppet, Salt Stack), Docker, shell scripts, etc. Vagrant can be extended through a plug-in interface to support additional provisioners, providers, or other features.

Image for post
Image for post

Beyond Vagrant, you can validate configuration of your systems using test automation with a tool called Test Kitchen.

The automation will be written ServerSpec or InSpec automation scripts. Test Kitchen uses Vagrant as its default driver to create disposable test systems, and can use a variety of other backend drivers to support cloud providers, Docker, or other virtualization solutions.

Like Vagrant, Test Kitchen can be extended through its plug-in system to add other test drivers, test provisioners, test verifiers, or other features.

Image for post
Image for post

For container environments with Docker, there’s a tool called Docker Machine that creates a sandboxed Docker environment using a variety of backend virtualization solutions, where the free open source VirtualBox is the default. Docker Machine configures the client to use the sandboxed Docker environment.

Image for post
Image for post

Another tool called Minikube can use Docker Machine to run a small sandboxed Kubernetes cluster in a virtualization solution. With these, you can easily develop and test infrastructure leveraging both Docker and Kubernetes.

One thing great about these systems that can be used to communicate to each other as long as they are all on VirtualBox. VirtualBox creates virtual private networks that are accessible from your host system. You can for example run a Kubernetes cluster, and test deployments from kubectl running on a virtual guest system created by either Test Kitchen or Vagrant.

This is a tutorial that demonstrates how to install and use these tools on a macOS High Sierra 10.3.5 host.

Required Prerequisite

Apple macOS does not have a package management system for installing FOSS (free and open-source software).

Image for post
Image for post

We can compensate for this by installing Homebrew. This will require administrative access to your computer and will need to download and install XCode command line tools as a part of the process if not installed already:

# variables for readibility
PREFIX=https://raw.githubusercontent.com
PATH=
Homebrew/install/master/install
URL=${PREFIX}/${PATH}
# install homebrew w/ ruby install script
/usr/bin/ruby -e "$(curl -fsSL ${URL})"

Apple has made it difficult to install packages that require installing services that use kernel extensions (or KEXT) starting with macOS High Sierra 10.13. For more info, you can read Tech Note 2459. To work around this, we can run this command before installing VirtualBox specifically:

sudo spctl –master-disable

Virtualbox

Image for post
Image for post

The central cross platform tool (Linux, Windows, macOS) to make all of this magic work is VirtualBox.

Installing VirtualBox

With Homebrew, you can install VirtualBox by running this

brew cask install virtualbox
brew cask install virtualbox-extension-pack

Afterward, we can check to see if it is installed:

# Test Version
vboxmanage --version
5.2.16r123759

Vagrant

Image for post
Image for post

Vagrant automates virtual machines and containers. It is essentially an orchestrator to juggle running different systems and quickly provision them with your favorite change configuration solution, like shell, CFEngine or CAPS (Chef, Ansible, Puppet, Salt Stack).

With Vagrant, we can easily download and run system images from Vagrant Cloud or other sources, or even run images you created locally with a tool like Packer).

Installing Vagrant

We can install Vagrant easily with:

brew cask install vagrant

Using Vagrant: Mac on Mac

In this demo, we can run macOS guest system on top of macOS host. This image was created and installed earlier using Boxcutter scripts:

cd 
mkdir mymacosx && cd mymacosx
vagrant init my/macos-1012 && vagrant up
vagrant ssh --command 'sw_vers'
ProductName: Mac OS X
ProductVersion: 10.12.6
BuildVersion: 16G1036
Connection to 127.0.0.1 closed.

Note (Off Topic): Due to current issues (#32 and #26), only earlier versions of macOS (aka Mac OS X) can be used, specially, 10.12.3 and earlier. They can be upgraded manually to 10.12.6.

Now let’s do something a little more colorful, and download and install screenFetch on our virtual guest:

URL=https://github.com/KittyKatt/screenFetch/archive/master.zip
vagrant ssh --command "curl -OL ${URL}"
vagrant ssh --command 'unzip master.zip'
vagrant ssh --command './screenFetch-master/screenfetch-dev'

This will display something like following:

Image for post
Image for post
ScreenFetch running on Mac OS X 10.12.6

Using Vagrant: Windows on Mac

We can run through this same process with another image built by Matt Wrock’s Windows Packer scripts:

cd
mkdir mywindows && cd mywindows
vagrant init my/win2016 && vagrant up

Out of the box, Windows 2016 does not support SSH. So we can download an installer for BitVise SSHD server on the host, and copy it to the guest afterward:

curl -LO https://dl.bitvise.com/BvSshServer-Inst.exe

We can connect to our systems system through RDP (Remote Desktop Protocol), assuming you’ve installed Microsoft Remote Desktop on your host:

# connect with Remote Desktop Protocol 
# user = vagrant, password = vagrant
vagrant rdp

We can access the BitVise SSHD installer from the virtual file server created by VirtualBox, by hitting the CMD key (which corresponds to the Windows key) in the RDP session and then typing:

\\VBOXSVR\vagrant\BvSsServer-Inst.exe

Follow through options for the installer, the Personal Edition will work for us, and once finished, in the Bitvise SSH Server Control Panel, click on the Start Server link. Once started, the status will change to running, and look something like this:

Image for post
Image for post
BitVise SSH Server running

With SSHD installed, we can now log in SSH from the host:

# connect with Secure Shell 
# user = vagrant, password = vagrant
vagrant ssh

Afterward, we’ll have a Command Shell prompt, and can type ver to verify:

C:\Users\vagrant>verMicrosoft Windows [Version 10.0.14393]C:\Users\vagrant>

From here, if we want use screenFetch, we need to get a bash shell by installing MSYS2, which we can install using Chocolatey package management system. In the same SSH session, type these commands:

:: env variables for readability
SET PSHL=%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe
SET OPTS=-NoProfile -InputFormat None -ExecutionPolicy Bypass
SET OBJ=System.Net.WebClient
SET SCRPT='https://chocolatey.org/install.ps1'
SET CMD=iex ((New-Object %OBJ%).DownloadString(%SCRPT%))
:: install chocolatey
%PSHL% %OPTS% -Command "%CMD%"
:: setup local path
SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
:: install msys2 package
choco install -y msys2
:: run the bash shell
c:\tools\msys64\usr\bin\bash.exe

From bash shell, we can now install and run screenFetch:

# install unzip package
PATH
=/usr/bin:$PATH
pacman -S unzip
# install screenfetch
URL=https://github.com/KittyKatt/screenFetch/archive/master.zip
curl -OL $URL
unzip master.zip
# run screenfetch
./screenFetch-master/screenfetch-dev

This will give us:

Image for post
Image for post
ScreenFetch running on MSYS2 on Windows Server 2016

Test Kitchen

Image for post
Image for post

Test Kitchen is a test harness, and similar to Vagrant, it orchestrates several systems, provisions them, and then facilitates running tests on them. It in fact uses Vagrant as the backend driver to create the virtual guest systems, but can be configured to use Docker or other backends.

This is useful to test out systems before pushing changes to a staging or production environment.

Installing Test Kitchen

If you have a ruby environment, you can install Test Kitchen with gem install test-kitchen.

For demonstration purposes, and easy scaffolding, we’ll install Test Kitchen using ChefDK set of tools:

brew tap chef/chef
brew cask install chefdk

Using Test Kitchen with ChefDK

For this small demo, we can use the Chef to auto-generate a cookbook, which includes a default Test Kitchen configuration:

Image for post
Image for post
Test Kitchen configuration

To get started, we generate a cookbook, and bring up the environment.

# Generate example
chef generate cookbook helloworld && cd helloworld
# Create Ubuntu and CentOS systems
kitchen create

This will create Ubuntu and CentOS systems using Vagrant as the backend driver. We can then view the running systems with kitchen list:

Image for post
Image for post

Run Something on Test Systems

Let’s download a screenFetch shell script locally:

wget https://github.com/KittyKatt/screenFetch/archive/master.zip
unzip master.zip
mv screenFetch-master/ ${HOME}/.kitchen/cache/

The ~/.kitchen/cache is a global shared directory for all Vagrant guest systems managed by Test Kitchen. We can exchange files between the host and virtual guest systems using this mechanism.

When we run the script on the host, it will look like this:

Image for post
Image for post
Screenfetch running on host

With screenFetch script available from the shared directory, let’s use it across all systems.

# Install pciutils on CentOS (required by screenfetch)
kitchen exec centos --command='sudo yum -y install pciutils'
# Install a snap on Ubuntu (avoids warnings w/ screenfetch)
kitchen exec ubuntu --command='sudo snap install hello-world'
# Run screenfetch script on all systems
kitchen exec default* \
--command='sudo /tmp/omnibus/cache/screenFetch-master/screenfetch-dev'

This will print something pretty for all clients:

Image for post
Image for post
Screenfetch script on all test-kitchen clients

Docker Machine

Image for post
Image for post

Docker Machine is an automation tool that can spin up a virtual system for facilitating Docker. Specifically on macOS, you have to run Docker on a virtual guest, as Linux containers automated by Docker only run on Linux. Docker Machine automates this process.

Install Docker Toolbox

Image for post
Image for post

We can get a recent version of Docker Machine from Docker Toolbox package. This will also install the Docker client as well.

brew cask install docker-toolbox

Using Docker

With these tools installed, let’s run some containers, using Docker client on the host after starting a Docker daemon running on a virtual system:

# Create a docker machine environment called default
docker-machine create --driver virtualbox 'default'
# Tell docker engine to use machine's docker (defaulting to default)
eval $(docker-machine env)
# Run a container fetched from docker hub
docker run docker/whalesay cowsay Hello World
_____________
< Hello World >
-------------
\
\
\
## .
## ## ## ==
## ## ## ## ===
/""""""""""""""""___/ ===
~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ / ===- ~~~
\______ o __/
\ \ __/
\____\______/

Minikube

Image for post
Image for post

Minikube allows you to run Kubernetes cluster locally. It leverages off of Docker Machine to create a single-node Kubernetes cluster.

Some features supported are: DNS, NodePorts, ConfigMaps and Secrets, Dashboard, Container Runtimes (Docker, rkt, CRI-O), and Enabling CNI (Container Network Interface) and Ingress.

Installing Minikube

Install Minikube using brew:

brew cask install minikube

Installing Kubectl Client

We need to install a Kubernetes client like Kube Control, or kubectl:

brew install kubectl

Using MiniKube

We can now start up our cluster and deploy some containers on our cluster. When we start up Minikube, it will make the necessary client configurations needed for kubectl.

# Start minikube environment
minikube start --vm-driver=virtualbox
# Deploy Something
kubectl run hello-minikube \
--image=k8s.gcr.io/echoserver:1.4 \
--port=8080
kubectl expose deployment hello-minikube \
--type=NodePort

We check to see if the pod(s) are running.

kubectl get pod
NAME READY STATUS RESTARTS AGE
hello-minikube-6c47c66d8-ft5dm 1/1 Running 0 31s

Once the pods are ready, we can connect to the running service’s endpoint:

curl $(minikube service hello-minikube --url)
CLIENT VALUES:
client_address=172.17.0.1
command=GET
real path=/
query=nil
request_version=1.1
request_uri=http://192.168.99.101:8080/
SERVER VALUES:
server_version=nginx: 1.10.0 - lua: 10001
HEADERS RECEIVED:
accept=*/*
host=192.168.99.101:30764
user-agent=curl/7.54.0
BODY:
-no body in request-

Conclusion

Tools Installed

In total, we have installed 7 tools (1 virtual management platform, 4 x orchestration systems, and 2 client tools):

printf "\nVirtualBox %s\n" $(vboxmanage --version) && \
vagrant --version && \
kitchen --version && \
docker-machine --version && \
docker --version && \
minikube version && \
printf "Kubectl Client: %s\n" \
$(kubectl version | awk -F\" \
'/Client/{ print $6 }')
VirtualBox 5.2.16r123759
Vagrant 2.1.2
Test Kitchen version 1.22.0
docker-machine version 0.14.0, build 89b8332
Docker version 18.03.0-ce, build 0520e24
minikube version: v0.28.2
Kubectl Client: v1.11.1

We installed these piece by piece, but if you wanted to install these all in one command, you could create a Brewfile and install them all at once with brew bundle --verbose.

Image for post
Image for post

From Bash, you can create and run using this:

cat <<-'BREWFILE_EOF' > Brewfile
cask 'virtualbox'
cask 'virtualbox-extension-pack'
cask 'vagrant'
tap 'chef/chef'
cask 'chefdk'
cask 'docker-toolbox'
cask 'minikube'
brew 'kubectl'
BREWFILE_EOF
brew bundle --verbose

Virtual Guest Systems Created

With these 4 orchestration tools, we have created 6 virtual guests on VirtualBox:

$ vboxmanage list runningvms | cut -d'"' -f2
default
default-ubuntu-1604_default_1532853917716_29098
default-centos-7_default_1532854036210_69596
minikube
mymacosx_default_1532856501604_64351
mywindows_default_1532856645901_82466

And with the graphical Virtualbox Manager console, we can see the same thing:

Image for post
Image for post

Cleaning Up

Now that we are finish, we can stop the running virtual machines hosting our development and test solutions, and optionally remove them completely.

######## vagrant w/ macosx 10.12 cleanup ########
cd ~/mymacosx
vagrant halt # stop running vm guest
vagrant destroy # delete vm guest entirely
######## vagrant w/ win2016 cleanup ########
cd ~/mywindows
vagrant halt # stop running vm guest
vagrant destroy # delete vm guest entirely
######## testkitchen cleanup ########
cd ~/helloworld
kitchen destroy # destroys all test systems
######## dockermachine cleanup ########
docker-machine stop # stop vm hosting docker
docker-machine rm default # remove vm entirely
######## minkube cleanup ########
minikube stop # stop kubernetes cluster
minikube rm # remove vm hosting cluster and kubectl config entries

After stopping the virtual guests, vboxmanage list runningvms will show an empty list. If we removed them as well, then vboxmanage list vms will also show an empty list.

Enable Kernel Extension Block

We disabled this Apple feature earlier, we can re-enable it:

sudo spctl –master-enable

Final Thoughts

The main goal of this is to show you what is possible and create some interest and passion around these set of tools.

With these cross platform tools, we can easily facilitate experimentation and learning activities, and drive DevOps patterns from local development and test into a continuous integration and continuous delivery pipeline.

Resources

Here are some links for these tools.

Virtualbox

Vagrant

Vagrant supports has many backend provider solutions besides Virtualbox and has built in provisioner support for popular change configuration solutions.

Test Kitchen

Test Kitchen test harness has rich community support for a variety of backend driver support, provisioner support, and verifier support:

Docker Machine

Docker Machine has a few options for backend providers outside of Virtualbox:

Minikube

Vagrant Boxen

Change Configuration

Systems Integration Testing

Written by

Linux NinjaPants Automation Engineering Mutant — exploring DevOps, Kubernetes, CNI, IAC

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store