Virtualization on MacBook M1/M2 (ARM64)

Step-by-Step Guide for DevOps on Apple Silicon with Vagrant and QEMU

For most local development workflows, Docker, is often the preferred tooling, but there sometimes are need to run a full virtual machine, ideally with popular tooling like Vagrant, which can manage virtual machines, connectivity, as well as provisioning the system with scripts or a tool like Ansible.

On new Macbooks M1/M2/M3, that comes with arm64 processor, this is not so straightforward. This article covers a few quick steps where you can use QEMU to run virtual machines with either arm64 virtual machines using Apple’s hypervisor, or use a emulator to run amd64 virtual machines.

Requirements

• Operating System: macOS 12.6.3 or higher
• Hardware: Apple M1 or newer generation
• Software: Homebrew for installing packages

Tools Installed

• QEMU 8.2.1
• Vagrant 2.4.1
• Vagrant vagrant-qemu plugin 0.3.5

Installation

The initial steps will be to get the virtual machine solution (QEMU) and install Vagrant with the QEMU plugin:

# install VM solution
brew install qemu

# install Vagrant with QEMU support
brew install --cask vagrant
vagrant plugin install vagrant-qemu

Configure and Launch arm64 image

Afterward, set up a Vagrant configuration file, download the VM image, start the virtual machine, and log into the VM.

# create configuration file for arm64 vm image
cat <<EOF > Vagrantfile
Vagrant.configure("2") do |config|
  config.vm.box = "perk/ubuntu-2204-arm64"
  config.vm.provider "qemu" do |qe|
    qe.ssh_port = "50022" # change ssh port as needed
  end
end
EOF

# download and startup VM using hvf
vagrant up --provider=qemu

# log into the VM
vagrant ssh

📓NOTE: The default ssh port is 50022 for all virtual machines, even ones that are not actively running. Thus you may need to adjust the port to avoid port collisions. You can run lsof -nP -iTCP:50022 to investigate in anything is locking the port.

This will use virtualization with the the native hypervisor built into the macOS.

Testing the virtual machine (arm64)

Once inside you can install inxi and neofetch:

sudo apt-get update
sudo apt install -y inxi neofetch

Then test with using neofetch:

And test the environment by running with inxi by typing:

inxi --system --machine -- cpu -- network --disk --info

You should see something like this:

Configure and Launch x86 image

The default settings for vagrant-qemu plugin will not work for running Intel based images, so we’ll have to make some adjustments to the Vagrant configuration (Vagrantfile).

Run this in a separate directory to keep configurations for amd64 and arm64 separated.

📓 NOTE: If you are using the same directory, you’ll need to run vagrant destroy to clean up the previous virtual machine.

cat <<EOF > Vagrantfile
Vagrant.configure("2") do |config|
  config.vm.box = "generic/ubuntu2204"
  config.vm.provider "qemu" do |qe|
    qe.arch = "x86_64"
    qe.machine = "q35"
    qe.cpu = "max"
    qe.net_device = "virtio-net-pci"
    qe.ssh_port = "50023"
  end
end
EOF

# download and startup VM using q35 emulator
vagrant up --provider=qemu

# log into the VM
vagrant ssh

Here’s is a break down of some of the setting changes:

• arch: specify x86_64 (intel) to run amd64 images.
• machine: default is virt,accel=hvf,highmem=on, which uses built-in hypervisor on the macOS. For x64_64, we’ll need to use the emulator, so use q35.
• cpu: default is home, change this to max.
• net_device: the default virtio-net-device will not work as there is no virtio-bus with the emulator. Change this to virtio-net-pci.
• ssh_port: the default is 50022, so we need to change to another port. Run lsof -nP -iTCP:50022 to see the if the port is in use.

Testing with the emulator (x86_64)

Once inside you can install inxi and neofetch:

sudo apt-get update
sudo apt install -y inxi neofetch

Then test with using neofetch:

And test the environment by running with inxi by typing:

inxi --system --machine -- cpu -- network --disk --info

You should see something like this:

Update: Limitations

I recently learned, as of May 2024, that port forwarding does not currently function due to vagrant-qemu issue 40. This feature is needed if you wish interact with the application running on the virtual guest. For example, if you are running a web application that is accessible on port 8080, you would use port forwarding to make this port accessible from the host.

From what I can gather, the current QEMU tools require elevated privileges in order to facilitate port forwarding functionality (see QEMU issue 1364). There’s a required entitlement, which is “a right or privilege that grants an executable particular capabilities”), to be able use such networking features (specially the com.apple.vm.networking API) without the need for privilege escalation. The UTM project, as an example, which also uses QEMU tools, has registered to use an entitlement for this library.

In this vagrant-qemu issue 40, there was mention socket_vmnet from the LIMA project. This facility will provide vmnet.framework support for QEMU. The service will itself have to run as root, but the QEMU tools will not require running as root.

It looks like the current direction is to wait and see what QEMU project does, as they are actively looking into this with QEMU issue 1364. Once QEMU team implements a solution, vagrant-qemu can be updated to support it. If this doesn’t happen, then maybe the socket_vmnet solution could be looked at as an alternative.

In the mean time, you may have to use the QEMU tools directly to setup the necessary infrastructure required for port forwarding, and then setup port forwarding.

Resources

Articles

These are articles that I have come across when researching this.

• ARM64 VM on macOS with libvirt + QEMU by 박윤식 (Yun-shik Park) on December 9, 2021. Shows how to use qemu-img to create qcow image, VNC client, and virsh.
• Vagrant QEMU on Apple Silicon
• ARM64 VM on macOS with libvirt + QEMU

Gists

Small code snippets or mini-articles created by authors that got off the ground with QEMU on Apple Macbook M1.

• Linux Virtual Machines with Private Network on an Apple M1 Device by David Andreoletti.
• Running Linux and Windows on M1 with QEMU by citruz and last updated on June 1, 2022.
• How to run Windows 10 on ARM or Ubuntu for ARM64 in QEMU on Apple Silicon Mac by Yoshimasa Niwa and last updated on October 2, 2021.

Vagrant Box Images

• Vagrant Box generic/ubuntu2204 (intel only)
• https://app.vagrantup.com/perk/boxes/ubuntu-2204-arm64 (arm64)
• https://app.vagrantup.com/ppggff/boxes/centos-7-aarch64-2009-4K (arm64, used for testing)

Other

• Vagrantfile — Vagrant + Ubuntu + QEMU provider tested in MacOs M1 ARM64 by eduardomp (Eduardo Medeiros Pereira) on June 18, 2023. Content looks invalid, but interested readers have added corrections to help fix it.
• vagrant-xhyve: works with xhyve or hyperkit on Intel_x86, so maybe it might work on arm64.
• Creating a Ubuntu Desktop 18.04 Virtual Machine on macOS with QEMU: Covers Macbook (intel) steps to use QEMU to run Ubuntu OS.
• Windows Insider Preview Downloads for ARM64. Note that this requires registered with Windows Insider program.

Related Articles

Vagrant with Macbook (intel)

This is a combination and how-to and survey of operating systems launched with Vagrant using Virtualbox.

Vagrant with Macbook (intel)

Conclusion

That’s all there is to this. The tool QEMU has been around for a long time, and supports a variety of emulators for different processors and supports native virtualization. On macOS, virtualization default is Hypervisor Framework (hvf), and on Linux, KVM is supported. The QEMU tool has always been common tool to manage both emulators and a variety of virtualization solutions.

For those not familiar with Vagrant, this is a tool that manages virtual machines through a single intuitive interface command-line interface and configuration file called Vagrantfile. Vagrant also automates connections to the virtual machine, such as ssh, and provisioning the virtual machine using scripts, change configuration such as Ansible, or containers with tools like Docker.

The plugin for Vagrant vagrant-qemu is a wrapper for the qemu command line tool. With this, you can download images that support qemu from https://app.vagrantup.com.