Vault AppRole Auth: The Easy Way

Securely Storing Secrets with HashiCorp Vault

From the Iron Age to the Cloud Age, the practice of storing secrets in text files was common. However, this method poses significant security risks as it’s usually only a matter of time before these secrets are accessed by unauthorized parties.

In this tutorial, we will demonstrate how to securely store static secrets using Haschicorp Vault, specifically through the creation of the AppRole identity that is utilized by the application, Dgraph, which has built in support for the AppRole. This method leverages the powerful capabilities of Vault to enhance the security of storing secrets needed for applications.

Setting Up the Environment

We will use Docker to set up a test environment that includes a single node Vault server and Dgraph, a distributed graph database that integrates seamlessly with Vault. Dgraph offers advanced features like ACLs for configuring secure access and encryption capabilities for database security.

Objectives:

• Launch a Vault server using Docker.
• Set up Dgraph with Vault to handle secrets securely.

Step-by-Step Guide

1. Launch Vault with Docker: We will start by launching a Vault server in a Docker container. This setup provides a clean and isolated environment for learning and testing.
2. Configure Vault: Next, we’ll set up Vault using the CLI to initialize the server, create roles, and configure policies. This process ensures that Vault can manage access to the secrets efficiently.
3. Storing Secrets: Instead of storing secrets in a file on disk, we will save them directly in Vault. This approach significantly enhances security by encrypting secrets and controlling access through defined policies.
4. Integrate Dgraph with Vault: Once Vault is configured, we’ll proceed to set up Dgraph. We will configure Dgraph to use Vault for securely handling secret information such as keys for ACLs and database encryption.
5. Testing and Validation: Finally, we will demonstrate how Dgraph accesses secrets from Vault, ensuring that the setup works as expected and that the integration provides the necessary security enhancements.

About AppRole

In IT infrastructure, access to resources is managed through identities or accounts, such as users or groups. For automation, Vault employs a special identity called the AppRole. This identity is designed for processes with minimal human interaction, allowing applications and services secure and efficient resource access. The AppRole operates with a Role ID (similar to a username), a Secret ID (akin to a password generated on demand), and a Token (issued upon verification of the IDs by Vault, granting temporary resource access based on configured policies).

This tutorial will guide you in creating two roles:

1. Admin Role: For operations engineers to manage Dgraph configuration secrets.
2. Dgraph Role: For the Dgraph application to access these secrets from Vault.

Requirements

These are the tool requirements used for this tutorials

Tool Requirements

• Docker Desktop† (macOS or Windows) or Docker Engine (Linux)
• Hashicorp Vault installed on host for easy client access
• POSIX Shell (Bash or Zsh) with GNU Grep. Windows can install MSYS2 to get these tools.
• jq is a json query tool useful for extraction information from JSON

Tested Environments

This was tested on the following environments

macOS Monterey 12.6.3 build 21G419
--------------------------------------------------
* Docker Desktop for macOS 4.29.0
  * Docker Engine 26.0.0
    * Plugin: Compose v2.26.1
* GNU bash, version 5.2.21(1)-release (aarch64-apple-darwin21.6.0)
* grep (GNU grep) 3.11
* jq 1.7.1
* Vault v1.16.2

Windows 11 Home [WinNT 10.0.22631.34467] with MSYS
--------------------------------------------------
* Docker Desktop for Windows 4.29.0
  * Docker Engine 26.0.0
    * Plugin: Compose: v2.26.1
* MSYS
  * GNU bash, version 5.2.26(1)-release (x86_64-pc-msys)
  * grep (GNU grep) 3.0
  * jq 1.7.1
* Vault v1.16.1

Pop!_OS 22.04 LTS (Ubuntu Jammy)
--------------------------------------------------
* Docker Engine version 26.1.0, build 9714adc
  * Plugin: Compose version v2.26.1
* GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)
* grep (GNU grep) 3.7
* jq 1.6
* Vault v1.16.2

Windows Tips

You can install tools such as Docker Desktop, Hashicorp Vault, and MSYS2 using the Chocolatey package management system with these commands in PowerShell with the Run As Administrator option:

choco install msys2 
choco install vault 
choco install docker-desktop

After setting up MSYS2, launch the MSYS2 environment and run the following to update packages and install jq:

# update packages
pacman -Suy
# install jq
pacman -S mingw-w64-x86_64-jq curl

To make the commands vault, jq, and vault available, we need to setup a new path:

DOCKER_PATH="/c/Program Files/Docker/Docker/resources/bin/"
CHOCO_PATH="/c/ProgramData/chocolatey/bin"
MINGW_PATH="/mingw64/bin/"

export PATH=$PATH:$MINGW_PATH:$CHOCO_PATH:$DOCKER_PATH

macOS Tips

If you can Homebrew installed, you can install the Docker Desktop, Hashicorp Vault, GNU grep, and jq.

# Docker Desktop
brew -cask install docker
# Hashicorp Vault
brew tap hashicorp/tap
brew install hashicorp/tap/vault
# GNU Grep and jq
brew install grep jq

GNU Grep will have to be configured in the PATH to take precedent over the limited BSD grep:

export PATH="$HOMEBREW_PREFIX/opt/grep/libexec/gnubin:$PATH"

Ubuntu Tips

Debian, Ubuntu, and most other Linux distributions are equipped with a standard POSIX environment, which includes GNU Bash and GNU Grep. For tools like curl and jq, you can install these with the following on Debian based distros like Ubuntu:

sudo apt-get update && sudo apt install curl jq 

For Vault and Docker with the Compose plugin, visit these sites for further instructions:

• Hashicorp Developer — Getting Started: Install Vault
• Docker Docs: Install Docker Engine

Project Setup

This setup will setup the configuration for this tutorial

Setup Docker Compose

Run the following below to setup the compose environment.

## Setup Compose Env configuration
cat << EOF > .env
DGRAPH_VERSION=v23.1.1
VAULT_VERSION=1.16
EOF

## Setup Compose configuration
cat << 'EOF' > compose.yml
services:
  zero1:
    image: dgraph/dgraph:${DGRAPH_VERSION}
    command: dgraph zero --my=zero1:5080 --replicas 1 --raft idx=1
    ports:
      - 6080:6080
    container_name: zero1

  alpha1:
    image: dgraph/dgraph:${DGRAPH_VERSION}
    ports:
      - 8080:8080
      - 9080:9080
    environment:
      DGRAPH_ALPHA_CONFIG: /dgraph/config/config.yaml
    volumes:
      - ./dgraph/alpha.yaml:/dgraph/config/config.yaml
      - ./dgraph/vault_secret_id:/dgraph/vault/secret_id
      - ./dgraph/vault_role_id:/dgraph/vault/role_id
      - ./dgraph/backups:/dgraph/backups
      - ./dgraph/export:/dgraph/export
    command: dgraph alpha --my=alpha1:7080 --zero=zero1:5080
    container_name: alpha1

  vault:
    image: hashicorp/vault:${VAULT_VERSION}
    container_name: vault
    ports:
      - 8200:8200
    volumes:
      - ./vault/config.hcl:/vault/config/config.hcl
      - ./vault/data:/vault/data
    environment:
      VAULT_ADDR: http://127.0.0.1:8200
    entrypoint: vault server -config=/vault/config/config.hcl
    cap_add:
      - IPC_LOCK
EOF

Setup Vault Server

The config.hcl will configure the Vault service.

mkdir -p ./vault/data

cat << EOF > ./vault/config.hcl
storage "raft" {
  path    = "/vault/data"
  node_id = "vault1"
}

listener "tcp" {
  address     = "0.0.0.0:8200"
  tls_disable = "true"
}

api_addr      = "http://127.0.0.1:8200"
cluster_addr  = "http://127.0.0.1:8201"
ui            = true
disable_mlock = true
EOF

Setup Dgraph Server

The alpha.yaml will configure the Dgraph database service to utilize Vault to store secrets that are necessary to enable ACLs and encryption on disk.

mkdir -p ./dgraph/{backups,export}

cat << EOF > ./dgraph/alpha.yaml
vault:
  addr: http://vault:8200
  acl_field: hmac_secret_file
  acl_format: raw
  enc_field: enc_key
  enc_format: raw
  path: secret/data/dgraph/alpha
  role_id_file: /dgraph/vault/role_id
  secret_id_file: /dgraph/vault/secret_id
security:
  whitelist: 10.0.0.0/8,172.0.0.0/8,192.168.0.0/16
EOF

Launch Vault

Using Docker Compose, we can start up the Vault server with the following commands.

📓 IMPORTANT: GNU grep needs to be installed for PCRE (Perl Compatible Regular Expressions) with the -oP switch.

## launch vault server
docker compose up --detach "vault"

## configure client access
export VAULT_ADDR="http://127.0.0.1:8200"

## unseal
vault operator init | tee -a unseal.creds
for NUM in {1..3}; do
  vault operator unseal \
    $(grep -oP "(?<=Unseal Key $NUM: ).*" unseal.creds)
done

# export the results for use in other steps
export VAULT_ROOT_TOKEN="$(
  grep -oP "(?<=Initial Root Token: ).*" unseal.creds
)"

Configure Vault AppRole

Below are instructions to setup necessary role identities and access in Vault so that Dgraph can retrieve secrets.

Enable Auth

vault login $VAULT_ROOT_TOKEN
vault auth enable approle

You can run vault auth list to see that approle is enabled:

Create Policies

The policies define access to resources within Vault that can be assigned to roles.

# Dgraph Policy
cat << EOF > ./vault/policy_dgraph.hcl
path "secret/data/dgraph/*" {
  capabilities = [ "read", "update" ]
}
EOF

# Admin Policy
cat << EOF > ./vault/policy_admin.hcl
# kv2 secret/dgraph/*
path "secret/data/dgraph/*" {
   capabilities = [ "create", "read", "update", "delete", "list" ]
}

path "secret/metadata/dgraph/*" {
   capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Mount the AppRole auth method
path "sys/auth/approle" {
  capabilities = [ "create", "read", "update", "delete", "sudo" ]
}

# Configure the AppRole auth method
path "sys/auth/approle/*" {
  capabilities = [ "create", "read", "update", "delete" ]
}

# Create and manage roles
path "auth/approle/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Write ACL policies
path "sys/policies/acl/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}
EOF

# Upload Policies
vault policy write admin ./vault/policy_admin.hcl
vault policy write dgraph ./vault/policy_dgraph.hcl

You can verify the admin policy with running the following:

vault policy read admin

This should show something like the following:

You can verify the dgraph policy the following command:

vault policy read dgraph

This should show something like the following:

Create Role Identities

Run the following to create the needed role identities:

##############
# create admin role
############################
vault write auth/approle/role/admin \
  policies="admin" \
  token_ttl="1h" \
  token_max_ttl="4h"

# verify
vault read auth/approle/role/admin

##############
# create dgraph role
############################
vault write auth/approle/role/dgraph \
  policies="dgraph" \
  token_ttl="1h" \
  token_max_ttl="4h"

# verify
vault read auth/approle/role/dgraph

You can verify the admin role was created the following command:

vault read auth/approle/role/admin

This should show something like the following:

You can verify the dgraph role was created with the following command:

ault read auth/approle/role/dgraph

This should show something like the following:

Configure Vault KV

You can enable KV v2 by using the commands below. This activates a key-value secrets engine that supports secret versioning, which is crucial for accessing previous versions of a secret. This feature is important for restore operations, using an backup from earlier Dgraph clusters that may have been encrypted with an earlier secret.

vault login $VAULT_ROOT_TOKEN
vault secrets enable -path=secret kv-v2

You can run vault secrets list to see that kv is enabled at the path secret/:

Create Secrets

Use the admin role ID to test its privileges for creating secrets, in addition to actually creating the secrets:

##############
# login using admin role
############################
ROLE_ID=$(vault read auth/approle/role/admin/role-id -format=json \
  | jq -r .data.role_id
)

SECRET_ID=$(vault write -f auth/approle/role/admin/secret-id -format=json \
  | jq -r .data.secret_id)

# generate token for admin role
VAULT_ADMIN_TOKEN=$(vault write auth/approle/login \
  role_id="$ROLE_ID" \
  secret_id="$SECRET_ID" \
  --format=json \
  | jq -r .auth.client_token
)

# login using admin token
vault login $VAULT_ADMIN_TOKEN

Now that you are logged in using the admin token, you can test saving the secrets (preferably random secrets) to save into Vault.

##############
# write dgraph secrets using admin role
############################
randpasswd() {
  NUM=${1:-32}

  # macOS scenario
  if [[ $(uname -s) == "Darwin" ]]; then
    perl -pe 'binmode(STDIN, ":bytes"); tr/A-Za-z0-9//dc;' \
      < /dev/urandom | head -c $NUM
  else
    # tested with: GNU/Linux, Cygwin, MSys
    tr -dc 'a-zA-Z0-9' < /dev/urandom | fold -w $NUM | sed 1q
  fi
}

# create using a random secrets
vault kv put secret/dgraph/alpha \
  enc_key="$(randpasswd)" \
  hmac_secret_file="$(randpasswd)"

Read Secrets using Dgraph Role

Now that the secrets have been created, the retrieval needs to be tested using the dgraph role with more restrictive permissions.

##############
# login using dgraph role
############################
ROLE_ID=$(vault read auth/approle/role/dgraph/role-id -format=json \
  | jq -r .data.role_id
)

SECRET_ID=$(vault write -f auth/approle/role/dgraph/secret-id -format=json \
  | jq -r .data.secret_id)

VAULT_DGRAPH_TOKEN=$(vault write auth/approle/login \
  role_id="$ROLE_ID" \
  secret_id="$SECRET_ID" \
  --format=json \
  | jq -r .auth.client_token
)

vault login $VAULT_DGRAPH_TOKEN

Using the Dgraph Token, you can confirm access to the secrets in Vault by executing the command below.

vault kv get secret/dgraph/alpha

This should show something like the following:

Once we confirm that this process is effective, we can securely store the role ID and secret ID for integration with Dgraph’s configuration.

# test access to secrets using dgraph role credentials
if vault kv get secret/dgraph/alpha; then
  # save credentials earlier
  echo $ROLE_ID > ./dgraph/vault_role_id
  echo $SECRET_ID > ./dgraph/vault_secret_id
fi

Launch Dgraph

Now that the necessary secrets have been stored in Vault and we’ve verified access to the credentials using the dgraph role-id, we are ready to launch Dgraph.

Execute the following commands to start Dgraph Zero followed by Dgraph Alpha.

##############
# launch Dgraph Zero
############################
docker compose up --detach "zero1"

##############
# launch Dgraph Alpha if credentials exist
############################
if [[ -f "./dgraph/vault_role_id" && -f "./dgraph/vault_secret_id" ]]; then
  docker compose up --detach "alpha1"
fi

You can verify these settings were applied through the logs with the following command:

# print related key/values from logs 
printf "\n%-30s %s\n" "Key" "Value"
printf "%-30s %s\n" "-----------------------------" "--------"
docker compose logs alpha1 \
  | grep  -oP '(?<=x.WorkerConfig: ){.*' \
  | tr ' ' '\n' \
  | grep -E 'AclEnabled|HmacSecret|EncryptionKey' \
  | sort \
  | tr ':' '\t' \
  | xargs printf "%-30s %s\n"

This should show something like the following.

You can also see the features enabled by querying the health of Dgraph Alpha with the following command:

curl --silent http://localhost:8080/health \
  | jq -r '.[].ee_features | .[]' \
  | sed 's/^/* /' \
  | grep --color --extended-regexp 'acl|encrypt.*|$'

This should show something:

Login into Dgraph

Now that ACLs are activated, Dgraph will need an access token for interaction. You can log in using the root identity with the initial default password or password.

DGRAPH_ADMIN_USER="groot"
DGRAPH_ADMIN_PSWD="password"
export DGRAPH_HTTP="localhost:8080"

export DGRAPH_TOKEN=$(curl --silent \
  --request POST \
  --data "{
    \"userid\": \"$DGRAPH_ADMIN_USER\", 
    \"password\": \"$DGRAPH_ADMIN_PSWD\", 
    \"namespace\": 0 
}" \
  http://$DGRAPH_HTTP/login | grep -oP '(?<=accessJWT":")[^"]*'
)

📓 IMPORTANT: For both production and even testing environments, it’s vital to change the default password for the root identity ‘groot’. Vault can be used to securely store and manage this password. For proper access control, you will want to set up an administrative account by creating a new user and assigning them to the ‘guardians’ group. Additionally, create regular user accounts that lack administrative privileges to further secure system access.

Testing Dgraph

When interacting with Dgraph, you will need to add X-Dgraph-AccessToken to the header with the token that was previously acquired from a login operation. More information regarding Access Control Lists from Dgraph documentation.

Getting Started Tutorial

If you would like to try out some of the basic features, I have a Getting Started Tutorial from a article that I wrote earlier. Even though this article was explicitly written for Kubernetes, the Getting Started Tutorial will work on either Docker and Kubernetes, as the Dgraph is available on localhost.

Dgraph on Kubernetes

When going through this tutorial, and don’t forget to add the access token to the header. For example, here’s one of the queries in that tutorial with the additional header:

curl "$DGRAPH_HTTP/query" --silent --request POST \
  --header "X-Dgraph-AccessToken: $DGRAPH_TOKEN" \
  --header "Content-Type: application/dql" \
  --data $'{ me(func: has(starring)) { name } }' \
  | jq .data

Testing an Export Operation

The export feature will export the whole Dgraph database in RDF or JSON formats. When encyption at rest is enabled, the exported database will be encypted.

For this administrative chore, we will create a GraphQL query.

# Construct export mutation query
cat << EOF > ./dgraph/export.graphql
mutation {
  export(input: { format: "json" }) {
    response {
      message
      code
    }
  }
}
EOF

# Issue Export Operation
curl --silent \
  --header "Content-Type: application/graphql" \
  --header "X-Dgraph-AccessToken: $DGRAPH_TOKEN" \
  --request POST \
  --upload-file ./dgraph/export.graphql \
  http://$DGRAPH_HTTP/admin | jq

This will create an export file structure similar to:

You can find further information regarding these files with:

find ./dgraph/export/ -name '*.gz' \
  | xargs -n 1 file \
  | awk -F/ '{print $NF}'

This should show something like the following if the files were encrypted:

Testing a Backup Operation (Enterprise feature)

The binary backups feature will, as expected, backup the Dgraph database with either full or incremental backups. This can be used to restore the database to a Dgraph cluster. When encyption at rest is enabled, the backups will be encrypted.

# Construct backup mutation query
cat << EOF > ./dgraph/backup.graphql
mutation {
  backup(input: {
    destination: "/dgraph/backups"
    forceFull: true
  }) {
    response {
      message
      code
    }
  }
}
EOF

# Issue Binary backup operation
curl --silent \
  --header "Content-Type: application/graphql" \
  --header "X-Dgraph-AccessToken: $DGRAPH_TOKEN" \
  --request POST \
  --upload-file ./dgraph/backup.graphql \
  http://$DGRAPH_HTTP/admin | jq

When completed, this will create an file structure similar to:

You can get information about the backup files with:

find ./dgraph/backups/ -name '*.backup' \
 | xargs -n 1 file \
 | awk -F/ '{print $NF}'

A result that is SUCCESSFUL will show something like the following:

Addendum: PowerShell Random password

This tutorial is written for using a POSIX shell like bash or zsh. In case a few brave souls prefer to use PowerShell for their adventures, here’s a script below can generate random 32 character alphanumeric.

#!/usr/bin/env powershell
# Credit to Armin Reighter (2021-07-31): 
# * https://arminreiter.com/2021/07/3-ways-to-generate-passwords-in-powershell/
Function Get-RandomPassword ([int]$Length)
{
    Add-Type -AssemblyName System.Web
    $CharSet = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'.ToCharArray()
    $rng = New-Object System.Security.Cryptography.RNGCryptoServiceProvider
    $bytes = New-Object byte[]($Length)
    $rng.GetBytes($bytes)
    $Return = New-Object char[]($Length)
    For ($i = 0 ; $i -lt $Length ; $i++)
    {
        $Return[$i] = $CharSet[$bytes[$i]%$CharSet.Length]
    }
    
    Return (-join $Return)
}

# null coalescing - default first arg
$num = if ($args[0]) { $args[0] } else { 32 }

Get-RandomPassword $num

Cleanup

# remove allocated Docker resources 
docker compose stop && docker compose rm
# remove persisted vault data
rm -rf ./vault/data/*

Previous Article

Previously, I wrote this article in 2021. This article covers using the curl command instead of the vault CLI.

HashiCorp Vault with AppRole Auth

Resources

Source Code

• https://github.com/darkn3rd/blog_tutorials/tree/vault-approle-dgraph-v2024.05.03/vault-docker/approle/dgraph

Vault Tutorials

• Vault policies
• AppRole pull authentication

Conclusion

This tutorial showed how to boost secret security and management in cloud-native setups using Hashicorp Vault and AppRole authentication. Using Docker, we set up a test environment to securely manage secrets in Vault and Dgraph, necessary for enterprise features like access-control-lists and encryption-at-rest.

AppRole is designed for automated systems with minimal human intervention. Combined with the KV secrets engine and randomly generated passwords, it ensures that only administrators and the designated service, Dgraph, can access secrets via Vault login. This method stands in stark contrast to less secure environments where secrets might be stored on developers’ laptops or in insecure locations without audit trails.

Securing these secrets with Dgraph’s features like encryption-at-rest and access-control-lists prevents unauthorized Dgraph access. Even if the database or its backups are accessed externally, the data remains encrypted. The decryption key is securely locked away with Vault, ensuring further protection.

The integration of Dgraph with Vault highlights the importance of secure data management in modern applications, demonstrating that security compliance and operational efficiency are achievable with the right tools and configurations.

Thank you for following this tutorial. We encourage you to keep applying these security practices to your projects and hope you find these guidelines and configurations helpful for your operational needs.