Image for post
Image for post

Using Azure Blob Storage

Using Azure Blob Storage with MinIO Azure Gateway

Azure like other cloud providers has support for an object store, called Azure Blob Storage. This solution is similar to S3 or GCS, but unfortunately, most applications out there will not have direct support for Azure Blob Storage.

The Tools

These are the tools required to fully use this tutorial:

Creating Azure Blob Storage

Creating object storage on Azure will involve the following steps:

  • storage account: container for Azure storage objects (names must be globally unique)
  • authorize access: authorize with SAS or Azure Active Directory.
  • container : contains properties, metadata, and zero or more blogs.
  • blob: any single entity comprised of binary data, properties, and metadata.

The Script

Below is a script that will create the required components, and grant access to your current signed-in account access to the newly created storage account. Download the script below as create_blob.sh:

create_blob.sh
export MY_RESOURCE_GROUP=my-superfun-resources
export MY_LOCATION=eastus2
export MY_STORAGE_ACCT=my0new0unique0storage
export MY_CONTAINER_NAME=storage-blob-test
bash create_blob.sh

Upload a File

Create a file and upload it:

touch helloworldaz storage blob upload \
--account-name ${MY_STORAGE_ACCT} \
--container-name ${MY_CONTAINER_NAME} \
--name helloworld \
--file helloworld \
--auth-mode login

Check the Results

Now we can verify that the file(s) exists:

az storage blob list \
--account-name ${MY_STORAGE_ACCT} \
--container-name ${MY_CONTAINER_NAME} \
--auth-mode login | jq '.[].name'

Launching MinIO Azure Gateway

We will use a docker environment managed through the tool Docker Compose to run the following:

Step 1: Build Compose Environment with Storage Credentials

First we need to get extract the AccountName and AccountKey from the connection string. These will be configured values that MinIO Azure Gateway will use to access Azure Blob Storage.

create_env.sh
export MY_RESOURCE_GROUP=my-superfun-resources
export MY_STORAGE_ACCT=my0new0unique0storage
bash create_env.sh

Step 1: Create Docker Compose Configuration

The following docker-compose.yml file is the configuration we can use for this small demo:

docker-compose.yml

Step 2: Create the Client Dockerfile

For the azure-client, we want to install the MinIO client tool as well as s3cmd tool. These can be used to access the gateway using S3 API. Download this Dockerfile:

Dockerfile

Step 3: Create Entrypoint Script

Because the azure-client container needs access to the azure-gateway container when initially starting up, we can use an entrypoint script to inject some values. Download this entrypoint.sh

entrypoint.sh

Step 4: Build and Run

With all of the files above in place, we should have a directory structure that looks like the following:

.
├── create_blob.sh
├── create_env.sh
├── docker-compose.yml
├── Dockerfile
└── entrypoint.sh
docker-compose build
docker-compose up --detach

Step 5: Test with MinIO Client and s3cmd

Using the environment variable we setup before for the container name, we can run the mc command inside the container with the following:

export MY_CONTAINER_NAME=storage-blob-test
docker exec --tty azure-client mc ls myazure/$MY_CONTAINER_NAME
export MY_CONTAINER_NAME=storage-blob-test
docker exec --tty azure-client s3cmd ls s3://$MY_CONTAINER_NAME

Cleaning Up

Docker Environment

When finished, you can remove the locally running docker containers created with Docker Compose using the following:

docker-compose stop && docker-compose rm

Azure Cloud Resources

To delete all the resources we created, you can use this script below as delete_blob.sh:

export MY_RESOURCE_GROUP=my-superfun-resources
export MY_STORAGE_ACCT=my0new0unique0storage
export MY_CONTAINER_NAME=storage-blob-test
bash delete_blob.sh

Resources

Source Code

Azure Docs

MinIO Server

Client Tools

Docker Stuff

Articles

Conclusion

This is my first article on topics related to Azure. I wanted to cover using object-stores this is by far the most often used cloud resource beyond instances (virtual machines) and containers.

Written by

Linux NinjaPants Automation Engineering Mutant — exploring DevOps, Kubernetes, CNI, IAC

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store