So you are ready to move your application to Azure but it is not fully optimized for the cloud. You looked into App Services (web apps), but to be able to really get the best of them, it would need some definitive improvements in the code. You go back to your team and management and propose them a lift and shift that would make everyone happy. You then plan your improvements in your timeline and estimates. How can you effectively lift and shift your application so that it can be highly available (within one region) and scalable? In this series of posts, I will show you how you can package your application in an image and have it ready for deployment for your IaaS VMs setup through a scale set. I will also show you how you can use that image to create a Virtual Machine Scale Set and use extensions to post provision your virtual machines.

What will we do in this series

I decided to do a series of posts about this topic as it touches a variety of aspects. I will use a concrete example that may or may not have happened to you and I plan to cover

• Building a managed image from an Ubuntu image as base, and setting up a web server to host an application (this post)
• Creating a Virtual Machine Scale Set using ARM templates
• Adding a post provisioning extension that will be fetching files from a blob storage that is required for the application to run once a virtual machine in the set has started

What is Packer?

Packer is a tool that allows us to create machine images in an automated way. With the tool, you can just pass it your configuration, in JSON, and it will generate you a machine image for your platform. Packer supports a variety of platforms, but today we will focus on its Azure counterpart.

Azure Machine Images, which one to embrace?

Virtual machine images in Azure come in 2 flavors: Unmanaged images and Managed images.

A managed image is an Azure resource that comes from a generalized virtual machine image. A generalized image is a snapshot of an already installed Operating System without the machine specific settings and without user’s settings. A managed image is stored on managed disks, which are like a physical disk in an on-premises server but virtualized¹.

Unmanaged images are VHDs or virtual hard-disks images that are not stored on managed disks. VHDs in Azure are stored in storage accounts.

In this series, I will create a Managed image.

Getting started

Lets get started with Packer locally to create the managed Linux image (Ubuntu). You can (and should definitely) automate this process using Azure DevOps with the Packer build task, but this will not be covered here. If there’s some interest about that, let me know and I can definitely write about it. Don’t be shy to use the contact section, or to contact me on twitter!.

Setting up the build with Packer

Go ahead and download yourself a copy of Packer.

Packer uses a JSON template structure to organize its builds, a task that will produce an image for a single platform. I will be using the the following sections of the template:

• Variables. Variables are elements used to store information to be referenced in our template. These variables help with not hard-coding our values in the template for re-use (hint these can be used for a CI build!).
• Builders. Builders are components of Packer that are able to create a machine image for a single platform. In my case, I will use the Azure builder.
• Provisioners. Provisioners are components of Packer that install and configure software within a running machine prior to that machine being turned into a static image.

First I will create a packer.json file, with the content below, that I will use as the template.

An empty packer JSON template for this scenario looks like this:

Defining the variables

As mentioned above, we need to have elements to store our information. Below are the elements/variables we will be needing:

• subscription_id – Subscription under which the build will be performed.
• tenant_id – The Active Directory tenant identifier with which your client_id and subscription_id are associated.
• client_id – The Active Directory service principal (SP) associated with your builder.
• client_secret – The password or secret for your service principal.
  • note: there are also other means of authentication for your SP such as certificate and JWT. Find them here.
• managed_image_prefix – The prefix that will be used used to name the managed image.
• managed_image_resource_group_name – The resource group name where the image will be stored
• ssh_username – The ssh username that packer will use to connect to the machine to run our provisioners. This accounts has admin rights.
• ssh_password – The ssh password that packer will use to connect to the machine to run our provisioners. It is important to set this here, otherwise packer will automatically generate a random password that you won’t have access to. It is important to set it as we will use it in on of our provisioners.
• ApplicationArtifacts – The application artifacts
• WorkingDirectory – The directory where all files required for the provisioners are located.

Creating a SP for use with the builder

I’m a big fan of PowerShell for automation in Azure so I tend to favor PowerShell to create Azure resources. The same can be done with the Azure Portal or Azure CLI.

To create a Service Principal, use the following code snippet

We will set the actual values of these variables in another file that will pass to Packer to validate and build the image.

Create another file named packer.parameters.json and copy the variable object from above. Fill in the values with the values relative to your environment.

Tip

You can instruct packer to read your variables from different places as shown here. If for instance, you want Packer to read environment variables, you can replace a variable with the content {{env `MY_ENV_VAR`}} . Example:

Sensitive variables

You can control the sensitivity of variables so that they aren’t logged to Packer logs and UI by using the sensitive-variables section. In this case, I set the client_secret variable as sensitive. The Packer UI and logs will replace the value of “client_secret” with <sensitive>

Defining our builder

Since I’m building for Azure, I will use the Azure-ARM builder provider by Packer. In the builder, I specify the base image and VM size that Packer will use to create our managed image. I also set the properties of the builder which mainly are taken from our variables. The {{user }} instruct Packer to use our user defined variables.

Also to note that I’m using timestamp as a way to uniquely identify my images. Use whatever you see fit that fits your company / team governance.

Note

If you don’t supply the property build_resource_group_name, Packer will create a new resource group and provision a VM (and its associated resources) within that resource group. To do so, your SP needs to have owner access in the subscription.

If you supply the build_resource_group_name property, your SP needs to have owner access in the resource group specified in that variable.

Defining our provisioners

In this example, I will use 3 provisioners. The last is mandatory to generalize your image. The same applies if you build a managed image based on a windows image.

#1 – Copy the application artifacts file

Provisioner of type file

#2 – Run the setup script to install and configure the OS and application. This includes for instance updating the OS, installing the required software(s) that I need, downloading and installing my web server, setting up permission and users, and so on.

Provisioner of type shell

As you can see here, before running the script (which will be uploaded to the machine with a random name), we execute a command. This command instructs Packer to gain root privileges using sudo. This is why it was necessary to setup a ssh password so that we can pass it to sudo using stdin.

Also if you’re writing your script on windows, make sure it is saved with LF as the line feed and not CRLF. CRLF will not work in Linux. If this is a constraint for you you can also use a dos2unix to convert the file to use proper line feeds.

#3 – Generalize the image so that it can be reused

Provisioner of type shell

The skip_clean was added because Packer reported that customers have reported issues with the deprovision process where the builder hangs. One solution is to set skip_clean to true in the provisioner. This prevents Packer from cleaning up any helper scripts uploaded to the VM during the build.

Final look

Your template should now look like this

Run the process

I am now ready to validate that the template is valid. I do so by running the following command

If everything is successful, I start the image building process

Structure and organize managed images

Azure has a great product for us called Shared Image Gallery (SIG) that can help with structuring and managing managed images. The main features of SIG are:

• Manage global replication of images.
• Version and group images for easier management.
• Make the images highly available with Zone Redundant Storage (ZRS) accounts in regions that support Availability Zones. ZRS offers better resilience against zonal failures.
• Share images across subscriptions, and even between Active Directory (AD) tenants, using RBAC.
• Scale your deployments with image replicas in each region.

Shared Image Gallery is also very appealing for teams and companies, as mentioned by Microsoft, if you have a large number of managed images that you need to maintain and would like to make them available throughout your company / teams. If this is your case, you can use SIG as a repository that makes it easy to share your images.

If any of the above points hits home for you, definitely consider this product.

Adding Shared Image Gallery into our Packer build

If you wish to enable Packer to push to Shared Image Gallery, add the following into the builder object for Azure-ARM

The first thing we need to do is add the following variables to the template:

• shared_image_gallery_name – The name of the SIG resource you created in Azure
• shared_resource_group_name – The resource group in which SIG resides in
• image_name – The image name definition in SIG
• image_version – The version of your image

Once this is done, we need to add the following to the Azure-ARM builder. Be sure to change the replication_regions array with values that make sense for you! This property tells SIG to replicate your image in the regions you define.

Remember however, that before pushing your image, you need to create a definition in SIG otherwise you will get an error when packer will try to push it.

Final words

Microsoft has introduced the Azure Image Builder that is currently in preview. With this service, you will also be able to create managed and unmanaged images like packer for Azure. Considering this product is still in preview, it does come with some limitations. Be sure to read about them before fully embarking into it.

If you want to read more about Packer and creating a windows image instead of a Linux one, you can read about it here.