Resources to learn Azure Resource Manager (ARM) Language

Azure Resource Manager(ARM) was announced in Spring 2014. It is a completely different way of deploying services on Azure platform. It matters because before the release of ARM it was only possible to deploy one service at a time. When you were deploying applications using PowerShell or Azure CLI you had to deploy all the services via a script. As the number of services increased the scripts got increasingly complex and brittle. Over the past year ARM capabilities have evolved rapidly. All future services will be deployed via ARM cmdlets or templates. The current Azure Service management API’s will be eventually deprecated. Even when using ARM you have two choices:

  • Imperative: This is very similar to how you were using Service Management API’s to provision services.
  • Declarative: Here you define the application configuration with a JSON Template. This template can be parameterized. Once this is done a single PowerShell cmdlet New-AzureResourceGroupDeployment deploys your entire application. This deployment can span regions as well. You can define dependencies between resources and deployment process will deploy them in the order necessary to make the deployment successful. If there are no dependencies it parallelizes the deployment. You can repeatedly deploy the same template and the deployment process is smart enough to determine what changed and only deploy/update the services that changed. ARM templates can not only provision the infrastructure they also also execute tasks inside the provisioned VM’s to fully configure your application. On Windows VM’s you can either use DSC or PowerShell scripts to customize it. On Linux you can use bash scripts to customize the VM after it has been created.

AWS has had a similar capability for many years. It is called CloudFormation. While ARM and CloudFormation are similar and are trying to achieve similar goals there are some differences between them as well.

Resources

If you believe in DevOps and work with Microsoft Azure platform understanding ARM will be beneficial. Another thing worth mentioning is that ARM templates will allow you to deploy services in your private cloud when Azure stack is released. I want to share some helpful resources to make it easier for you to learn ARM.

  1. Treat your Azure Infrastructure as code is an excellent overview of ARM and its benefits: https://www.linkedin.com/pulse/treat-your-azure-infrastructure-code-krishna-venkataraman?trk=prof-post
  2. ARM Language Reference: https://msdn.microsoft.com/en-us/library/azure/Dn835138.aspx?f=255&MSPPError=-2147217396
  3. Azure Quick Start Templates at Github: If you are like me you learn from examples. Here is a large repository of ARM templates. https://github.com/Azure/azure-quickstart-templates
  4. Ryan Jones from Microsoft posted many simple ARM samples here: https://github.com/rjmax/ArmExamples
  5. Full Scale 180 blog is another excellent resource to learn how to write ARM templates.  http://blog.fullscale180.com/building-azure-resource-manager-templates/   I especially like the Couchbase Sample: https://github.com/Azure/azure-quickstart-templates/tree/master/couchbase-on-ubuntu
  6. If you still want to use the imperative method of deploying Azure resource check out this sample for Joe Davies that walks you through the process provisioning a VM here: https://azure.microsoft.com/blog/2015/06/11/step-through-creating-resource-manager-virtual-machine-powershell/
  7. Here is a sample showing how to lock down your resources with Resource Manager Lock. http://blogs.msdn.com/b/cloud_solution_architect/archive/2015/06/18/lock-down-your-azure-resources.aspx
  8. Neil Mackenzie posted a sample for creating a VM with a instance IP address here: https://gist.github.com/nmackenzie/db9a4b7abdee2760dba8 https://onedrive.live.com/view.aspx?resid=96BA3346350A5309!318670&app=OneNote&authkey=!APNWE3DZp1C-RjY
  9. Alexandre Brisebois posted a sample showing how to provision Centos VM using an ARM.  In this example he shows how to customize the VM after its creation using  a bash script. https://alexandrebrisebois.wordpress.com/2015/05/25/create-a-centos-virtual-machine-using-azure-resource-manager-arm/
  10. Kloud Blog has a nice overview of how to get started with ARM and many samples: http://blog.kloud.com.au/tag/azure-resource-manager/
  11. If you want learn about best practices for writing ARM templates this is a must read document. https://azure.microsoft.com/en-us/documentation/articles/best-practices-resource-manager-design-templates/
  12. This blog post shows how you can use output section of the template publish information about newly created resources. http://blogs.msdn.com/b/girishp/archive/2015/06/16/azure-arm-templates-tips-on-using-outputs.aspx
  13. Check out this list of resources for ARM by Hans Vredevoort. It is very comprehensive. https://onedrive.live.com/view.aspx?resid=96BA3346350A5309!318670&app=OneNote&authkey=!APNWE3DZp1C-RjY
  14. This blog post shows how you can use arrays, length function, resource loops, outputs to provision multiple storage accounts http://www.rajinders.com/2015/08/14/adventures-with-azure-resource-manager-part-i/

 

Samples

As I work with ARM templates I am constantly developing or looking for samples that can help me. These sample templates were created by product teams in Microsoft but have not been integrated into Quick Start templates yet. I will use this section to document some of the helpful samples I have found.

  1. Azure Web Site with a Web Job Template: This template was created by David Ebbo. This is the only ARM template sample that shows you how to publish webjobs with an ARM template. https://github.com/davidebbo/AzureWebsitesSamples/blob/master/ARMTemplates/WebAppWithWebJobs.json
  2. Length Function: As I began learning the template language I found it annoying that I had to pass in Array and its length as separate parameters. I just found a sample created by Ryan Jones which shows how to calculate length of an array. https://github.com/rjmax/ArmExamples/blob/master/copySampleWithLength.json

Tools

ARM documentation is still evolving and sometimes it is difficult to find samples you are looking for. If you are trying to create a new template and you cannot find any documentation here are few things that may be helpful

  1. Azure Resource Explorer: This is an essential tool for anybody writing ARM templates. You can deploy a resource using the portal and use the resource explorer to see the JSON schema for the resource you just created. You can make changes to the resources: https://resources.azure.com/
  2. ARM Schemas: This is the location where MSFT ARM teams are posting their schemas. https://github.com/Azure/azure-resource-manager-schemas

Debugging

You can view the logs using these PowerShell cmdlets.

  1. Get-AzureResourceLog: Gets logs for a specific Azure  resource
  2. Get-AzureResourceGroupLog: Get logs for a Azure resource group
  3. Get-AzureResourceProviderLog Gets logs for an Azure resource provider
  4. Get-AzureResourceGroupDeploymentOperation Get logs for the deployment operation

When your template deployment operation fails the error message may not have enough detail to tell you the reason for failure. You can go to the preview azure portal and examine the audit logs. You can filter by resource group, resource type, and time range. I was able to get detailed error message from the portal.

Surprises

In addition to running the cmdlet Switch-AzureMode –Name AzureResourceManager I also had to enable my subscription for specific Azure resource providers. When I was using Service Management API’s this was not necessary. As an example to be able to provision virtual networks with ARM I had to run the following cmdlet:

Register-AzureProvider –ProviderNamespace Microsoft.Network

Even though Template language can work with JSON arrays it cannot determine the number of elements in the JSON array so you have to pass the count separately. on 08/04/2015 I removed the previous line as length function is now available.

I hope these resources are helpful. If you are aware of other helpful ARM resources feel free to mention them in comments on this  blog post and I can add them to my list.

I will be posting ARM samples on my blog as well.

Installing Java Runtime in Azure Cloud Services with Chocolatey

I recently wrote a blog post about installing Splunk on Azure Web/Worker roles with the help of a startup task. You can see that blog post here. In this blog post I will show you how to install Java runtime in web/worker roles. Azure Web/Worker roles are stateless so the only way to install third party software or tweak windows features on web/worker roles is via startup tasks.

Linux users have had the benefit of tools like apt, yum etc to download and install software via command line. Chocolatey provides you with similar functionality on Windows Platform. If you into DevOps and automation on Windows platform you should check out Chocolatey here. It has nearly 15000 packages already available.

Once you have Chocolatey installed installing java is a breeze. It is as simple as

 choco install javaruntime -y  

The statement above is self explanatory. Option –y  answers y to all the questions including accepting the license so you are not prompted to answer any questions.

I already provided detailed steps to define startup tasks in my  previous blog post. So I will just share the startup script along with the service definition file that shows how to deploy Java runtime in Azure web/worker role with a startup task.

Step 1

Create a startup.cmd file and add it to your worker/web role implementation. It should be saved as “Unicode (UTF-8 without signature) – Codepage 65001”.

Set the “copy to output directory” property of startup.cmd to “copy if newer”

Line 9 checks to see if the startup task ran successfully  and end if it did

Line 16 installs chocolatey

Line 22 install java run time

Line 26 only execute if java was installed successfully and it creates startupcomplete.txt file in approot directory.

1:  SET LogPath=%LogFileDirectory%%LogFileName%  
2:     
3:  ECHO Current Role: %RoleName% >> "%LogPath%" 2>&1  
4:  ECHO Current Role Instance: %InstanceId% >> "%LogPath%" 2>&1  
5:  ECHO Current Directory: %CD% >> "%LogPath%" 2>&1  
6:     
7:  ECHO We will first verify if startup has been executed before by checking %RoleRoot%\StartupComplete.txt. >> "%LogPath%" 2>&1  
8:     
9:  IF EXIST "%RoleRoot%\StartupComplete.txt" (  
10:    ECHO Startup has already run, skipping. >> "%LogPath%" 2>&1  
11:    EXIT /B 0  
12:  )  
13:    
14:  Echo Installing Chocolatey >> "%LogPath%" 2>&1  
15:    
16:  @powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin  >> "%LogPath%" 2>&1  
17:    
18:  IF ERRORLEVEL EQU 0 (  
19:    
20:       Echo Installing Java runtime >> "%LogPath%" 2>&1  
21:    
22:       %ALLUSERSPROFILE%\chocolatey\bin\choco install javaruntime -y >> "%LogPath%" 2>&1  
23:    
24:       IF ERRORLEVEL EQU 0 (            
25:                 ECHO Java installed. Startup completed. >> "%LogPath%" 2>&1  
26:                 ECHO Startup completed. >> "%RoleRoot%\StartupComplete.txt" 2>&1  
27:                 EXIT /B 0  
28:       ) ELSE (  
29:            ECHO An error occurred. The ERRORLEVEL = %ERRORLEVEL%. >> "%LogPath%" 2>&1  
30:            EXIT %ERRORLEVEL%  
31:       )  
32:  ) ELSE (  
33:    ECHO An error occurred while install chocolatey The ERRORLEVEL = %ERRORLEVEL%. >> "%LogPath%" 2>&1  
34:    EXIT %ERRORLEVEL%  
35:  )  
36:    

 

Step 2

Update the service definition file to define the startup task.

Lines 5 through 19 define the statup task.

1:  <?xml version="1.0" encoding="utf-8"?>  
2:  <ServiceDefinition name="AzureJavaPaaS" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition" schemaVersion="2015-04.2.6">  
3:   <WorkerRole name="MyWorkerRole" vmsize="Small">  
4:    <Startup>  
5:     <Task commandLine="Startup.cmd" executionContext="elevated" taskType="simple">  
6:      <Environment>  
7:       <Variable name="LogFileName" value="Startup.log" />  
8:       <Variable name="LogFileDirectory">  
9:        <RoleInstanceValue xpath="/RoleEnvironment/CurrentInstance/LocalResources/LocalResource[@name='LogsPath']/@path" />  
10:       </Variable>  
11:       <Variable name="InstanceId">  
12:        <RoleInstanceValue xpath="/RoleEnvironment/CurrentInstance/@id" />  
13:       </Variable>  
14:       <Variable name="RoleName">  
15:        <RoleInstanceValue xpath="/RoleEnvironment/CurrentInstance/@roleName" />  
16:       </Variable>  
17:      </Environment>  
18:     </Task>  
19:    </Startup>  
20:    <ConfigurationSettings>  
21:     <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" />  
22:    </ConfigurationSettings>  
23:    <LocalResources>  
24:     <LocalStorage name="LogsPath" cleanOnRoleRecycle="false" sizeInMB="1024" />  
25:    </LocalResources>  
26:    <Imports>  
27:     <Import moduleName="RemoteAccess" />  
28:     <Import moduleName="RemoteForwarder" />  
29:    </Imports>  
30:   </WorkerRole>  
31:  </ServiceDefinition>  

 

Step 3

Publish the cloud service to Azure. I enabled remote desktop to be able to verify if the worker role was configured successfully.

Verification

I used Remote Desktop to log into the worker role. I  looked in

C:\Resources\Directory\d063631e14c1485cb6c838c8f92cd7c3.MyWorkerRole.LogsPath and found startup.txt

It had the following content. As you can see below that java was installed successfully.

1:  Current Role: MyWorkerRole   
2:  Current Role Instance: MyWorkerRole_IN_0   
3:  Current Directory: E:\approot   
4:  We will first verify if startup has been executed before by checking E:\StartupComplete.txt.   
5:  Installing Chocolatey   
6:  Installing Java runtime   
7:  Chocolatey v0.9.9.8  
8:  Installing the following packages:  
9:  javaruntime  
10:  By installing you accept licenses for the packages.  
11:    
12:  jre8 v8.0.45  
13:   Downloading jre8 32 bit  
14:    from 'http://javadl.sun.com/webapps/download/AutoDL?BundleId=106246'  
15:   Installing jre8...  
16:   jre8 has been installed.  
17:   Downloading jre8 64 bit  
18:    from 'http://javadl.sun.com/webapps/download/AutoDL?BundleId=106248'  
19:   Installing jre8...  
20:   jre8 has been installed.  
21:   PATH environment variable does not have D:\Program Files\Java\jre1.8.0_45\bin in it. Adding...  
22:   The install of jre8 was successful.  
23:    
24:  javaruntime v8.0.40  
25:   The install of javaruntime was successful.  
26:    
27:  Chocolatey installed 2/2 package(s). 0 package(s) failed.  
28:   See the log for details (D:\ProgramData\chocolatey\logs\chocolatey.log).  
29:  Java installed. Startup completed.   
30:    

I also verified that e:\startupcomplete.txt file was created.

I verified that java was installed in D:\Sun\Java directory

You can get the source code for this entire project from my GitHub Repository https://github.com/rajinders/azure-java-paas.