My blog entry for the Azure Advent Calendar, you may be asking what this is?
Thank you for letting me part of this great community event, the YouTube channel is here:- https://www.youtube.com/channel/UCJL9wCcmeMBbah4J0uOWIPg
Using the Twitter Hashtag you can keep up to date with the current Tweets in relation to Azure Advent Calendar:- #azureadventcalendar
As the Title suggests, my video entry is Azure Virtual Machine & Associated Storage Cost Saving Tips, this blog will discuss what I will be talking about on the video along with various links to articles in relation
What shall you be learning from this blog and video?
How to cost save when deploying Azure Virtual Machines, displaying various ways on how to achieve this with costing examples
Suggestions and areas to look at within Virtual Machine deployments within your environment, will be including examples of cost savings and recommended deployment procedures to help you with cost saving within your Virtual Machine environment
Storage deployments associated with Azure Virtual Machines, how can we save when deploying these resources?
On top of Virtual Machine cost saving, looking at associated Storage deployments that you may be deploying along with Virtual Machines, including Blob Storage and Azure Files, will also reference Lifecycle Management as well and how it can assist you in cost saving
Azure related tools to assist with the above
Azure has numerous tooling to assist with the two above scenarios, throughout my video and blog I will be displaying some related tooling to assist you!
Lets dive in and look at suggestions and areas for you to look at within your Virtual Machine deployments to assist in potential cost saving
Lets get bursting, Burst series deployments..
Azure B-series Virtual Machines are an economic series that provides a lower-cost option for workloads that are not usually CPU intensive and can run at a low to moderate CPU baseline.
Be careful! Not all workloads can be converted directly to a B-series deployment without careful consideration into how your application and environment runs. You do need to strike a balance between cost and CPU consumption. B-series VMs are only allocated a baseline amount of CPU power.
B-Series VMs are designed to offer that “burstable” performance when needed, applications deployed to these VMs will leverage flexible CPU usage and as mentioned suitable for workloads that may over time require a spike in their CPU performance, otherwise the workloads run at a long time using only a small portion of the CPUs performance.
|Size||vCPU||Memory: GiB||Temp storage (SSD) GiB||Base CPU Perf of VM||Max CPU Perf of VM||Initial Credits||Credits banked / hour||Max Banked Credits|
As long as its usage is below the baseline, the VM accumulates credits, which can then be used to consume CPU exceeding baseline. If your virtual machine becomes too CPU-intensive, however, it’ll be throttled down to baseline performance until enough credits are available.
Baselines and credit limits vary according to the size of each B-Series VM. Additionally, whenever the VM gets powered down, all accumulated credits are lost. As such, B-Series VMs are truly designed for low-use or predictable workloads that are required to be available at all times.
Costing example of using a Burstable VM
|Virtual Machine: B4MS||Region: UK South|
|Per hour cost:||£0.1528|
|Monthly cost (730 hours)||£148.80|
|Virtual Machine: DS3_v2||Region: UK South|
|Per hour cost:||£0.4382|
|Monthly cost (730 hours)||£357.18|
Burstable VMs are great, although a lot to consider in reference to the above, lets summarise:-
- What is the Baseline vCPU performance? This calculates if your VM is under the vCPU baseline, if so – your burstable credits increase. If you go over the baseline, your credits will decrease
- Peak utilisation – If you cannot bank credits, you will end up in a situation where your VM cannot burst when required, you may need to consider a VM resize
- Powering VMs Off – Do note that you lose any built up CPU credit when this happens
Reserve them Instances
Knowing you will be using the same VM sizes in the coming year or more?
Reserved instances are for you, purchasing one-year or three-year term Azure Reseved instances with a single, upfront payment could save you up to 72% of your VM cost – along with Azure Hybrid Benefit even up to 80%!
As the name indicates Azure Reserved Instances are those VMs that you run, but which you have reserved for long term usage.
The prices for Reserved VM Instances are significantly lower than non-reserved instances and so anytime there is a need to have long running VMs (say you are sure it is going to be used for a year or so) then you should opt to use Azure RIs to gain cost savings.
Significantly reduce costs—up to 72 percent compared to pay-as-you-go prices—with one-year or three-year terms on Windows and Linux virtual machines (VMs).
Costing example below of a 1 year and 3 year Reserved instance potential saving:-
|VM: DS4_v2(2 vCPU 8GB RAM)||Region: UK South|
|per hour cost:||£0.8765|
|Monthly cost (730 hours)||£677.10|
|1 year Reserved per hour cost:||£0.5103|
|1 year Reserved Monthly cost (730 hours)||£409.78|
|3 year Reserved per hour cost:||£0.4334|
|3 year Reserved Monthly cost (730 hours)||£353.63|
Lower your total cost of ownership by combining RIs with pay-as-you-go prices to manage costs across predictable and variable workloads.
Give it a go, reserved instances even have a purchase experience, where Microsoft will make a number of recommendations based on your last 30 days usage and will include potential further savings.
Further documentation in relation to Azure Reserved Instances and the process:-
Azure Hybrid Benefit
Already have Windows Server licenses with software assurance or Windows Server subscription?
You can save up to 40% of your virtual machine cost! •Azure Hybrid benefit calculator: https://azure.microsoft.com/en-gb/pricing/hybrid-benefit/
Can save up to 40 percent on Windows Server virtual machines with Azure Hybrid Benefit for Windows Server.
The Azure Hybrid Benefit is a pricing benefit for customers who have licenses with Software Assurance, which helps maximize the value of existing on-premises Windows Server and/or SQL Server license investments when migrating to Azure.
With this benefit, for each license we will cover the cost of the OS (on up to two virtual machines!), while you just pay for base compute costs.
Eligible customers can save up to 40%* on Azure Virtual Machines (infrastructure as a service, or IaaS), and save up to 55% on Azure SQL Database (platform as a service, or PaaS) and SQL Server on Azure Virtual Machines (IaaS) with Azure Hybrid Benefit, which increases to up to 80% when combined with Azure Reserved Instances.
Azure DevTest Labs
“Fast, easy and lean dev-test environments” – azure.microsoft.com
DevTest labs consists of pre-configured base images or Resource Templates that can be reused. These configurations have all the required setup, tools and software that you require to build a complete environment.
These configurations help developers and testers quickly create environments in Azure for a development and testing perspective of the application. It takes minimal effort to provision Windows & Linux VMs in these DevTest Labs – throughout using reusable templates and artefacts while minimising waste and constantly controlling costs
Features of an Azure DevTest Labs include:
- Simplify cost management
- Quickly setup environments
- Integrate with existing tool chains
- Tailor to your environment
Additionally by using DevTest Labs, you can management costs by numerous configurations including:
- Number of VMs allowed per lab
- Maximum number of VMs allowed per user
- Set VM Size Limits (CPU/Memory)
- VM startup/shutdown times
- Select VM expiry dates
Note: Azure DevTest Labs do not have any additional cost to use this service, you simply pay for the VMs used.
Remembering throughout DevTest labs create once, use multiple times
Deploy to a different region!
May sound drastic but deploying virtual machines into a different region can save money, various Azure regions are cheaper than others for VM deployments. If you are not required for data compliance etc to have your VMs in a specific region(s) then looking at another region is a viable option in relation to cost saving.
Lets look at a costing example of deploying a VM to a different region
|Virtual Machine: D11v2||Region: UK South|
|Per hour cost:||£0.2833|
|Monthly cost (730 hours)||£206.81|
|Virtual Machine: D11v2||Region: East US|
|Per hour cost:||£0.197|
|Monthly cost (730 hours)||£143.81|
Cost saving of more than £60 by selecting a different region – perfect for a development/testing environment where latency and other region related issues may not be effective.
Virtual Machine Up-Time
A simple procedure to implement that can have a big impact on your monthly cost! Have a review of your virtual machines, do they need to be on 24/7?
A quick calculation:-
|Virtual Machine: D11v2||Region: UK South|
|Per hour cost:||£0.2833|
|Monthly cost (730 hours)||£206.81|
If you reviewed this VM’s uptime and decided you only need availability 24 hours a week, this would be: £27.20
Imagine your development environment consisting of 10-20 virtual machines, this would be a great saving!
Of all the strategies and suggestions for saving costs on VMs, shutting down a VM when not needed is the most propitious one
Autoscaling – Virtual Machine Scaleset
An Azure VM scale set can automatically increase or decrease the number of VM instances that run your application. This automated and elastic behaviour reduces the management overhead to monitor and optimise the performance of your application.
You create specific rules for your scale set groups to scale when required, whether than be time specific or performance related such as >60% CPU over X periodic time, scalesets are great!
When any of the thresholds are met that you set, the VM will scale up to an additional VM and continue to do so until the threshold rule is met. Once the rule has expired it will remove any newly added VMs
For autoscaling VMs, you need to ensure your VMs are fully automated
- Consistent scalability
- Can scale on demand
- Provides high availability and application resiliency
Do you need that bigger VM instance? Resize those over-provisioned VMs!
Quite a few times you may create an environment from pre-requisites that contained a VM spec; after a number of months begin to review your VM usage to determine if you really need that VM spec or if a smaller instance is viable. This can drastically save cost over time.
A common issue when creating new cloud resources is figuring out the right size to use. Azure offers a large number of options to accommodate different requirements (more RAM, more CPU power, SSD drives, GPUs, etc.), but even within the same family of machines, choosing the right size matters.
Ways this can be monitored include:
- Your third-party monitoring solution such as Zabbix/Nagios
- Azure Monitor (recommendations)
- View Virtual Machine metrics from the Virtual Machine within Azure Portal
When provisioning VMs for various application workloads it is worth spending time and effort on finding the correct metrics that the application under load generates.
These are the CPU load, IO throughput, memory requirements, etc. It might take a day or two get this information by testing/loading the application prior to migrating it but the end result will save you money in the long running of the environments history.
Using Azure Storage with your VMs
Need a storage account for your build?
Decide on the correct storage-type, will your environment need Geographically Redundant storage? Storage account types, from a replication perspective:-
- LRS – Locally Redundant Storage
- ZRS – Zone Redundant Storage
- GRS – Geographically Redundant storage
- RA-GRS – Read Access GRS
The more replication happens the more expensive the storage, consider this whenever you are deciding on the Storage Account type you will be using.
On top of this, creating your VM with Managed Disks there is various considerations for this as well, including type of Managed Disk (SSD, HDD) along with actual size (127Gb, 512Gb etc) – all will increase the price. Read further on understanding the pricing structure of Managed Disks
Azure Files offers fully managed file shares in the cloud that are accessible via the industry standard Server Messaging Block (SMB) Protocol
Azure file shares can be mounted concurrently by cloud or on-premises deployments of Windows, Linux, and macOS. Additionally, Azure file shares can be cached on Windows Servers with Azure File Sync for fast access near where the data is being used.
- Simplify cloud development
- Replace file server functionality
- SMB Authentication with Azure AD
Consider Azure Files for a potential over your standard VM file servers/SMB shares, I have included Planning an Azure Files Deployment
Azure Storage LifeCycle Management
Azure Blob storage lifecycle management offers a rich, rule-based policy for GPv2 and Blob storage accounts
Data sets have unique lifecycles. Early in the lifecycle, people access some data often. But the need for access drops drastically as the data ages.
Some data stays idle in the cloud and is rarely accessed once stored. Some data expires days or months after creation, while other data sets are actively read and modified throughout their lifetimes.
Azure Blob storage lifecycle management offers a rich, rule-based policy for GPv2 and Blob storage accounts. Use the policy to transition your data to the appropriate access tiers or expire at the end of the data’s lifecycle.
Creating a policy lets you:-
- Delete blobs at the end of their lifecycles
- Create rules to be ran once per day at the Storage Account Level
- Transition blobs to cooler storage after a period (hot to cool, hot to archive or even cool to archive available)
Analyze your spending
Azure has numerous tools to assist you with analyzing and auditing your current spend.
These tools can be used to identify where you’re spending unnecessary and unused resources can be removed or shutdown to save money.
Lets look at some of the tools!
Azure Pricing Calculator
Azure Pricing calculator allows you the ability to pretty much price out your environment, create price estimates and even has available example scenarios of what you may be considering – its great!
Use Azure Policies
Azure Policies are used to create, assign and manage policies. These policies enforce different rules and changes within your Azure Resources, so they stay compliant with you company standards
- VM SKU Size
- VM SKU limitations
- Storage Types
- Resource Location Creation
- VM Tagging
This scope could range from a management group to a resource group. The term scope refers to all the resource groups, subscriptions, or management groups that the policy definition is assigned to.
Policy assignments are inherited by all child resources. This design means that a policy applied to a resource group is also applied to resources in that resource group. However, you can exclude a subscope from the policy assignment.
Azure Advisor helps you optimise your Azure resources for high availability, security, performance and cost by providing personalised recommendations based on your usage and configurations
Can give you various recommendations including:-
- Suggested VMs to resize
- VMs to shutdown
- Cost Utilisation
- Shows estimated Cost Savings
Azure Cost Management is a useful tool to analyse your costs but is only available to Enterprise Agreement customers although it will be available to other customers in the future.
You can also use Cloudyn which provides the base for Azure Cost Management although the majority of the useful features are a paid for service.
If you use a CSP they may be able to provide better options for you to manage the cost of your Azure resources.
What can you do with Cost Analysis?
- Check costs to track your Azure spending
- Create cost alerts when thresholds are met
- Create budgets and spending limits
Auditing your Azure Usage
I would recommend auditing your azure usage on a monthly basis – them few hours to begin with could have quite a cost saving in the coming months.
A good auditing tool is found within your Azure subscription in the portal, viewing this can even show you, your current burn rate which you can potentially see cost savings without any Azure recommendations!
Some auditing recommendations:-
- Ensure you set out your resources in Azure in a common way and use tags so that you can easily manage and report on their use and purpose.
- When it comes to removing resources or reporting on usage this structure becomes invaluable and can result in big cost savings
- Speaking wider than just VMs it can be easy to leave unused resources sitting within your subscription and depending on the subscription size – possibly forgotten about! This can result in additional unnecessary costings.
- Use tags and other strategies to categorize how the asset will be used (dev, test, prod, etc.)
Automation in Azure can be used to monitor, automatically scale-up or down your VM, and shut it down based on criteria such as working hours or workload. The possibilities are infinite.
The possibilities really are endless with what you can achieve with Automation within Azure to assist you with cost saving,
- Runbooks to start/stop VMs, even Azure Firewall if being used
- Update management (automate patching)
- Functions for various assistance with VMs & Blobs, automate resource tagging
- Azure DevOps – CI/CD pipeline for your VMs/environment
Introduction to Azure Automation