What is Azure Blob Storage?

What is Azure Blob Storage?

by | Sep 23, 2020 | Azure, Azure Blobs, Blob Storage, Storage Accounts

Azure Blob Storage

Blobs, Blobs and more Blobs.

If you have ever had the need to store large amounts of files and data, then Azure’s Blob Storage is made for you.

Microsoft’s Azure Cloud provides huge benefits with not only their fantastic services, locations, availability and support, but also in their almost seemingly infinite capacity.

Azure Blob Storage is not only scalable, durable and almost always available it also provides flexibility to scale as your business requirements need.

A huge benefit to using Azure services is the pay as you go service model, that allows you to only pay for the services you consume. No more need to over provision local file servers hard drives for expected capacity, with Azure Blob Storage you upload your files to Azure and only pay for the space you need.

Azure Blob Storage

What is Azure Blob Storage?

Azure Blob Storage Overview

Azure Blob Storage (Blob stands for Binary Large Object) is storage provided by Microsoft’s Azure for unstructured data. Perfect for massive amounts of data. Example use cases are as a target for your log or analytics data, or Blob Storage can be used as a backup and archival location, and even things like files, pictures and music files. Basically Azure Blob Storage is a great dumping ground for huge amounts of your data. A Blob is actually a file which is stored in a directory like structure called a Container, then within an Azure Storage Account, and an Azure Resource Group and finally an Azure Subscription.

Access to each Azure Blob is provided by a HTTPS link directly to the Blob itself, meaning you can access the file from anywhere in the world with an internet connection. Obviously if you dont want the data exposed to the whole world, you can lock this down to meet your security needs.

Azure Blob Storage Hierachy


Azure Storage Account Structure

Concept Description
Azure Resource Group
 Used to contain related resources together for a solution. Logical collection of configuration items within Azure. Can contain Virtual Machines, Virtual Networks, and other items.
Azure Storage Accounts
 Top level of Storage Services within Azure. Contains Blobs, Queue Tables, File Shares and Virtual Machine disks. Can be accessed from anywhere in the world with an internet connection. Default limit of 250 Storage Accounts per region and per subscription, with no limits on the number of Azure Storage Containers or Blobs contained within.
Azure Storage Containers
 Used like a folder that contains all your blobs. Unlimited amount of Storage Containers per Storage Account.
Azure Blobs

Any type of file that is unstructured and stored within an Azure Storage Container. Can store large amounts of Blobs within each container, and these could be things like document files, images, and other multimedia.

 

 

Azure Blob Storage Availability

Additionally when creating your Azure Storage Account you can choose how much redundancy / availability for your Azure Blob Storage. While Microsoft provides an SLA on the uptime of your storage, choosing the correct availability of your Azure Storage Account will ensure your Azure Blobs are accessible in the event of failure.

  • LRS – Locally Redundant Storage – Keeps a synchronous copy of your Azure Blobs three times within the same Azure Datacentre. This is the least cost option
  • ZRS – Zone Redundant Storage – Replicates your Azure Blobs synchronous across three Azure Availability Zones in the primary region.
  • GRS – Geo-Redundant Storage– Copies your Azure Blobs synchronous three times within the same Azure Datacentre, then copies the same Azure Blobs asynchronously across to  one other single location in another secondary Azure Region.
  • GZRS – Geo-Zone-Redundant Storage– Copies your Azure Blobs synchronous across three Azure Availability Zones in the primary region, then copies the same Azure Blobs asynchronously across to  one other single location in another secondary Azure Region.

Azure Blob Storage Tiering

Microsoft provides different storage tiering models for the storing of your data.

Each one has a different storage pricing model in Azure (per GB) and access requirements.

  • Hot – Best for data that  is accessed frequently. (most expensive per GB)
  • Cool – Great for data that is accessed infrequently. (not as expensive as Hot, but no where as cheap as the Archive Tier)
  • Archive – Perfect for data that is very rarely accessed. (cheapest per GB)

Azure Blob Storage Types

Azure Blob Storage has three different types

These are;

  • Block Blobs – Perfect for storing documents, text files or even your media files
  • Append Blobs – cannot be modified and requires you to upload a new blob.
  • Page Blobs – are used for storing Azure Virtual Machine disks.

Azure Blob Storage Limitations

Although Azure Blob Storage seems limitless, there are always some technical limitations that you should be aware of.

Some of these limitations are;

  • Service Level Agreement – Microsoft provides an uptime 99.99% on Azure Blob Storage
  • Maximum size of Storage Account – 500TB
  • Maximum size of a Single Blob – 5TB
  • Number of Blocks in a Blob – 50,000 blocks
  • Maximum size of a block – 100MB
  • Minimum size of a block – 64KB
  • Maximum Storage Account Capacity – 5PB
  • Number of Storage Accounts per Subscription – 200
  • Tiering – Only the Hot and Cool Access Tiers can be set at the root of the Storage Container. Archive Tier is per individual Blob. (you can select multiple files using Cloud Storage Manager to change to the Archive Tier. This isnt possible using the Azure Portal)
  • Archive Tier stores the data offline. This requires time to retrieve the Blob from the offline storage. There is additional cost to retrieve this data and the retrieval time can take several hours.
  • Archive Tier Rehydration – When a Blob is in the Archive Tier the data cannot be modified as it is actually offline. To modify a Blob in this Tier you would first need to rehydrate the Blob to an Online Tier (Hot or Cool)

Azure Blob Storage Unstructured Data

Azure Blob Storage Configurations and Options

 

Azure Blob Storage Pricing

 

As with anything cloud-based, be careful which options you select as there will be cost impacts. As an example, Locally Redundant Storage is the cheapest availability option for Microsoft’s Azure Blob Storage, whereas Geo Zone Redundant Storage provides the highest Availability, but costs significantly more.

For Azure Blob Tiering, the Archive Tier is the cheapest at almost a tenth of the cost of Cool and Hot Tiers..

Tiering LRS – Locally Redundant Storage ZRS – Zone Redundant Storage GRS – Geo-Redundant Storage RA-GRS – Read Access Geo-Redundant Storage GZRS – Geo Zone Redundant Storage RA-GZRS – Read Access Geo Zone Redundant Storage
Blob Storage – Archive Tier 0.03 cents for 10GB Option not available for Archive Tier 0.07 cents for 10GB 0.07 cents for 10GB Option not available for Archive Tier Option not available for Archive Tier
Blob Storage – Cool Tier 0.21 cents for 10GB 0.26 cents for 10GB 0.41 cents for 10GB 0.53 cents for 10GB 0.48 cents for 10GB 0.60 cents for 10GB
Blob Storage – Hot Tier 0.28 cents for 10GB 0.35 cents for 10GB 0.56 cents for 10GB 0.70 cents for 10GB 0.66 cents for 10GB 0.82 cents for 10GB

Now while the price does fluctutate at times, and even differentiates between the different Microsoft Azure Datacentres, the pricing table above has been given as a reference to the differences in costs and options on your Azure Storage.

To work out how much the cost differences, you can use the Microsoft Azure Pricing Calculator to work out the cost impacts to your solution

Azure Blob Storage Best Practices

Azure Blob Storage is a highly scalable, durable, and cost-effective object storage solution from Microsoft Azure. It is a great option for storing unstructured data, such as text and binary data, in the cloud. To get the most out of Azure Blob Storage, it’s important to follow some best practices. In this article, we will discuss the key best practices for Azure Blob Storage.

Use appropriate storage tiers:

Azure Blob Storage offers three storage tiers: Hot, Cool, and Archive. Hot storage is optimized for frequent access to data, Cool storage is optimized for infrequent access, and Archive storage is optimized for long-term data retention. Choose the right storage tier based on your data access patterns and the costs associated with each tier.

Enable versioning:

Versioning allows you to keep multiple versions of the same blob, so you can easily recover from accidental deletions or updates. To enable versioning, you can use the Azure portal, Azure CLI, or Azure Storage REST API.

Use a content delivery network (CDN):

A CDN can help distribute your blobs globally and improve the performance and responsiveness of your applications. You can configure a CDN for your Blob Storage account by using the Azure portal or Azure CLI.

Use shared access signatures (SAS) wisely:

SAS is a secure way to grant access to your blobs without exposing your storage account key. However, it’s important to limit the scope of access granted by the SAS and to set an appropriate expiration time.

Enable encryption:

Azure Blob Storage supports encryption of data at rest using Azure Storage Service Encryption. This feature encrypts your data before it is written to disk and automatically decrypts it when you access it.

Use Azure Blob Storage events:

Azure Blob Storage events allow you to trigger serverless functions or logic apps when specific events occur in your storage account. You can use events to implement complex workflows or integrate with other Azure services.

Monitor and manage your storage account:

Regularly monitor the usage and performance of your storage account using Azure Monitor. You can set up alerts to receive notifications when certain thresholds are met, and you can also use Azure Policy to enforce policies and control access to your storage account.

Monitor usage and storage consumption:

Reduce cost by using Cloud Storage Manager to gain insights in to your cloud storage consumption.

Use the right tools for the job:

Azure Blob Storage provides a number of tools and SDKs for working with your blobs, including Azure Portal, Azure CLI, and Azure Storage REST API as well as our own Cloud Storage Manager. Choose the tool that best fits your needs and make sure to follow the best practices for each tool.

In conclusion, following these best practices can help you get the most out of Azure Blob Storage and ensure that your data is secure, scalable, and accessible. Whether you’re storing unstructured data or building applications that rely on Blob Storage, following these best practices can help you maximize your investment and minimize risks.

Do you want information on all your Blobs in Azure?

Azure Blob Storage Insights

Now that you have some background in to what Azure Blob Storage can do for you and are ready to take the jump and start uploading huge amounts of data to your Azure Storage Containers.

Cloud Storage Manager provides you with further insights in to your Azure consumption. Providing you with quick to see capacity information, searching through all your Blob Storage, as well as historical reporting of your Azure Storage Accounts consumption.

If you want to see exactly what is in your Azure Storage Accounts, download Cloud Storage Manager and test it for yourself for free.


Cloud Storage Manager Main Window

Azure Blob Storage Architecture

Azure Blob Storage is designed with a distributed architecture that provides high availability, durability, and scalability. The architecture comprises three layers:

Front-End Layer

The front-end layer handles incoming requests from clients and routes them to the appropriate back-end nodes.

Back-End Layer

The back-end layer consists of multiple storage nodes that store the data in a distributed manner. The data is stored in a redundant manner to ensure high availability and durability.

Blob Service Layer

The Blob Service Layer provides the APIs and SDKs for accessing the Blob Storage service. It also provides features such as authentication and authorization, metadata, and access control for Blob Storage.

How to view the size of all your Azure Storage Accounts

How to view the size of all your Azure Storage Accounts

by | Sep 17, 2020 | Azure, Azure Blobs, Cloud Computing, Storage Accounts

Why you should monitor your Azure Storage

Monitoring Azure storage usage is important for businesses because of the cost implications and the impact it can have on operations.

  1. Cost implications: Azure storage is a pay-as-you-go service, meaning businesses are charged based on their usage of storage, data transfer, and the number of storage transactions. Unmonitored storage usage can quickly lead to unexpected and unnecessarily high charges. Monitoring Azure storage usage helps businesses stay aware of their storage consumption and avoid overspending on storage costs.
  2. Impact on operations: Azure storage is a critical component of many business operations and applications. If storage usage is not properly monitored, it can result in a shortage of storage capacity, slow performance, and disruptions to operations. Monitoring storage usage can help prevent these issues by alerting businesses when their storage capacity is reaching its limit and allowing them to take action to prevent outages.

In summary, monitoring Azure storage usage is important for businesses to control costs, ensure adequate storage capacity, and avoid disruptions to operations. By monitoring storage usage, businesses can make informed decisions about their storage requirements and take proactive steps to manage costs and maintain optimal performance.

Do you really know how much Azure Storage you are using?

As more and more organisations dive into Azure, storage consumption continues to grow and so does the cost to businesses. Azure offers almost a limitless supply of storage, but this comes at a cost so it’s important to know exactly where that storage is going.

There are a few ways to get this information, either via the Azure portal, Azure Storage Explorer or even PowerShell. However, all these tools have limitations and to be honest, rather lacking and not that straightforward.

So first I’ll show you how to get this information using the Azure portal and Azure Storage Explorer and then show you how you can get a better picture and experience using our FREE TOOL, Cloud Storage Manager.

How to get the size of all Storage Accounts using the Azure portal

  1. Log into the Azure portal
  2. Select Monitor from the left hand panel or use the search bar at the top
  3. Select Storage Accounts from the left hand pane under the Insights section
  4. Use the Subscriptions drop down list and select which Subscriptions you want to view
  5. Select Capacity on the right pane next to Overview
  6. Use the Storage Accounts drop down list to select which storage accounts you are interested in viewing, or select All.

Here you will see the consumed storage for each of your selected subscriptions and storage accounts.


Azure Storage Accounts

Azure Storage Capacity View

This view will provide you with a quick overview of the capacity used for each storage account. Although this will tell you how much storage is being consumed, you cannot drill down to see what is consuming the storage. To go any further you need to jump into the Storage Account view, select Containers and then blindly browse each container looking for blobs and in particular the size of each blob to find out where this storage is actually going.


Browsing your Azure Blobs using Azure Storage Explorer.

Another option is to use Azure Storage Explorer which is now built into the Azure portal (preview).

  1. Log into the Azure portal
  2. Select or search for Storage Accounts
  3. Select the Storage Account you are interested in
  4. Select Storage Explorer on the blade on the right

From with the Azure Storage Explorer Application,  you can browse the containers of that storage account and view the size of each blob inside each container.


Azure Storage Explorer

Finding more indepth information on your Azure Blob Usage

However, you are in the same boat as before, blindly searching through Containers looking for the size of blobs not knowing where they are. This may not be an issue if you only have 2 or 3 containers with 10 or less blobs in each. But what if you have 10, 50 or more Containers and hundreds or thousands of blobs?

How would you know which Container or blob is consuming the space? In short, you can’t, at least not easily.

Even with the full blown Azure Storage Explorer, the task is quite tedious because you have to browse each individual Container in each Storage Account and get the statistics to calculate the storage consumed. That is not a fun job.

Azure Storage Consumption Overview

This is where Cloud Storage Manager gives you the easiest, quickest and best experience by far.

With Cloud Storage Manager you can instantly see how much storage each Subscription, Azure Storage Account, Container and blob is consuming. It provides you with a simple explorer like view to browse through each level.

Here you can immediately see how much storage is being consumed at every level.


Azure Storage Tree View

Azure Container Sizes

And by selecting the Container View tab on the right hand pane also gives you an instant view to see how much storage each Container in Azure is consuming.

A percentage column outlines the overall size of that container relative to its parent Storage Account.


Azure Container Contents

You can also right click on the Container you are interested in and select Jump to Container to expand and select that Azure Container on the tree pane.


Azure Blob Sizes

As you can see, the Container has been selected on the Storage Tree pane on the left. And on the right you can select the BLOB view tab to see the blobs in that container and of course the size of each blob. The Blob view is sorted based on size, so if even if you have hundreds or thousands of blobs, you will see which ones are taking up the most space first.


Cloud Storage Manager Blobs Tab

Azure Blob Information

You can also right click on the Blob to get additional options, such as delete the blob, change the object tier or view the blob properties.


Cloud Storage Manager Azure Storage Blobs Tab

As you can see, Cloud Storage Manager is easily the quickest and simplest way to know where your Azure storage is being consumed, and its all for FREE.

Cloud Storage Manager

Gain insights into your Azure Storage with our FREE to use Tool.
FREE DOWNLOAD

Send download link to:

I confirm that I have read and agree to the End User License Agreement.

Download the FREE TOOL and test it for yourself.

Conclusion. Which tool is right for you?

Azure portal is a web-based interface provided by Microsoft Azure to manage cloud resources and services. Although it provides a user-friendly interface for managing storage accounts, it has some limitations. The Azure portal may not provide an intuitive way to manage large amounts of data, as it requires multiple clicks and navigation to access information, and can be slow when working with a large amount of data. Additionally, the Azure portal does not provide the ability to compare data across storage accounts or view data usage trends over time.

Azure Storage Explorer is a standalone app that enables users to manage their Azure storage accounts. While it provides a more comprehensive view of storage accounts compared to the Azure portal, it still has limitations. For example, it does not provide the ability to monitor data usage in real-time, or set alerts for when storage thresholds are reached. Additionally, it does not provide the ability to automate data management tasks such as moving data to lower-cost storage tiers, which can result in higher storage costs for users.

Cloud Storage Manager is a FREE tool that provides a unified view of all cloud storage resources and enables users to manage, monitor, and optimize their cloud storage usage. It provides monitoring of data usage, and also provides a cost-optimization reports that allows users to identify and eliminate unnecessary data, reducing overall storage costs. Additionally, it provides a unified view of data across multiple storage accounts, enabling users to make informed decisions about storage utilization and cost optimization.

In conclusion, while both the Azure portal and Azure Storage Explorer have their uses, Cloud Storage Manager provides a more comprehensive and cost-effective solution for managing cloud storage resources. With its real-time monitoring, alerts, and automation features, it can help users reduce storage costs and improve the overall efficiency of their cloud storage operations.

How to create an Azure storage lifecycle management policy

How to create an Azure storage lifecycle management policy

by | Aug 25, 2020 | Azure, Azure Blobs, Blob Storage, Cloud Computing, How To, Storage Accounts

How to create an Azure storage lifecycle management policy​

Whether you are using our Cloud Storage Management software to gain insights into your Azure storage environment, or are just trying to work out how to save costs within Azure, creating a lifecycle management policy is a great idea to help you save in your Azure storage costs.

Why is an Azure Lifecycle Management Policy important?

Azure Storage Lifecycle Management is a feature provided by Microsoft Azure that helps users manage the lifecycle of their data stored in Azure Blob storage. It allows users to transition their data to different storage tiers (Hot, Cool, Archive) based on their data access patterns and save costs in their Azure storage environment. The storage tiers have different costs per gigabyte of data, with the Hot tier being the most expensive and the Archive tier having the most cost savings. It is important because it enables users to save costs on their storage and manage their data effectively based on their business needs. Additionally, it helps ensure that the data is stored in the appropriate tier for its intended usage, improving performance and reducing costs.

Azure Storage Tiering Overview

Azure has three different tiers for your blob storage. These storage tiers are;

Hot – Used for frequently accessed data. Best suited for data that your user base accesses daily, think files and photos etc

Cool – Used for infrequently accessed data. Well suited for data that maybe accessed, but not that often.

Archive – Used for rarely accessed data, like backups or data that you need to keep for historical reasons.

Each of these Storage Tiers has a cost associated that Microsoft will charge you per gigabyte of data. The Hot Tier obviously being the most expensive, the Cool Tier is a little cheaper and the Archive Tier having considerable cost savings.

As an example at the time of writing this page, the cost per gigabyte in US dollars for each Tier is as below. (this may vary depending on your agreement with Microsoft)

Azure Blob Storage Costs

PREMIUM HOT COOL ARCHIVE
First 50 terabyte (TB) / month $0.15 per GB $0.0184 per GB $0.01 per GB $0.00099 per GB
Next 450 TB/month $0.15 per GB $0.0177 per GB $0.01 per GB $0.00099 per GB
Over 500 TB/month $0.15 per GB $0.0170 per GB $0.01 per GB $0.00099 per GB

As the table above shows, there are considerable savings when you move your blobs down to the lower tiers and creating an Azure Storage Lifecycle Management Policy.

Azure Blob Storage Tiering

Change your Storage Tier

Microsoft Azure provides tiering for your blob data, that you can set as the default level. (either upon creation of the storage account or at a later date). To check the default storage tiering of your storage account go to the Azure Portal, choose configuration, and then the access tier that the blobs default to in that storage account is shown.

It must be noted that only the Hot and Cool tiers can be set as the default and not the Archive tier.


Azure Storage Tiering

What are some of the benefits of creating an Azure Storage Lifecycle Management Policy?

OK, so now that you see there are some real benefits in changing the tiering of your blob storage, how do I create one you ask?

Well first off let’s look at what you will need to make sure is in place first.

Tiering of blob object storage is only available in Blob Storage and General Purpose v2 (or GPv2) accounts. If you have GPv1 storage you will need to convert that first to GPv2.

Premium storage does not provide any tiering, as this tier is for fast access using SSD based drives. (this maybe coming at a later date)

Changing tiers of storage may incur increased costs. Be very careful when applying the change to your data, as rehydrating blobs from the archive tier can be costly.

How to create your first Azure Storage Lifecycle Management Policy.

Open the Azure Portal

In your Azure portal, go to your storage account that you want the lifecycle policy to apply to and then choose Lifecycle Management.


Azure Lifecycle Management

Create a Azure Storage Lifecycle Policy Rule

 Once the right hand side of your browser has populated, choose Add Rule to start the wizard


Azure Lifecycle Management Rule

Add Lifecycle Policy Rule

Now that the new rule has shown up we need to fill in a few details. You will need to give the rule a Name and then choose what you want to happen with your object data.

As an example I have shown in the below rule that the blobs will move to cool storage after not being accessed in 90 days, then to archive storage in 180 days, then finally being deleted in 365 days.

If you are happy with what you have set, just click Review + add and Azure will go on to apply those settings to your storage account, or if you want to be granular and exclude some containers / paths then click on Next: Filter Set.


Azure Lifecycle Management New Rule

Azure Storage Lifecycle Policy Exclusions

On this page you can now exclude any containers or paths that you do not want this policy to apply to. Click Next: Review + add. 

Azure Lifecycle Management Filter

Azure Storage Lifecycle Validation

 If all goes well you should be presented with a screen as below, saying that your Validation Passed. 

Click on Add and Azure will now apply those settings to your storage account.

Azure will now go through all your Blobs and set them to the tiering and settings you have specified. 

You have successfully created an Azure Storage Lifecycle Management Policy

Azure Lifecycle Management Validation

Reduce your Azure Blob Storage Costs

Now you may ask, how do I know how much storage I’ve consumed or when were my blob files last accessed?

Easy. First run and install our Cloud Storage Manager software, then let it run a scan against your Azure Storage environments. Once the Scan has completed you can then run one of the many reports to understand and optimise your Azure Blob Storage.

Download a Free Trial and test it for yourself.

Free

Cloud Storage Manager Icon

Maximum Azure Storage limited to 30TB.

Typically for small or personal environments usually consisting of 3 or less Azure Subscriptions and consuming under 30TB of Azure Blob Storage.

Free Forever (until your Azure storage goes over 30TB).

Advanced

Cloud Storage Manager Icon

Maximum Azure Storage limited to 1PB

For medium sized environments typically consisting of less than 5 Azure Subscriptions.

Yearly license subscription of $500 USD per year which includes updates and support.

Enterprise

Cloud Storage Manager Icon

Unlimited Azure Storage.

For use in large environments typically consisting of more than 10 Subscriptions and consuming more than 1PB of Azure Blob Storage.

Yearly license subscription of $1000 USD per year which includes updates and support.

Cloud Storage Manager is licensed based on the size of your Azure Subscriptions, Azure Storage Accounts, Containers and finally each Blob.

Each version has the same great functions including scheduled scans of your Azure Blob Storage and reporting.

FREE DOWNLOAD

Send download link to:

I confirm that I have read and agree to the End User License Agreement.

FAQ for Azure Lifecycle Management

What is Azure Storage Lifecycle Management?

Azure Storage Lifecycle Management is a feature that allows users to automate the transition of their data to different storage tiers or classes based on the data’s age or access patterns.

How does Azure Storage Lifecycle Management help in reducing costs?

By automatically moving data to the appropriate storage tier based on its age or access patterns, Azure Storage Lifecycle Management helps to reduce storage costs by ensuring that you are only paying for the most expensive storage tier that you actually need.

Can I still access my data after it has been transitioned to a different storage tier?

Yes, you can still access your data even after it has been transitioned to a different storage tier. The only difference is the retrieval time, which may be slower for data stored in the Archive tier compared to the Hot and Cool tiers.

Can I revert a transition made by Azure Storage Lifecycle Management?

Yes, you can revert a transition made by Azure Storage Lifecycle Management, but you may incur additional charges for moving the data back to a more expensive storage tier.

Is Azure Storage Lifecycle Management available for all Azure storage services?

Currently, Azure Storage Lifecycle Management is available for Azure Blob storage.

What are the different storage tiers that can be managed by Azure Storage Lifecycle Management?

Azure Storage Lifecycle Management allows you to manage data across four storage tiers: hot, cool, archive, and deleted. The hot tier is for frequently accessed data, the cool tier is for infrequently accessed data, the archive tier is for rarely accessed data, and the deleted tier is for data that has been marked for deletion.

How does Azure Storage Lifecycle Management work with data protection?

Azure Storage Lifecycle Management integrates with Azure data protection features such as Azure Backup and Azure Site Recovery, to ensure that your data is protected even as it transitions between storage tiers.

Can I customize the transition policies for my data in Azure Storage Lifecycle Management?

Yes, you can create custom transition policies in Azure Storage Lifecycle Management that are specific to your data and your business requirements. You can specify the time-based or usage-based triggers for data transitions, and you can also set rules for data retention.

Can I track the data movement and monitor the performance of my storage infrastructure with Azure Storage Lifecycle Management?

Yes, you can use Azure Storage Lifecycle Management to monitor and track the data movement in your storage infrastructure, as well as to measure the performance of your storage tiers. You can also use Azure Monitor to set up alerts and notifications for specific events, such as data movement or storage tier changes.

Is Azure Storage Lifecycle Management supported for all types of data in Azure Storage?

Azure Storage Lifecycle Management is supported for all types of data in Azure Blob Storage, including block blobs, append blobs, and page blobs. It is not currently supported for other types of data in Azure Storage, such as files and queues.

Which storage account or storage accounts can you use lifecycle management?

The Storage Accounts that support Lifecycle Management Policies are Blob Storage Accounts that have block blobs and append blobs in general-purpose v2 and premium block blobs.

How to schedule a snapshot on multiple Virtual Machines.

How to schedule a snapshot on multiple Virtual Machines.

by | Sep 8, 2019 | How To, Microsoft HyperV, Snapshot / Checkpoint, VMWare

How to schedule the snapshot of multiple VMware or Hyper-V Virtual Machines with auto deletion.

Every so often you need to schedule a snapshot on a virtual machine, whether that be a VMware virtual machine or a Hyper-V virtual machine, or even multiple virtual machines because someone or something is going to be changed and you need a quick way to roll them back to a running state, just in case things go belly up. It’s a very common task that administrators need to do. Other times you need to create a snapshot on a schedule, whether it’s a one-off snapshot at a specific date and time, or a recurring one that happens every week or month. Doing this using the standard vCenter console is fairly simple and easy to do. But how do you schedule it for multiple VMs? And to then also automatically remove the snapshot? Or what if you have a combination of VMware and Hyper-V? Doing that usually involves a combination of scheduled tasks via the console and PowerShell with PowerCLI.

But the easiest and simplest way by far is to use Snapshot Master. Hopefully you have installed and are running Snapshot Master in your environment, if not, you can download and request a 30 Day trial key and try it out for yourself.

Creating a schedule to snapshot one or multiple virtual machines and then automatically deleting the snapshot after a period you define is a walk in the park.

OK, so now that I’ve talked up Snapshot Master and how easy it is to create a scheduled snapshot on one or multiple VMs either VMware or Hyper-V with auto-deleting the snapshot, how do you actually do it?

Snapshot Scheduling Use Cases

Use Case Why Schedule a Snapshot?

Testing and Development

 To quickly roll back changes made to a virtual machine during testing and development.

Software Upgrades

 To create a snapshot before performing a software upgrade, in case the upgrade causes issues that need to be rolled back.

Disaster Recovery

 To have a recent snapshot available in the event of a disaster or data loss.

Compliance

 To maintain compliance with data retention policies or audit requirements.

Resource Optimization

 To free up disk space on a virtual machine by automatically deleting old snapshots after a specified time period.

Performance Tuning

 To create a snapshot before making changes to a virtual machine’s configuration or performance settings, in case the changes negatively impact performance.

By scheduling snapshots, you can ensure that you have a recent backup of your virtual machines that can be easily restored in the event of issues or data loss. Additionally, scheduling snapshots can help you meet compliance requirements, optimize resource usage, and improve the overall performance of your virtual environment.

Launch Snapshot Master

The first step is to open Snapshot Master. Ensure that you have already installed Snapshot Master on your system. If not, you can download it and request a 30-day trial key to test it out.

Snapshot auto delete 1

Select the Virtual Machines you want to Schedule a Snapshot

Once you launch Snapshot Master, select the virtual machines you want to snapshot. You can do this by either placing a tick in the checkbox or double-clicking the row.

Snapshot auto delete 2

Take a Virtual Machine Snapshot

Right-click anywhere on the screen, move to Selected VMs, and click Take Snapshot. A dialog box will appear to give the job a name and description. Tick the box to automatically delete the snapshot and use the drop-down list to select the number of days after which it should be deleted, then press OK.

Snapshot auto delete 3

VM Snapshot Name

Give the job a name and description. Tick the box to automatically delete the snapshot and use the drop down list to select the number of days it should be deleted in and press OK.

Snapshot auto delete 4

VM Snapshot Execute Now or Scheduling

 Click Schedule job.

Note, if you need to run a pre or post script along with this job, you may call it here.

Snapshot auto delete 5

Schedule VM Snapshot Creation

Click New and setup the schedule for this job.

Press OK when done.

SnapShot Master will now start the snapshot / checkpoint creation process, at the schedule you have created.

Snapshot auto delete 6Snapshot auto delete 7

Scheduled VM Snapshot Job Notification

You will receive a notification letting you know the job has been scheduled.

Snapshot auto delete 8

Scheduled VM Snapshot Email Alerts

That’s all there is to it. Doesn’t get much easier than that. And if you have setup email notifications, you will receive an email detailing the success and failures of the job.

Snapshot auto delete 9

SnapShot Master FAQs

Question Answer

What is Snapshot Master?

 Snapshot Master is a software tool that simplifies the process of creating and managing snapshots of VMware and Hyper-V virtual machines.

How does Snapshot Master work?

 Snapshot Master provides a user-friendly interface for scheduling and creating snapshots of virtual machines, as well as automatically deleting old snapshots after a specified time period. It also supports pre- and post-snapshot scripts for more advanced customization.

Can Snapshot Master handle both VMware and Hyper-V virtual machines?

 Yes, Snapshot Master can create and manage snapshots for both VMware and Hyper-V virtual machines.

Is Snapshot Master easy to use?

 Yes, Snapshot Master is designed to be simple and user-friendly, with an intuitive interface that makes it easy to schedule and create snapshots of virtual machines.

Can I try Snapshot Master before purchasing it?

 Yes, a free 30-day trial version of Snapshot Master is available for download.

What are the system requirements for Snapshot Master?

 Snapshot Master requires Windows 7 or higher and .NET Framework 4.5 or higher. It also requires administrative privileges to install and run.

Does Snapshot Master provide email notifications?

 Yes, Snapshot Master can send email notifications to inform you of the success or failure of snapshot jobs.

Can Snapshot Master automate the snapshot creation and deletion process?

 Yes, Snapshot Master can automatically create and delete snapshots on a schedule that you define.

What other features does Snapshot Master offer?

 Snapshot Master also includes the ability to clone virtual machines, and provide email reports of your virtual environment.

Automate VM Snapshot creation and deletions

Taking snapshots on multiple VMs has never been easier than with Snapshot Master. It’s a simple, automated process that helps to ensure the protection of your virtual machines. Snapshot Master is also capable of managing other aspects of your virtual machines. You can download a free fully functioning trial of Snapshot Master to explore its capabilities.

Free Fully Functioning Trial available to download.

Step by step guide on how to install Microsoft’s System Centre Virtual Machine Manager (SCVMM)

Step by step guide on how to install Microsoft’s System Centre Virtual Machine Manager (SCVMM)

by | Aug 30, 2019 | How To, Microsoft HyperV

How to install Microsoft’s System Centre Virtual Machine Manager (SCVMM)


scvmm install

System Centre Virtual Machine Manager (SCVMM)

 If you run Microsoft’s Hyper-V as your virtualisation platform rather than the ever popular VMware ESX then you probably want a way to manage your Hyper-V hosts as well as the virtual machines that reside on this. Well you can do a lot of the functions that SCVMM does with the free tool for managing Hyper-V, the Hyper-V Manager (which comes as part of the Windows Operating System).

So if they are similar why would we use System Centre VMM over and above the Hyper-V Manager you ask and have to pay more money to Microsoft?

Well I list out the main differences here to hopefully guide you to a decision before going through the installation process of SCVMM later on down the page.

SCVMM vs Hyper-V Manager Key Differences

Hyper-v Manager

  • Can manage the Hyper-V hosts and the virtual machines that reside on it.
  • Can manage a single host or multiple Hyper-V hosts that are members of a failover cluster.
  • Requires Failover Cluster manager to manage multiple hosts.
  • Requires powershell to setup affinity rules.
  • No single pane of glass to manage your complete Hyper-V environment which requires you to utilise the Hyper-V Manager console, Failover Cluster console and then powershell to do only some of what SCVMM can do.

System Center Virtual Machine Manager

SCVMM can do all the Hyper-V Manager does from a single console.;

  • Configure and manage your datacentre components as a single fabric in SCVMM. (Datacentre components include virtualization servers, networking components, and storage resources.)
  • SCVMM can add, provision, and manage Hyper-V and VMware virtualization hosts and clusters. SCVMM can manage VMware ESX servers and its virtual machines in limited capacity. It does however require you to have a VMware vCentre to connect to, to be able to manage your VMware virtual machines.
  • Add networking resources to the SCVMM fabric, including network sites defined by IP subnets, virtual LANs (VLANs), logical switches, static IP address and MAC pools.
    SCVMM provides network virtualization, including support for creating and manage virtual networks and network gateways. Network virtualization allows multiple tenants to have isolated networks and their own IP address ranges for increased privacy and security.
    Using gateways, VMs on virtual networks can connect to physical networks in the same site or in different locations.
  • SCVMM can discover, classify, provision, allocate, and assign local and remote storage.
    SCVMM supports block storage (fibre channel, iSCSI, and Serial Attached SCSI (SAS) storage area networks (SANs)).
  • The SCVMM fabric retains a library of file-based and non file-based resources that are used to create and deploy VMs and services on virtualization hosts.
    File-based resources include virtual hard disks, ISO images, and scripts.
    Non file-based resources include templates and profiles that are used to standardize the creation of VMs. Library resources are accessed through library shares.
  • Integrates with Azure (Though you cannot change settings of the Azure VMs but you can see them and at least stop and start them from your SCVMM console)

The biggest issue with using Hyper-V Manager over System Centre VMM is that it cannot scale to manage large environments, and with having three different consoles (Hyper-V Manager, Failover Cluster Manager and then Powershell) can be quite daunting to use to it’s full ability.

SCVMM Capacity Limits

Planning for your SCVMM deployment is a good practice and you want to make sure to size it right for your environment. The suggested limits of the software Microsoft details for SCVMM deployment are in the table below;

Entity Recommended maximum count
Physical hosts 1000
Virtual Machines 25000
Services 1000
User roles 1000
Clouds 20
Virtual networks 2000
logical networks 20
Library resources 1000
Library Objects (templates, profiles) 100

SCVMM Limits

Recommended hardware requirements are;

Hardware VMM server VMM database VMM library VMM console
Processor (minimum) 8 core Pentium 4, 2 GHz (x64) 8 core Pentium 4, 2.8 GHz 4 core Pentium 4, 2.8 GHz 2 core Pentium 4, 1 GHz CPU
Processor (recommended) 16-core, 2.66 GHz CPU 16 core 2.6 GHz CPU 4 core 2.8 GHz CPU 2 core 2 GHz CPU
RAM (minimum) 4 GB 8 GB 2 GB 4 GB
RAM (recommended) 16 GB 16 GB 4 GB 4 GB
Hard drive (minimum) 4 GB 50 GB Based on size/amount of stored files 10 GB
Hard drive (recommended) 10 GB 200 GB Based on size/amount of stored files 10 GB

SCVMM Operating System Requirements

The recommended Windows Operating System requirements for SCVMM 2019 are

  • Windows 2016 Server Core
  • Windows 2016 Full Desktop Experience
  • Windows 2019 Server Core
  • Windows 2019Full Desktop Experience.

SCVMM Database Requirements

The last piece to the SCVMM puzzle, is the database it requires. Obviously this database is another Microsoft product, SQL Server.

The supported Microsoft SQL database versions for SCVMM 2019 are;

  • SQL 2016
  • SQL 2017

How to install System Centre Virtual Machine Manager.

A step by step guide to install SCVMM.

Now that you can see the benefits to installing and using SCVMM to manage your Hyper-V hosts and it’s Virtual Machines. We will now get to the installation part to guide you through the process. I am going to assume here that you already have your Hyper-V hosts up and running and are ready to start your installation. To continue on with this installation guide I will run you through my current infrastructure setup. I have a newly deployed Hyper-V host and have prebuilt some generic virtual machines that are running on it. I am using a demo version of SCVMM which you can download for yourself from the Microsoft site (link here) and I have already predownloaded the files ready to start the installation.

So what are the prerequisites to install SCVVM? Obviously besides the requirements detailed up above, you will need a functioning Hyper-V environment to install the SCVMM on to, as well as administrator access to manage the Hyper-V hosts. As I have downloaded an image from Microsoft and if you are doing the same you will need Domain Administrator access to add the SCVMM Windows server to the domain.

Launch SCVMM Installer

Since the files are downloaded to my location machine, browse to the SCVMM install files through windows explorer. To start the install, just run the Executable file


Installing SCVMM

Now the installation starts, click on NEXT on the SCVMM install welcome screen.


scvmm installation

SCVMM License Agreement

Accept the license agreement (make sure to read them if you wish) and click on NEXT.


scvmm install license agreement

SCVMM Installation Location

The SCVMM install needs to extract the downloaded files. Choose an appropriate folder location and make sure you have enough free disks space (approximately 28GB is needed) then click on NEXT.


scvmm install location

SCVMM File Extraction

You are now ready to extract the SCVMM files needed for the installation. Confirm the settings and folder looks right then click on EXTRACT.


scvmm install file extraction

SCVMM Install Extracting

The installation files will now extract from your downloaded media. This will take a little bit of time to complete the extraction process.


scvmm install extracting

SCVMM File Extraction Completed

Now the extraction of SCVMM is complete. Click on FINISH.


scvmm install extraction complete

SCVMM Import

Next we need to connect to one of your Hyper-V hosts with the Hyper-V Manager console to start the import of the SCVMM Virtual Machine. Open up your Hyper-V console and connect to your host.


scvmm hyper-v

SCVMM New Virtual Machine

We now need to build a new VM for the SCVMM server. Click on ACTION, then NEW, then VIRTUAL MACHINE.


scvmm new vm

New Virtual Machine Wizard

The New Virtual Machine Wizard starts up, click on NEXT to proceed.


scvmm new virtual machine

New Virtual Machine Location

On the Specify Name and Location tab give your VM a name and you can choose where to store the VM files. When you have done that click on NEXT.


scvmm new vm location

VM Generation Selection

The Specify Generation window, make sure that Generation 1 is selected then click on NEXT


VM Memory Settings for SCVMM

Remember the minimum memory suggested by Microsoft above? Well if you dont, the minimum memory recommended was 4GB so as this is just a lab but I want some performance Im changing that to 8GB. When you have selected the memory size click on NEXT.


VM Virtual Switch Settings

Choose with virtual switch you want to deploy this VM to then click NEXT.


scvmm install networking

SCVMM Virtual Hard Disk Settings

This is one of the most important parts of the install process, make sure to change the virtual hard disk to Use an existing virtual hard disk and browse to the extracted location of the SCVMM download. When you have done that click NEXT.


SCVMM VM Deployment

Review everything on the Summary Tab and if you are happy to proceed with the SCVMM deployment click on Finish


scvmm install vm deployment

Power ON SCVMM VM

Now go back to your Hyper-V console and browse through your virtual machines until you find your newly deployed SCVMM virtual machine


Start SCVMM Installation

To start the installation you will need to power it on and open a console session so you can see what is happening.


scvmm start installation

Install Windows 2019 Server

We have to now run through the Windows 2019 server installation. Choose your country / region then click NEXT


win2019 install

Windows 2019 Server License Agreement

More license terms for you to read through, when you are happy to accept them, click on ACCEPT.


win2019 license agreement

Local Administrator Password Creation

Now provide a local administrator password and then click FINISH then the server will restart


local admin password

Login to SCVMM Server

Once Windows 2019 server has installed, log on to the server with the local administrator credentials you put in previously


win 2019 logon screen

SCVMM join Active Directory Domain

We now need to join the SCVMM virtual machine to the domain, so open up CONTROL PANEL then go to SYSTEM. Change the virtual machine name so you can easily identify it in the future and add the VM to your Active Directory Domain and dont forget to Reboot the server.


Install SQL Server for SCVMM

So after your SCVMM server has rebooted and is joined to your domain, we next need to start the installation of the SQL Database if you have not got one already. On the desktop of my Windows 2019 VM, I can see Microsoft has placed two icons for me to use to start the SCVMM installation. Luckily in this case they tell me to start the SQL Server install first, so we need to DOUBLE CLICK on that.


scvmm install sql

SQL Product Key

On the Product Key page for the SQL install, put in your license details and click on NEXT.


sql install product key

SQL License Agreement

Accept the license terms and click NEXT.


sql license agreement

SQL Global Rules

On the Global Rules tab, the SQL installation will check to see if the prerequisites are met for the installation. If all went well as in the picture click on NEXT.


sql install global rules

SQL Features

The Select Prepared Features tab check the settings are correct and click NEXT


sql install features

SQL Features Confirmation

The SQL installation for SCVMM now lets you review the Features are correct. Again, review the features you have chosen are right for your environment then click NEXT


sql install confirmation

SQL Instance Selection

On the SQL Instance Configuration you need to change this to the Default Instance (if you are installing a fresh copy of SQL, if you are using another server please point the instance to what you have setup. Click  NEXT to proceed.


sql install instance

SQL Service Account

Confirm the Server Configuration tab is correct (if you want to change the accounts you can, but Im leaving this all default.) Click NEXT.


sql install service account

SQL Database Engine Configuration

The Database Engine Configuration tab make sure to add your account so that you can manage the SQL instance. In my screenshot below I added the Domain Admins for my lab environment. Click Next to continue.


sql install database engine

Review SQL Installation Options

We are almost ready to start the SCVMM SQL server installation. Double-check all these settings look correct for your environment then click on COMPLETE to start the install.


sql install review

SQL Installing

The installation will now run through the process to install the SQL server components on your Virtual Machine. Maybe go grab a coffee to give this sometime to complete and if all went well you should see something similar to the following screen.


Starting SCVMM Installation

Finally we are on to the fun bit, we can start the configuration of SCVMM. I rebooted the server after the SQL install just to be sure that the services all started correctly and there was no errors detected.
Locate the Configure VMM icon and then DOUBLE CLICK IT.


start scvmm installation

SCVMM Setup

Once the Microsoft System Centre 2019 Virtual Machine Manager setup wizard starts up, put in your details, your company details and your SCVMM licence key then click NEXT.


SCVMM Diagnostics and Usage Information

The SCVMM Diagnostics and Usage Data page describes that Microsoft does collect some data from you. Please read through this then click NEXT.


scvmm install diagnostics

SCVMM Update

It is always a good idea to turn on Microsoft Update to patch the system should any vulnerabilities be found, but this is up to you and your environment. I chose to turn on Microsoft Update, then clicked on NEXT.


scvmm microsoft update

SCVMM Database Configuration

On the Database Configuration tab make sure the settings are correct for SQL server install you did previously and when confirmed click on NEXT.


scvmm install database configuration

SCVMM System Account

Now to the Configure service account and distributed key management tab, change it to Local System account, and then click NEXT


scvmm install account

SCVMM Library Configuration

For Library Configuration you can choose to create a new library or an existing share. Im leaving it default here as its a new install of SCVMM. Click NEXT to proceed


scvmm library configuration

SCVMM Installation Summary

Finally we are almost there, SCVMM is almost configured. Review the details on the Installation Summary page and when you are happy with the details click INSTALL. 

This will start the install so be sure everything is correct.


scvmm install summary

SCVMM Installation Summary

Hopefully you got a successful screen saying that SCVMM is now installed and configured, but if you didnt the best place to look is at the SCVMM install logs located %SYSTEMDRIVE%ProgramDataVMMLogs ServerSetup.log.


scvmm installation successful

Schedule a Hyper-V Checkpoint

Now that you have SCVMM installed in the next blog post I will cover off how to add your Hyper-V Hosts and perhaps while you are already here you can see how easy it is to schedule snapshot creations and deletions of your Hyper-V or VMware virtual machines with SnapShot Master.

 Download and try it for yourself with a free and fully functioning 30 day trial.