by Mark | Jan 25, 2023 | Azure, Azure Blobs, Blob Storage
Azure storage accounts offer powerful, cost-effective options for managing your data and applications. With various services such as blobs, queues, files, and tables, you can use Azure Storage to store and access virtually limitless amounts of data effectively. This guide will walk you through all the basics of setting up and using an Azure Storage Account.
What is Azure Storage and How Does It Work?
Azure storage consists of durable, conveniently located and cost-effective cloud storage services. It offers a range of storage options to accommodate different budget and performance needs. Data stored in Azure Storage can be accessed via various protocols, such as HTTP/HTTPS for web applications and SMB for applications running on Windows Virtual Machines. Additionally, Azure Storage is secure, compliant with global standards, redundant and scalable.
Azure Storage is a cloud-based service provided by Microsoft Azure for storing and managing unstructured data, such as binary files, text files, and media files. Azure Storage includes several different storage options, including Azure Blob storage, Azure File storage, Azure Queue storage, and Azure Table storage.
Azure Blob storage is designed for unstructured data and is optimized for storing large amounts of unstructured data, such as text or binary data. Blobs can be in several formats like block blobs, page blobs and append blobs.
Azure File storage is a service that allows you to create file shares in the cloud, accessible from any SMB 3.0 compliant client.
Azure Queue storage is a service for storing and retrieving messages in a queue, used to exchange messages between components of a distributed application.
Azure Table storage is a service for storing and querying structured NoSQL data in the form of a key-value store.
All of these services allows you to store and retrieve data in the cloud using standard REST and SDK APIs, and they can be accessed from anywhere in the world via HTTP or HTTPS.
Azure Storage also provides built-in redundancy and automatically replicates data to ensure that it is always available, even in the event of an outage. It also provides automatic load balancing and offers built-in data protection, data archiving, and data retention options. With the use of Shared Access Signatures (SAS) you can control who and when can access the stored data.
In summary, Azure Storage is a set of services that enables the ability to store and manage unstructured data in the cloud, providing various storage options, accessibility, and built-in redundancy, security, and management features.
Managing Your Storage Accounts in Azure
Azure storage consists of durable, conveniently located and cost-effective cloud storage services. It offers a range of storage options to accommodate different budget and performance needs. Data stored in Azure Storage can be accessed via various protocols, such as HTTP/HTTPS for web applications and SMB for applications running on Windows Virtual Machines. Additionally, Azure Storage is secure, compliant with global standards, redundant and scalable.
Managing your storage accounts in Azure involves several different tasks, such as creating and configuring storage accounts, setting up access control, monitoring and troubleshooting storage accounts, and managing data stored in the accounts.
To create a new storage account, you can use the Azure portal, Azure CLI, or Azure PowerShell. Once the storage account is created, you can configure it by setting up access control, creating containers or file shares, and configuring data replication, encryption, and backup options.
Access control in Azure Storage is managed using shared access signatures (SAS) and Azure Active Directory (AAD) authentication. SAS allow you to control access to specific resources within a storage account, and can be used to grant time-limited access to specific users or applications. AAD authentication allows you to secure your storage accounts by requiring users to sign in with their Azure AD credentials.
Monitoring and troubleshooting storage accounts can be done using Azure Monitor, Azure Log Analytics and Azure Storage Analytics. Azure Monitor provides real-time telemetry and alerts, while Azure Log Analytics enables you to analyze and troubleshoot issues by querying logs and metrics. Azure Storage Analytics provide usage metrics, diagnostic logs and operation logs for your storage account.
Finally, managing data stored in your storage accounts can be done using Azure Storage Explorer, Azure CLI, and Azure PowerShell. Azure Storage Explorer provides a graphical user interface for managing data stored in your storage accounts, while Azure CLI and Azure PowerShell provide command-line interfaces for managing data.
In summary, managing storage accounts in Azure involves creating and configuring storage accounts, setting up access control, monitoring and troubleshooting storage accounts, and managing data stored in the accounts, with the help of a variety of Azure tools like Azure Monitor, Azure Log Analytics, Azure Storage Analytics, Azure Storage Explorer, Azure CLI and Azure PowerShell.
Overview of the Different Types of Storage Services
Azure Storage is an efficient and cost effective way to store data in the cloud. You can choose from a variety of storage services, each designed for a different purpose. These include blob storage for objects such as images, videos, and audio; file storage for shared access files and folders; table storage NoSQL key-value pairs; queue storage queues used to facilitate message communication between applications; and disk storage virtual disks used to create VMs.
Azure Storage provides several different types of storage services, each optimized for different types of data and use cases. These services include:
- Azure Blob Storage: This is object storage for unstructured data, such as text or binary files, images, and videos. Blob storage allows you to store and access large amounts of unstructured data, and is designed for scalability and high availability. It support 3 types of blobs: Block Blobs, Page Blobs and Append Blobs
- Azure File Storage: This service allows you to create file shares in the cloud that can be accessed using the SMB protocol, making it easy to work with file-based data using standard file system APIs. This service can be useful for scenarios where you need to share files among multiple VMs.
- Azure Queue Storage: This service provides a message queue that can be used to exchange messages between components of a distributed application. This can be useful for scenarios where you need to reliably send messages between different parts of your application.
- Azure Table Storage: This service provides a NoSQL data store that can be used to store and retrieve structured data in the form of key-value pairs. This can be useful for scenarios where you need to store and retrieve large amounts of structured data that doesn’t need to be queried with full-text search or join operations.
- Azure Disks and Disk Snapshots: These services allow you to create and manage virtual hard disks (VHDs) in Azure, which can be used to store persistent data for Azure VMs. You can also take snapshots of a disk, which allows you to take a point-in-time copy of the disk and use it to restore the disk or create new disks.
All these services are built on top of Azure Storage infrastructure and share common features like automatic replication, durability, high availability and can be managed via Azure portal, Azure Storage Explorer, Azure CLI and Azure PowerShell.
As always, there are limitations to technology. Azure Storage is no different, read this post to understand the Azure Storage limitations.
Using Blobs to Store Binary Data
Block blobs are used to store binary data, such as images, videos, documents and application installers. They allow you to upload large amounts of data and can support up to 195GB in size. Blob storage is a great way to store static objects like images or videos that your applications may need to access. Each file or block is stored as an atomic unit, meaning once uploaded the data cannot be further changed or modified.
Azure Blob storage is a service that can be used to store binary data, such as text or binary files, images, and videos. Blob storage supports three types of blobs: block blobs, page blobs, and append blobs.
Block blobs are the most common type of blob and are optimized for streaming. They can be used to store files such as images, videos, and documents. Each block blob can be up to 200 GB in size.
Page blobs are similar to block blobs, but they are optimized for random read and write operations. They can be used to store files such as virtual hard disks (VHDs) and SQL database files. Each page blob can be up to 8 TB in size.
Append blobs are similar to block blobs, but they are optimized for append operations. They are used to store log files and other data that is appended to over time. Each append blob can be up to 195 GB in size.
In order to store binary data in Azure Blob Storage, you can use the Azure Storage SDKs, Azure Storage REST API, or Azure Storage Explorer. You can upload data to a blob using the Put Blob operation, and you can download data from a blob using the Get Blob operation.
Once the data is in the blob, you can set permissions on the blob, set metadata, and even generate shared access signatures (SAS) to allow others to access the data with a specific set of permissions.
Additionally, you can use features like lifecycle management, geo-redundancy, encryption, and backups to ensure the data is protected and can be easily accessed and managed.
In summary, Azure Blob storage is a cost-effective, scalable, and highly available service for storing unstructured data, it provides three different types of blob storage tailored for specific use cases and scenarios, and it can be easily integrated with other Azure services for data management, security, backup and disaster recovery.
Understanding Tables, Queues & Files for Storage Operations
Tables are used to store structured non-relational data in a NoSQL format, meaning you can store large amounts of data without any predefined structure. This type of storage is an excellent option for applications that feed off large volumes of data and require rapid access, such as gaming and analytics applications. Queues are the perfect choice if you need to queue up messages or tasks and have them read by multiple receivers. Finally, files can be used to store disk level files or images that your application might need to read or write. All files stored in the file service are accessible via either REST API or SMB protocol.
Azure Storage includes several different services for storing and managing data, including Azure Table storage, Azure Queue storage, and Azure File storage. Each of these services is optimized for different types of data and use cases.
Azure Table storage is a NoSQL data store that can be used to store structured data in the form of key-value pairs. It is designed for storing large amounts of structured data that doesn’t need to be queried with full-text search or join operations. It is well suited for storing semi-structured data that doesn’t fit a traditional relational schema, or for storing metadata or log data.
Azure Queue storage is a service that provides a message queue that can be used to exchange messages between components of a distributed application. Queue storage can be used for reliable messaging between different parts of your application, for example, between a web frontend and backend worker roles, it allows you to decouple the components of your application.
Azure File storage is a service that allows you to create file shares in the cloud that can be accessed using the SMB protocol, making it easy to work with file-based data using standard file system APIs. Azure File Storage is a great fit for scenarios where you need to share files among multiple VMs, for example, when you have a distributed application.
In summary, Azure Table storage is designed for storing structured data, Azure Queue storage is designed for messaging, and Azure File storage is designed for file-based storage. Each service is optimized for different use cases and can be used together to create a complete data storage and management solution in Azure.
Now hopefully you understand a little bit more about Azure Storage and its various services. If you are using Azure Storage and need to gather insights in to your Storage Consumption, have a look and download a free trial of Cloud Storage Manager.
by Mark | Jan 18, 2023 | Azure Blobs, Blob Storage, Cloud Computing, How To, Storage Accounts
Azure Storage Authentication Options
Azure Storage Authentication is the process of verifying the identity of a client that is requesting access to an Azure storage account. Azure Storage supports several authentication options that can be used to secure access to storage accounts, such as:
- Shared Key Authentication: This method of authentication uses a shared key that is known to both the client and the storage account to sign request headers.
- Shared Access Signature (SAS) Authentication: This method of authentication uses a shared access signature (SAS) token to provide restricted access to a storage account. A SAS token can be generated for a specific resource or set of resources within a storage account and can be used to grant read, write, or delete access to that resource.
- Azure Active Directory (AAD) Authentication: This method of authentication allows you to secure access to a storage account using Azure AD. By connecting a storage account to Azure AD, you can use Azure AD authentication to grant access to specific users or applications that are already authenticated with Azure AD.
- OAuth Authentication: This method of authentication allows you to authenticate with Azure Storage using an OAuth 2.0 Bearer Token. The token is passed in the Authorization header and is verified by Azure Storage.
- Token-based Authentication: This method of authentication is an advanced authentication method and includes authentication with SAS, OAuth 2.0 bearer tokens and JSON Web Tokens (JWT)
Choosing the best authentication option depends on your requirements such as security, ease of use and ease of integration with existing systems or platforms. For example, for testing or development purposes, Shared Key authentication can be sufficient and easier to implement, but for production environments that requires high level of security or integration with enterprise environments you may prefer to use Azure AD or OAuth.
How to connect your Azure Storage Accounts to your On-Premise Active Directory
Connecting a storage account to an on-premises Active Directory (AD) allows you to secure access to the storage account using on-premises AD authentication. This can be useful in scenarios where you want to provide access to the storage account to a specific group of users or applications that are already authenticated with your on-premises AD.
Here’s an overview of the process for connecting a storage account to an on-premises AD:
- Create a Domain Name System (DNS) alias: To connect to the on-premises AD, you will need to create a DNS alias that points to the on-premises AD. This can be done by creating a CNAME record in your DNS server.
- Configure the storage account to use AD authentication: In the Azure portal, go to the storage account settings and enable AD authentication for the storage account. You will need to provide the DNS alias that you created earlier and specify the domain name of your on-premises AD.
- Create a group in the on-premises AD: To grant access to the storage account, you will need to create a group in your on-premises AD. This group will be used to manage access to the storage account.
- Assign the storage Blob Data Contributor role to the group: To grant access to the storage account, you will need to assign the storage Blob Data Contributor role to the group. This role allows the members of the group to manage blobs in the storage account
- Add users or computer to the group: To grant access to storage account you should add users or computer to the group you created in step 3
It’s worth to mention that this process requires that you have your own domain controller and DNS server, and that your azure storage account and your on-premise network should be connected through a VPN or ExpressRoute.
Also, it would require an ADFS or other third party solution to facilitate the integration and trust relationship between on-premises AD and Azure AD.
by Mark | Feb 3, 2022 | Azure, Azure Blobs, Comparison, Storage Accounts
Are you confused about which Azure Storage Service to use, for mapping a drive, or just dumping large amounts of data in to? This post goes indepth in to the differences between Azure Blob Storage and Azure File Services.
Azure Blob Storage vs Azure File Storage:
Choosing the Right Storage Solution for Your Business
In today’s digital age, data storage is an essential part of any business strategy. As more and more businesses move their operations to the cloud, choosing the right storage solution can make a significant impact on your organization’s efficiency and productivity. Azure, Microsoft’s cloud computing platform, offers two primary storage solutions: Azure Blob Storage and Azure File Storage.
In this article, we’ll take a closer look at both Azure Blob Storage and Azure File Storage and help you decide which one is the best fit for your business.
What is Azure Blob Storage?
Azure Blob Storage is a massively scalable object storage service that allows you to store and manage large amounts of unstructured data. It is ideal for storing files such as videos, images, audio, and text documents. Blob storage is designed to handle large amounts of data and can scale to meet your business needs, making it an excellent choice for businesses that require large-scale data storage.
Benefits of Azure Blob Storage
- Massive Scalability: Azure Blob Storage can handle massive amounts of data and can scale to meet your business needs.
- Cost-Effective: Blob storage offers a cost-effective solution for storing large amounts of unstructured data.
- Durability: Data stored in Blob storage is highly durable and can be replicated across different regions to ensure high availability.
- Security: Blob storage provides granular access control, allowing you to control who can access your data.
- Integration: Blob storage can easily integrate with other Azure services, making it easy to build scalable and reliable applications.
What is Azure File Storage?
Azure File Storage is a fully managed file sharing service that allows you to share files with both Windows and Linux clients. It is ideal for storing structured data such as application data and virtual machine files. With Azure File Storage, you can access your files from anywhere and at any time, making it an excellent choice for businesses with remote workers or multiple office locations.
Benefits of Azure File Storage
- Fully Managed: Azure File Storage is a fully managed service, which means that Microsoft takes care of the infrastructure and maintenance, allowing you to focus on your business.
- Easy to Use: Azure File Storage is easy to set up and use, with a simple interface that allows you to manage your files from anywhere.
- Secure: Azure File Storage provides granular access control, ensuring that your data is secure.
- Integration: Azure File Storage can easily integrate with other Azure services, making it easy to build scalable and reliable applications.
- Multi-Platform: Azure File Storage supports both Windows and Linux clients, making it a versatile option for businesses with a variety of operating systems.
Choosing the Right Storage Solution
Now let’s explore the key differences between Azure Blob Storage and Azure File Storage.
Structure
Azure Blob Storage stores data as blobs within containers, allowing for a flat storage structure. In contrast, Azure File Storage uses a hierarchical structure, with directories and subdirectories, similar to a traditional file system.
Access
Blob Storage supports REST APIs, Azure Storage SDKs, and Azure Storage Explorer for data access. File Storage allows access via the SMB protocol, enabling you to mount file shares directly to your virtual machines or on-premises systems.
Scalability
Blob Storage can scale to store petabytes of data, with individual blobs up to 4.75 TB in size. File Storage, while still scalable, has a maximum share size of 100 TiB.
Performance
Blob Storage provides higher throughput and lower latency than File Storage. However, File Storage offers better performance for small file read and write operations, which can be crucial for specific applications.
Security
Both services provide data encryption at rest and in transit. However, Blob Storage also supports object-level access control through shared access signatures, while File Storage relies on share-level access control.
When choosing between Azure Blob Storage and Azure File Storage, it’s important to consider your business’s needs. If you need to store large amounts of unstructured data such as videos, images, and audio, Azure Blob Storage is the best choice. If you need to store structured data such as application data and virtual machine files and share files with Windows and Linux clients, Azure File Storage is the better option.
Azure Blob Storage FAQs:
What types of data can I store in Azure Blob Storage?
You can store various types of unstructured data, including text, images, audio, and video files.
How scalable is Azure Blob Storage?
Azure Blob Storage is massively scalable and can handle large amounts of data. It can scale to meet your business needs.
Is my data safe in Azure Blob Storage?
Yes, your data is safe in Azure Blob Storage. Data stored in Blob storage is highly durable and can be replicated across different regions to ensure high availability. Blob storage also provides granular access control, allowing you to control who can access your data.
Can I integrate Azure Blob Storage with other Azure services?
Yes, you can easily integrate Azure Blob Storage with other Azure services, such as Azure Data Factory, Azure Functions, and Azure Stream Analytics.
How much does Azure Blob Storage cost?
Azure Blob Storage pricing is based on the amount of data stored, data access, and data transfer. You can find more information about Azure Blob Storage pricing on the Azure website.
Azure File Storage FAQs:
What types of data can I store in Azure File Storage?
You can store structured data such as application data and virtual machine files in Azure File Storage. It is designed for file sharing with Windows and Linux clients.
Is Azure File Storage easy to use?
Yes, Azure File Storage is easy to use, with a simple interface that allows you to manage your files from anywhere.
How secure is Azure File Storage?
Azure File Storage provides granular access control, ensuring that your data is secure. You can control who can access your data and what they can do with it.
Can I integrate Azure File Storage with other Azure services?
Yes, Azure File Storage can easily integrate with other Azure services, such as Azure Virtual Machines, Azure Backup, and Azure Site Recovery.
How much does Azure File Storage cost?
Azure File Storage pricing is based on the amount of data stored and data access. You can find more information about Azure File Storage pricing on the Azure website.
Azure Blob and File Storage Use Cases
Use Case |
Azure Blob Storage |
Azure File Storage |
Unstructured Data Storage |
Best suited for storing unstructured data, such as text, images, audio, and video files. |
Designed for structured data storage, such as application data and virtual machine files. |
Data Access |
Provides block-level access to data, making it ideal for streaming and storing large media files. |
Provides SMB-based file access, making it ideal for sharing files across Windows and Linux clients. |
Scalability |
Highly scalable and can handle large amounts of data. |
Can also scale to meet business needs, but not as well-suited for extremely large data sets. |
Security |
Provides granular access control, ensuring that data is secure. |
Provides granular access control, ensuring that data is secure. |
Integration |
Can integrate with other Azure services, such as Azure Data Factory, Azure Functions, and Azure Stream Analytics. |
Can easily integrate with other Azure services, such as Azure Virtual Machines and Azure Backup. |
How can I connect to Azure Blob Storage?
With Azure Blob Storage access is provided over HTTPS, so you cant traditionally mount Blob Storage as a mapped network drive to a Virtual Machine or Users desktop.
You can however browse all the Azure Blobs contained within this Storage Account through your web browser, using Microsoft’s Azure Storage Explorer, the Azure Storage Rest API, Powershell or even the Azure CLI. Alternatively, you can use our software, Cloud Storage Manager to do the same thing and more.
How can I connect to Azure File Storage?
As stated before, Azure File Services is more like your standard file server, in that you can create multiple network shares with folders and files within. These shares can of course be mapped directly to your machines as a network drive using the SMB protocol. To map a network drive to an Azure File Share, Microsoft lets you create a login script direct from the Azure Portal using powershell to connect the share.
Here is an example powershell script to connect an Azure File share to a users computer as a Y drive mapping.
$connectTestResult = Test-NetConnection -ComputerName storageaccountname.file.core.windows.net -Port 445
if ($connectTestResult.TcpTestSucceeded) {
# Save the password so the drive will persist on reboot
cmd.exe /C “cmdkey /add:`”storageaccountname.file.core.windows.net`” /user:`”localhoststorageaccoun`” /pass:`”longpasswordabcdefghijklmnopqrstuvwxyz`””
# Mount the drive
New-PSDrive -Name Y -PSProvider FileSystem -Root “\storageaccountname.file.core.windows.netfileservicestest” -Persist
} else {
Write-Error -Message “Unable to reach the Azure storage account via port 445. Check to make sure your organization or ISP is not blocking port 445, or use Azure P2S VPN, Azure S2S VPN, or Express Route to tunnel SMB traffic over a different port.”
}
With Azure Files you can also set NTFS permissions on the shares, folders and files. Additionally you can integrate this service with your Azure AD so that permissions can be set using normal AD groups and users.
One thing that differentiates Azure File Services from an on-premise File Server is that you can access the folders and files from anywhere with an internet connection.
Allow access to Azure Storage Account using a Shared Access Signature
The Shared Access Signature can be created to allow access over HTTP or HTTPS for a limited time to any of the Azure Storage Services. This includes Blob, File, Queue and Table Services.
You can be granular with the access you want to provide. Perhaps you have a third party that you only want to give access to your data and certain services for a fixed amount of time.
Options you can choose are;
- Allowed Services;
- Allowed Resource Types
- Allowed Permissions
- Read
- Write
- Delete
- List
- Add
- Create
- Update
- Process
- Immutable Storage
Choose this options carefully, as you can see you can be as granular as you want to allow access to your Azure Storage Accounts.
Whats a Use Case Example for Azure Blob Storage?
Azure Blob Storage has several use cases. Its great for the following types of data storage;
- Serving documents, images or videos direct to a browsers.
- Storing of files for quick and easy access
- Streaming video and audio
- Great for dumping of logs files for analysis
- A good target for backup files, archiving and disaster recovery.
Whats a Use Case Example for Azure File Storage?
Azure File Services is great for the following use cases;
- Replacing or supplementing your on-premise File Servers
- Allowing network share access to anywhere in the world with an internet connection
- Lift and Shift your applications to the Cloud that require a network share
- Improving the resilience and uptime of your network shares
- Removing the need to patch and maitain Windows File Servers
So what are the main differences between Azure Blob Storage and Azure File Services?
Item
|
Azure Blob Storage
|
Azure File Services
|
Tiering
|
Hot Cool Archive |
Premium Transaction Optimized Hot Cool Note – No Archive Tier available for Azure File Services |
Authentication
|
SAS Token Storage Access Keys Azure AD Integration Anonymous public read access |
SAS Token Storage Access Keys Azure AD On-Premise Active Directory – Synced with Azure AD |
Redundancy
|
LRS – Locally Redundant Storage ZRS – Zone Redundant Storage GRS – Geo Redundant Storage GZRS – Geo Zone Redundant Storage |
LRS – Locally Redundant Storage ZRS – Zone Redundant Storage GRS – Geo Redundant Storage GZRS – Geo Zone Redundant Storage |
Maximum Size
|
500TB |
100TiB per File Share |
Maximum File Size
|
Azure BLOBs are 2 types; Page and Block. Max Page is 8TiB, max block is 200TiB |
1 TB per File |
Authentication
|
SAS Token Storage Access Keys Azure AD Integration Anonymous public read access |
SAS Token Storage Access Keys Azure AD Integration
On-Premise Active Directory – Synced with Azure AD
|
Choosing the Right Storage Solution
When choosing between Azure Blob Storage and Azure File Storage, it’s important to consider your business’s needs. If you need to store large amounts of unstructured data such as videos, images, and audio, Azure Blob Storage is the best choice. If you need to store structured data such as application data and virtual machine files and share files with Windows and Linux clients, Azure File Storage is the better option.
In conclusion, Cloud Storage Manager is an effective tool that helps organizations to manage their Azure storage usage and costs. With its powerful features such as Map View, Tree View, Azure Storage Reporting, Azure Blob Search, and Azure Files Complete Overview, the software enables users to gain insights into their Azure Blob and Azure File storage consumption, identify the areas where costs can be reduced, and take appropriate actions. By visualizing Azure storage locations, browsing through Azure storage trees, generating reports, searching and scanning Azure storage accounts, users can quickly see where their storage is going and take back control of their Azure Blob storage costs. Cloud Storage Manager is a valuable tool that saves money, enhances productivity, and helps organizations to manage their storage accounts efficiently.