You may ask yourself how and where should we store the data in this kind of operation? The answer is Azure Storage. Azure storage is service for storing various data types and therefore it contains various types for you to choose:
Simple, distributed, cross-platform file system
- Lift and shift migration
- Simple and inexpensive
- Move data to cloud with no coding
Premium storage for I/O-intensive applications
- Low latency, high throughput
- Automatic triple replication
- Enterprise-grade durability
Massively-scalable object storage for unstructured data
- Cost-effective for massive volume
- Tiered storage options
- Single infrastructure with global reach
Durable queues for large-volume cloud services
- Simple, cost-effective messaging
- Decoupled component flexibility
- Resilient scaling and buffering
Flexible NoSQL database
- Key-value table storage
- Structured or unstructured data
- Low latency at Internet scale
Low cost storage for infrequently used data
- Data automatically encrypted at rest
- Seamless integration with hot and cool storage tiers
- Supported by leading Data Management partners
The best thing about Azure Storage is that you can use Command-Line (PowerShell or Azure CLI), various languages (Java, node.js, PHP, Ruby, Python, C++, iOS, Android, .NET) or REST services to code against it. Please check Azure Storage Documentation
for additional information.
How to start working with azure storage?
Obviously, you will need to have an Azure Subscription but let’s assume that this prerequisite is fulfilled. Since we will build the background task by using .NET we need some instance of the Visual Studio. If you're using Visual Studio 2015, install the Azure SDK for .NET (Visual Studio 2015)
. If you're using Visual Studio 2017, install the Azure development workload
Now that we have everything let’s create a new Storage account on the Azure Portal:
Once you created a Storage Account on the dashboard you will see all your services: Blobs, Files, Tables and Queues. On this dashboard you can configure security, application access keys and various access policies or you can monitor the activity of your storage account. You can create many storage accounts under one azure account.
One thing that is interesting from a developer’s perspective are Access Keys. Access Keys are very important part of the storage account and are used in Azure Storage SDK for accessing your storage. Access Keys are also used in WebJobs and Functions as AzureWebJobsDashboard and AzureWebJobsStorage connection strings (one key can be used for both).
Now that we have our Storage and Access Keys let’s create a WebJob that will be triggered by the message in Azure Queue. First let’s create one queue container. The best thing about queue container is that you can put any message you want into the container. In our example we will use JSON as a message and we will add a list of objects into the container.
Let’s think for a second on a WebJob in our previous example “Run continuous or triggered background task with WebJobs in Azure (part 1)”. What if we want to put a message from this WebJob into Queue and trigger by this event the second WebJob. How could we achieve this? First you will need to include Microsoft.WindowsAzure.Storage reference into your existing WebJob. We will create a method that will accept a collection of FxRate object (this can be your type of object) and it will push the message to queue.
How to create a queue triggered Web Job?
As you can see, message is pushed to a queue by simply serializing an object into JSON and creating CloudQueueMessage object and adding this message to queue.
//Push message to queue
var queueMessage = new CloudQueueMessage(JsonConvert.SerializeObject(entity));
Now that we have changed our first WebJob so that he will create a message and put it into queue let’s create a new WebJob:
Now that we have our template we will use generated Main method body and we will add additional method in Functions class
You maybe ask yourself what does the: “Please set the following connection strings in app.config for this WebJob to run: AzureWebJobsDashboard and AzureWebJobsStorage” means? For running WebJobs on Azure, you need to have Storage Account service. This is used for two reasons:
Azure Storage is a data component extension to your Azure Storage for triggered operations with queue, blobs or tables (NoSQL)
WebJobs is using Storage Account for storing logs.
Go to the Storage Account you’ve created earlier and go to the Manage Access Keys section and take the connection string and use them in your WebJob. You can use the same Azure Storage account for both connection strings.
Now that we add the keys we can create our QueueFunction method inside of a Functions class:
As you can see the important part of the function is in the first parameter:
[QueueTrigger("fxrates")] FXRatesInformation fxInfo
You have specified that you will be using Queue as a trigger and particular fxrates (the name of your queue container) and set FXRatesInformation as a message. This is how this object looks like:
As you see this is a plan .NET class and if you remember from the beginning we have serialized the same object in Continuous running WebJob into JSON. The beauty is that we didn’t have to deserialize the JSON into object again, the WebJob did this for us. And this are steps how you can create a queue triggered WebJob. You can use the same principle for blob storage. This is all on introduction of Azure WebJobs. Please check some interesting links below for additional information on Azure Storage and WebJobs:
Azure Storage Documentation
Create a file share in Azure Files
How to use Azure queue storage with the WebJobs SDK
How to Send Email Using SendGrid with Azure
Run Background tasks with WebJobs in Azure App Service
Azure WebJob with .Net Core Console Application
Introducing Windows Azure WebJobs