A useful TableQuery extension to retrieve all the results of a Microsoft Azure Table storage in a single asynchronous operation.

In the Azure Storage client library that you use to work with Azure Table storage, you may have notice that nothing will allow you to retrieve asynchronously all the results of a query in a single operation and that's probably what brought you here.

 

If you take a look at TableQuery<TElement> you will find two methods:

Same goes for CloudTable:

The legitimate question for you to ask is where are the methods ExecuteAsync and ExecuteQueryAsync?

Well those methods are not available in the library. If you want to get all the results of a query in one shot your only choice is to do so synchronously. If you want to get all the results in an async way, you will have to work with segmented results.

In many case asynchronously working with segmented results is the best option and that's probably why the Azure team built it this way. Working with segments is great, especially when you have a lot of records because you don't have to wait for all the results to start processing them, think parallel processing.

But in some case you want to get all the results before processing them, and if you are in an async context the following extension may help you.

 

Creation

In my scenario I know I'll always work in a small set of records, I'm in an async context and I need to get all the results before processing them.

The ExecuteAsync TableQuery extension looks like the following:

using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.WindowsAzure.Storage.Table;
using Microsoft.WindowsAzure.Storage.Table.Queryable;

namespace AzureTableStorage.TableQueryAsync
{
    public static class TableQueryExtensions
    {
        /// <summary>
        /// Initiates an asynchronous operation to execute a query and return all the results.
        /// </summary>
        /// <param name="tableQuery">A Microsoft.WindowsAzure.Storage.Table.TableQuery representing the query to execute.</param>
        /// <param name="ct">A System.Threading.CancellationToken to observe while waiting for a task to complete.</param>
        public async static Task<IEnumerable<TElement>> ExecuteAsync<TElement>(this TableQuery<TElement> tableQuery, CancellationToken ct)
        {
            var nextQuery = tableQuery;
            var continuationToken = default(TableContinuationToken);
            var results = new List<TElement>();

            do
            {
                //Execute the next query segment async.
                var queryResult = await nextQuery.ExecuteSegmentedAsync(continuationToken, ct);

                //Set exact results list capacity with result count.
                results.Capacity += queryResult.Results.Count;

                //Add segment results to results list.
                results.AddRange(queryResult.Results);

                continuationToken = queryResult.ContinuationToken;

                //Continuation token is not null, more records to load.
                if (continuationToken != null && tableQuery.TakeCount.HasValue)
                {
                    //Query has a take count, calculate the remaining number of items to load.
                    var itemsToLoad = tableQuery.TakeCount.Value - results.Count;

                    //If more items to load, update query take count, or else set next query to null.
                    nextQuery = itemsToLoad > 0
                        ? tableQuery.Take<TElement>(itemsToLoad).AsTableQuery()
                        : null;
                }

            } while (continuationToken != null && nextQuery != null && !ct.IsCancellationRequested);

            return results;
        }
    }
}

We can pay attention to two things here:

As you can notice the extension is simple, however the trickiest part comes when a TableQuery has the TakeCount property set. In that case we need to reevaluate the number of items to take after each segment results.

 

Example of use

Here is a quick example showing how to use the extension in a console application:

using System;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Azure;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Table;
using Microsoft.WindowsAzure.Storage.Table.Queryable;

namespace AzureTableStorage.TableQueryAsync
{
    class Program
    {
        static void Main(string[] args)
        {
            ExecuteSimpleTableQuery(CancellationToken.None).Wait();

            Console.ReadLine();
        }

        private static async Task ExecuteSimpleTableQuery(CancellationToken ct)
        {
            var table = await GetTable(ct);

            var query = table.CreateQuery<MyTableEntity>()
                .Where(r => !r.MyProperty.Equals(string.Empty))
                .Take(1200)
                .AsTableQuery();

            var results = await query.ExecuteAsync(ct);

            Console.WriteLine($"Results {(results.Any() ? string.Empty : "not ")}found.");
        }

        private static async Task<CloudTable> GetTable(CancellationToken ct)
        {
            //Retrieve the storage account from the connection string.
            var storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));

            //Create the table client.
            var tableClient = storageAccount.CreateCloudTableClient();

            //Retrieve a reference to the table.
            var table = tableClient.GetTableReference("mystoragetable");

            //Create the table if it doesn't exist.
            await table.CreateIfNotExistsAsync(ct);

            return table;
        }
    }
}

This TableQuery will let us get 1200 results asynchronously with entities having the property MyProperty not empty.

 

To go further

When a segmented query is executed a maximum of 1,000 entries is returned. If the value specified for the TableQuery TakeCount property is greater than 1,000, the library will ignore it and limit it to 1,000.

 

 

Summary

Thanks to this extension we have seen how to retrieve all the results of a Microsoft Azure Table storage in a single asynchronous operation. Of course I advise you to use it wisely, in scenarios where you specifically need to get all the results before processing them, for other scenarios you should stick to the segmented query.

 

You can download the example solution here:

Download full sources

Or

Browse the GitHub repository

(Note that the project uses WindowsAzure.Storage version 7.2.1)

 

Please feel free to comment or contact me if you have any question about this article.

Add a comment

(Will not be published)

Back to articles