Adding Multiple Messages To An Azure Service Bus Queue

13 May 2015

I don't know if this is a very common scenario, but I wrote some code earlier in the week to add a bunch of messages to a Service Queue:

QueueClient queueClient = GetQueue();

foreach(var item in myItems)
{
    BrokeredMessage message = new BrokeredMessage(Guid.NewGuid().ToString());

    queueClient.Send(item);
}

This works fine for 5 messages, but it's actually kind of slow for 2000 messages - each Send operation is bottlenecked by network, and if you're doing this on an Azure Website you will probably hit the Load Balancer timeout.

As it turns out, there's a way to cut down on the number of operations - all we have to do is use the SendBatch() method on the QueueClient. So my code ended up looking a little like:

QueueClient queueClient = GetQueue();

List<BrokeredMessage> messageList = new List<BrokeredMessage>();

long currentBatchSize = 0;

foreach(var item in myItems)
{
    BrokeredMessage message = new BrokeredMessage(Guid.NewGuid().ToString());
    currentBatchSize += message.Size;
    messageList.Add(message);

    if(currentBatchSize > 40000)
    {
        queueClient.SendBatch(messageList);
        messageList.Clear();
        currentBatchSize = 0;
    }
}

//send the final batch
queueClient.SendBatch(messageList);

There's a few things to note in the sample above:

SendBatch() has the same maximum size limit that Send() has; at the time of writing this is 256kb. If you attempt to send too many messages at one time it will throw a relevant Exception telling you so.

The BrokeredMessage.Size property is not accurate until the message is actually sent via the QueueClient - I imagine this is because the headers and other message metadata bits haven't been attached yet. (My messages went from 60ish to 300ish bytes pre and post sending which is how I came to the 40000 byte per batch limit, but may be different for your specific scenario)

By using .SendBatch() intelligently, I turned a 2 minute operation into a 2 second one. There's probably another limit to how this will scale, but that's a problem for another time!