C# ways of handling when being throttled by an API
I’ve been building a service application that is responsible to grab data from a REST API. The API has mechanisms in place to reduce abuse and make sure that everyone can consume its service in a fair way. That being said, this means that sometimes you may need to do a lot of requests to extract the data you need. If you get throttled, that is being told that you are sending too many requests and get served with a “temporary” ban, you will need to way and retry. You know it is a “temporary” ban so why would you send back an error (exception) to your client when it’s something you can possibly handle yourself. How can you deal with this in code?
If you are served with a HTTP status code 429, you should receive a Retry-After header, which contains the number of seconds after which you can send requests again to the API. You may get that header as well when you get a status code 503 (service unavailable), if the API you consume implemented it. Again, you should delay your requests to the API with the value, in seconds, that was returned in the header.
In case either the header is not available, you can use an algorithm technique called the exponential backoff. Exponential backoff is an algorithm that uses feedback to multiplicatively decrease the rate of some process, in order to gradually find an acceptable rate1.
In this post, I will show you 2 ways in C# in which you can achieve that.
Way #1 – Pure .NET way
The Microsoft Graph API implements, in their .NET SDK, the strategy mentioned above using a (HTTP) message delegated handler. A message handler is a class that receives an HTTP request and returns an HTTP response.
As shown in the documentation, the diagram below shows an example of two custom handlers inserted into the pipeline:
You can find the source code of this handler, coded by the Microsoft Graph SDK Team called the RetryHandler, here. The strategy is implemented in the Delay method.
Here’s an excerpt of the code (the Delay method):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
/// <summary> /// Delay task operation based on Retry-After header in the response or exponential backoff /// </summary> /// <param name="response">The <see cref="HttpResponseMessage"/>returned.</param> /// <param name="retry_count">The retry counts</param> /// <param name="cancellationToken">The cancellationToken for the Http request</param> /// <returns>The <see cref="Task"/> for delay operation.</returns> public Task Delay(HttpResponseMessage response, int retry_count, CancellationToken cancellationToken) { TimeSpan delay = TimeSpan.FromMilliseconds(0); HttpHeaders headers = response.Headers; if (headers.TryGetValues(RETRY_AFTER, out IEnumerable<string> values)) { string retry_after = values.First(); if (Int32.TryParse(retry_after, out int delay_seconds)) { delay = TimeSpan.FromSeconds(delay_seconds); } } else { m_pow = Math.Pow(2, retry_count); double delay_time = m_pow * DELAY_MILLISECONDS; delay = TimeSpan.FromMilliseconds(delay_time); } return Task.Delay(delay, cancellationToken); } |
If you are a user of the Microsoft Graph API .NET SDK, you can see that they handle the retry for you automatically, when you get 429’s or 503’s, as shown in the RetryHandler class. The default retry count is 10 times.
Way #2 – Using Polly
I am a huge fan of the Polly library. If you don’t know Polly, you don’t know what you have been missing out as a tool in your development. For short, Polly is a .NET resilience and transient-fault-handling library that allows developers to express policies such as Retry, Circuit Breaker, Timeout, Bulkhead Isolation, and Fallback in a fluent and thread-safe manner.
How can we implement the strategy done in standard .NET way (similar to the one found in the Microsoft Graph API SDK with the RetryHandler) with Polly? Using a few things: using the WaitAndRetryAsync method and using a Func to set the delay dynamically on the sleepProvider based on the Retry-After header (if found!).
Here’s a sample on how you can achieve that:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
private IAsyncPolicy<HttpResponseMessage> GetRetryPolicy(int numRetries = 10, int delayInMilliseconds = 10000) { var retryPolicy = Policy .Handle<HttpRequestException>() .OrResult<HttpResponseMessage>(r => r.StatusCode == HttpStatusCode.TooManyRequests || r.StatusCode == HttpStatusCode.ServiceUnavailable) .WaitAndRetryAsync(numRetries, sleepDurationProvider: (retryCount, response, context) => { var delay = TimeSpan.FromSeconds(0); // if an exception was thrown, this will be null if (response.Result != null) { if (!response.Result.Headers.TryGetValues("Retry-After", out IEnumerable<string> values)) return delay; if (int.TryParse(values.First(), out int delayInSeconds)) delay = TimeSpan.FromSeconds(delayInSeconds); } else { var exponentialBackoff = Math.Pow(2, retryCount); var delayInSeconds = exponentialBackoff * delayInMilliseconds; delay = TimeSpan.FromMilliseconds(delayInSeconds); } return delay; }, onRetryAsync: async (response, timespan, retryCount, context) => { // add your logging and what you want to do } ); return retryPolicy; } |
Pro tip: use the context to get your Logger instance
Hopefully these techniques can help you achieve your business goals.
