Web APIs: what you should not forget

In today’s day and age, you probably have heard of (REST) Web APIs as the way to consume data over the web. This is especially true with the emergence of microservices. You are probably building, have built or thinking of building a/some Web API(s). You are or may be planning to consume a/some Web API(s). In this post, I want to highlight certain things not to forget, on a technical stand point, when consuming and building Web APIs. Remember that those are just a start. There are multiple resources on the internet to help you adopt good practices. A good one that I recommend is the Microsoft API Implementation guide.

TL;DR

In building APIs:

Versioning
Verbs to nouns
Status codes overload
Response compression
Allow pagination
Fields filtering
Use UTC for Date/Time
Idempotent requests
Stress testing
Documentation

In consuming APIs:

Transient faults handling
Wrong usage of the HTTPClient class (in .NET)

Building APIs

Building APIs can be fun. You should remember it should also be fun to consume. Those APIs are going to be used by fellow developers and you want them to enjoy using your APIs. How can you help them have a pleasant experience? Below I highlight a few points when building your APIs, that can make the consumption a smooth sailing experience.

Versioning

You should be versioning your APIs because the code and the business requirements are in constant evolution. This will help you to smooth out changes to your APIs and slowly let your users migrate to your latest version without disrupting them too much. In .NET consider using a versioning library like ASP.NET API Versioning to help you version your API without too much hassle.

Implement, if you can, Hypermedia: a way to consume a Web API while allowing your clients to be decoupled and evolve independently. Like a web crawler, you should be able to grab the link section of the API from a table of content document. This means that your API resources should be auto-discoverable. With that mind set, you should never have to know the exact URL to access a resource.

Hypermedia is being slowly adopted by the APIs builders out there. However, it is still slow in adoption for a few reasons; one of them being that there isn’t a “fun” way yet to consume it and there isn’t a defacto specification for hypermedia (bummer!). Collection+JSON, JSON API, HAL, JSON-LD, and Siren are a few specifications which all have their strengths and weaknesses. You may want to read up on them. The ones that I have seen that are mostly used are JSON API and HAL. Siren is getting a lot of attention and is climbing up.

For the ones building RESTful Web APIs in .NET, you can have a look at the following libraries to help you get started with adding Hypermedia to your Web API. Remember that these are not the only ones out there.

halcyon: A HAL implementation for ASP.NET
JSONAPI: List of implementation libraries for JSONAPI
FluentSiren: Fluent API to build specification enforced siren entities.
Migrap.AspNetCore.Hateoas: HATEOAS (Hypermedia as the Engine of Application State) framework for ASP.NET Core

On that note, the dogmatic people/police of REST will tell you versioning is bad. But until there’s a consensus on the specifications (is there even going to be one?) and that the consuming of hypermedia, through libraries, becomes a lot easier, I am careful about going all out.

Verbs to Nouns

Your mindset should also shift from verbs to nouns. We are used to write our method names to start by a verb like get, create, add. Such naming convention is not clean in an API. Consider the following example:

Instead, you should think more in terms of REST verbs. Your querying should be able to be read like a sentence like

Status codes overload

Using status codes to return information in a Web API: good. Using too many status codes: bad. You want to keep it clean and simple. Having too many status codes makes it hard for the consumers to know what all the combinations are. What if you decide to add another code and the consumer is not aware of it?

Document what codes are returned by your API. Announce ahead of time any changes you make to your codes. This will help the consumers adapt themselves especially with their error handling strategy.

Response compression

Response compression should be available to your clients who request it. It can definitely diminish the network size of the transferred data. Having smaller data definitely lowers the request time. It is something that can be forgotten easily when building an API. Some may think it is a technical detail, but it is very valuable.

Allow pagination

Imagine a consumer making a request to get all the orders he ever made. Being an avid shopper of your site, he totals over 1000 orders. To reduce the size of the response request (even if it’s compressed), you may want to allow the user to page the results along with allowing him to pick his result set size.

There exists some ways to do this. You can use the envelop technique which wraps the result into an object with additional properties so your consumer know the size, etc or you can set them in a header (X-Paging-*). There is no right or wrong technique here. Pick one that suits your needs or how your consumers may prefer to consume your API.

Fields filtering

Piggy backing on my last example with the orders, an order can have a multitude of properties. Allow your users to filter which fields they want to receive. For example, when querying your API, they may only be interested in the total amount of each orders you have. As such, having a query parameter to allow filtering can make them receive what they want without having to retrieve all the properties of the order object.

Use UTC for Date/Time

Everyone has a timezone. You don’t know who in the world will be consuming your APIs. In order to have a date/time standard, consider transforming all your dates/time to the UTC standard and let the consumer convert the UTC date to his timezone.

Idempotent requests

As mentioned in the API Implementation guide of Microsoft, the code that implements these requests should not impose any side-effects. The same request repeated over the same resource should result in the same state (result) but the response status code can change.

Stress testing

So you finished building your API, and you are ready to put it in production. Have you ever considered stress testing your API to see how it reacts (memory, cpu, network, etc) when A LOT of requests come in? You wouldn’t want to put your API in production only to realize you have to put it offline because of a performance issue you haven’t really been aware of.

Consider using stress testing software like JMeter. They have a section on “how to build a web service test plan” so you can test your API.

Documentation

What is the first thing you do when you want to consume an API? You go through the documentation. Having a good documentation helps a developer know what your API is capable of, what it supports and can steer him/her into the right direction when coding. If you can, add examples on how to consume your resources. Detail your HTTP response codes. If you have authorization, go through the steps. The more examples you have, the easier it will be for us to get started. You want us to use your API, not sheer away from it. Check out an example how Microsoft documents its Cognitive Services API.

Consuming APIs

Transient faults handling

Now when it comes to consuming Web APIs, are you handling your transient faults? This is a common mistake I see when doing code reviews. Transient faults are errors that occur because of some temporary condition such as network connectivity issues or service unavailability. Typically, if you retry the operation that resulted in a transient error a short time later, you find that the error has disappeared¹. By the way, handling transient faults should not only be done on Web APIs, but on databases and anything that deals with network access.

In .NET, there are 2 libraries that I enjoy when it comes to handling transient faults: the Transient Fault Handling Application Block (TFHAB) from the Enterprise Library and Polly (check out my list of 3^rd parties library for .NET!).

Now there’s a catch. You can see here for instance, using the TFHAB, Microsoft has created a class that specifies which error codes (for databases), are transient. Some may or may not agree with all the codes there. When it comes to handling faults for a web client, you may not agree with all the defined exceptions/HTTP status codes that are considered transient. You have to determine what a transient fault is to you.

My buddy Alexandre Brisebois blogged a while back about Defining an HTTP Transient Error Detection Strategy For REST Calls using the Transient Fault Handling Application Block. Check it out!

Wrong usage of the HTTPClient class

In .NET, as Simon Timms described in his article, you have to be careful when using the HTTPClient class. Continuous instantiation and disposal of the HTTPClient object may create a socket exhaustion on your machine and affect performance. Consider reusing the HTTPClient object throughout your calls for better performance.

Before you go…

This is all nice and dandy. This gives you a few technical points of what you should not forget when building your APIs. But sometimes you have to think outside of the technical aspect. Watch out for my next post which I’d call existential. It will make you think a little more and may leave you with a retrospection on your APIs.