Designing APIs (Satter Ramblings)

I saw a tweet asking about best practices for APIs. I started writing a comment and realized very quickly that a post would be better. I have tried to be technology agnostic.

Think before Coding aka Design

First off, do not just code up an API based on your current needs. Yup, don't agile code the API and think you can refactor your way out of problems. In this case, coding is the dirty word, not design. Code first can be perilous. You can end up with bad code including state embedded in your auth token, meanless groupings of endpoints, inconsistent naming, to name a few. You don't need the design to handle everything coming over the life of the API. Do think about the problem set, target audience, and where the API fits in the architecture.

Before you even design the API you might want to answer the following questions;

What assumptions are you making?
What is the problem domain or goal?
Who is the target audience?
Will your API be Open Source Software (OSS)?

Assumptions

The most important item is to write out all of your assumptions. Believe me, you are making assumptions even if the assumption is not to assume. Assumptions are not bad as long as you document those assumptions.

Some assumptions to document:

Compatibility between API versions
3rd party dependencies used by your API
API supports internal usage, external usage, or both
Other assumptions

Tell your users if they can rely on your API as versions change. Some companies do not care about backward compatibility other companies like Microsoft try hard to stay backward compatible. I use to think that changing an implementation was ok as long as the inputs and outputs did not change. Then I joined the .NET Framework team. A proposal was made to change how a method worked. One problem, developers had come to rely on a side effect. Thus even though input and output didn't change, it still was a breaking change.

Will you have any 3rd party dependencies. This is a critical assumption to answer. A component, I have used in the past took hard dependencies. Since this was an internal API, this resulted in my codebase using some older components because of this one dependency. Having dependencies is not bad however make sure you do not force users of your API from upgrading those dependencies.

Another issue with 3rd party dependencies is to ensure compliance with the 3rd party component. This is especially true of internal APIs. Your choice of dependencies can have consequences for your users. The canonical example being using a GPL component. This may force your users to open source their code. I will talk more about OSS below.

Another common assumption is how is this API going to be used. Is this an app internal API, running remote, or both? When I design my APIs I don't make an assumption of where the API will run. I learned this in the middle of my career when I created a web service based home automation system. The system had several ways for users to interact with the system using Flash, web pages, remotes, keypads, etc. It made sense that the Flash client would make a web call. However, it made zero sense that the web pages would do the same thing. When your performance metric was the system had to be as fast as a light switch, any latency kills.

In addition, even if the API is a server-side API, will you provide client API. If you do provide a client API, what technology stacks will you support at first and in the future?

Finally, write out all your assumptions. You may not put them in your documentation. However, writing them out will help you and future developers with maintaining and extending the API.

Domain

Make sure you define your domain and subdomains. The terminology and knowledge of the domain can lead to better API design and usage.

One thing I have found useful is to create a paper prototype. Then use the paper prototype and have others try it as well. Just like popular apps typically have good UI/UX. Make sure your API has a good developer experience. You might even catch things that you missed in the original design and before coding. APIs can be hard to change once adopted.

Target Audience

Is this API being used by just you, your team, your organization, customers, or partners? While it should not matter, answering this question can impact the design and implementation. While I said to not code first, the reality is if it is just for you or your team you can. Things like compatibility and assumptions are much less important when the API has a small user base. Once you start talking about customers and partners then the expectations for the API rises.

External users expect:

Documentation including examples and depending on the API in multiple technology stacks
Security
Support
Usability
Localization

Open Source Software

There are two parts to Open Source Software consideration. Should you use OSS and should some or all of your API be OSS? What follows is extremely simplified guidance. Keep in mind that OSS is not just a technical decision. It is also a business decision as it can expand or collapse your potential user base. In addition, there are legal issues as well. This is a complex topic so do your research (https://dzone.com/articles/what-a-developer-needs-to-know-about-open-source-s) and you or your manager needs to bring business stakeholders into the conversation along with legal.

The first part should you use OSS is typically the easiest to answer. For most people, the answer is yes with one caveat. Make sure your use of OSS does not force your customers into something they may not want like having to open source their code. Note there are new licenses that can impact even users of a SAAS product. Get a good attorney who knows OSS licensing and compliance.

Should some or all of your software be OSS is even more complex. You might actually make some of your API OSS. Take a look at companies like Microsoft. The actual source code for Azure services are typically not OSS. However, the code needed to use the services is typically OSS and for good reason. Making the code needed to access and use the service may widen or collapse the potential user base. Most developers want OSS and more importantly, they want an active community when they run into issues. If you do employ an OSS strategy make sure you understand the impact.

Things to consider:

Will the benefits outweigh the cost?
What license model to use?
Allow the community to contribute source code?
What is the community process and requirements if accepting source code?
Who and how to manage the community?
Should the development process be done fully or partially in the open?

OSS is a complex topic. If you want assistance, feel free to contact me [email protected]. I worked on the initial OSS for Microsoft, https://www.satter.org/2018/04/microsoft-open-source.html.

Tips for design

The following are practices I have used to build API or wish the API developers had done.

Think of the API as Interfaces even if the implementation technology doesn't have an interface construct. Separating out the API definition from the implementation can lead to better APIs, especially if the audience is or will be in the future external developers. The external developers could be another group or organization. By thinking in terms of an interface, you can better support other technology stacks.

Make sure the interface is stateless or more accurately order independent. For example, create, read, update, and delete (CRUD) APIs should not have any order dependence. As long as the caller knows the required parameters they should be able to call for example delete without doing a read first. Definitely, do not require coordination of calls to do work.

Authorization is not a storage mechanism for state. I once used an API that used attributes of the authorization token. In order, to properly use the API certain data had to exist. The designers made assumptions about how the app would interact with the backend. This made debugging hard as you constantly had to decide was the endpoint failing or did I forget to set the proper values in the token.

While debugging was sometimes painful. A bigger issue was when one day a feature was requested that did not require one of the assumptions. This lead to some less than ideal UX and code.

Be consistent in your naming. Not only within areas of the API but across the API and domain. Naming can make or break an API. <rant>Personally one of the reasons I feel .NET Framework WPF took so long for broad adoption is due to names that were radically different from Winforms. IMHO, if WPF team had taken a goal of change the namespace from Winforms to WPF and WPF just worked. WPF adoption would have been nearly 100% overnight.</rant>

When returning lists of data think about the list. Can the developer

Fully define the search criteria?
Can data be returned in pages to speed up the perceived UI response?
Define how the data is sorted?
Specify the format of the data, for example, JSON or XML?

For languages that have threading syntax like .NET don't force the developer to use it. Your API should allow the developer to decide if he was to have the API call be in a separate thread. I believe a lot of .NET community seems to think threading is a magic bullet for performance. It is not and when misused it actually makes apps slower! Internally if your API needs to be threaded then do it but don't force it.

Keep things simple. My personal approach is that my API is focused on two groups. Group one is someone who justs wants to get the API hooked up and working. The other group is someone who wants full control over what happens. I create the API for the control freaks. Then I write overloaded methods and facades that make using the API as simple as possible. A good example is a method that returns data. I have a method that takes in everything to fully control filtering, sorting, paging, and the format. Then I have a version that could be as simple as GetOrders using all the default values. The method would return all the orders the caller is authorized to see.

Write the API as a class that implements the interface. Then write the code that exposes the API be a microservice, serverless function, REST, etc. That way you have multiple ways the code can be called. For example, a web page using a form and post back can call the API using dependency injection. A SPA or mobile app could call the API as a web service. A machine to machine interface could be as simple as dropping a message on a message queue. The same code implements the API and then you have wrappers around it to handle the call mechanism. Thus, the actual call mechanism concerns are separated out from the actual API work. In addition, the API is typically easier to test when separated out from the call mechanism.

Document, document, document. You can use a tool to document but make sure to go back and add to it. Read the document like you have never used the API and may have no knowledge of the domain. Is the documentation enough. Does it include the assumptions that users need to know?

Other tips: