What is API Drift, and What Can You Do About It?

According to a recent report from APIContext, 75% of production APIs do not match their specifications. That is an incredibly worrisome statistic. The majority of production APIs do not agree with their contract, which means many developers are ultimately left in the dark about how APIs function.

This figure indicates API drift, a worrying trend in the industry. But what is API drift specifically, and how does it come about? More importantly, how can we mitigate it? In this piece, we’re going to dive into API drift and give you some best practices to ensure proper handling of this huge issue.

What is API Drift?

APIs are highly complex systems. Beyond the obvious code base that comprises the API itself, you must also consider the resources it acts upon, the environments it traverses, the gateways and routers implemented to navigate multiple services, and much more. This complexity can easily overwhelm developers, and as such, APIs are as much about deploying a service as they are about planning the service and its lifecycle.

This planning can help untangle the complexity of the service, helping to navigate the murky waters. However, the problem with this planning is the assumption that it is preemptive — that planning occurs at the beginning stages of development, and then the “real work” begins. The reality is that this planning is paramount to the continual health and iteration of the service, providing guardrails for future development to ensure the API is growing in an expected and supportable way.

Unfortunately, APIs often drift away from this planning. When this occurs, the API implementation shifts away from its initial design, specification, or expected behaviors. Although it’s common for APIs to shift as business needs change, drift is the uncontrolled or poorly managed shifting away from the agreed-upon contract and understanding of the API.

API Drift Symptoms

API drift has some very clear side effects. For instance, drift results in a situation in which the documentation doesn’t accurately reflect the API or expectations of the API itself. Let’s dive into what these symptoms look like in the real world.

Firstly, API drift introduces substantial inconsistency with internal and external documentation. As the API shifts away from the core contract, the documentation built around the initial understanding will grow more and more distant from the reality of the code itself. Updates to the code can lead to outdated documentation, forming a substantial disagreement between expected form and function and the realities of the code in implementation.

This incongruence can also introduce issues with developer and user frustration. When end users expect a specific function, they often turn to documentation to understand what went wrong. When the code and documentation do not match, this can cause frustration at best and dangerous utilization at worst.

Arising from this frustration, it’s not uncommon to see high technical debt start to evolve. With developers scrambling to understand the API that has seemingly drifted so far from its specification and documentation, they might start making “good enough” solutions that introduce excess cost. This can further impact ongoing development and future maintenance, making for an unhealthy API with a bad consumption pattern.

The very core of the API can suffer from these effects as well. As developers make changes to the API interface, such as altering data formats, changing response forms and structures, or even implementing entirely new calls that are poorly documented, the ongoing use of the API can break, resulting in broken backward compatibility. This can cause existing clients to fail or behave in a way that feels fundamentally random.

Ultimately, the worst symptom of API drift can best be summed up as being “unpredictable.” When the API has drifted from its core contract, it will behave in ways that seem unpredictable to the end user, resulting in significant issues of use and degradation of developer morale.

Causes of API Drift

API drift is fundamentally an issue of poor planning and lack of rigorous controls. Accordingly, the causes of the drift itself are relatively obvious once you start digging into the problem at scale.

Firstly, poor testing and controls are a huge concern. Insufficient testing of the API and its interface against a well-documented stable contract can result in development that deviates from this core base. From this development, additional deviation can easily be introduced with minimal tracking to control it, resulting in unguided development which exacerbates the issue.

Inadequate processes for documentation maintenance also play a huge role in this issue. Changes to code should always be accompanied by updates to documentation, and when this rigorous control is not implemented, discrepancies can arise. Over time, these can compound, resulting in fundamentally divergent documentation.

Poor planning for feature development is also a major concern. While new features are welcome in any development process, this should be done with specific guardrails and processes. When this is poorly planned, and features are introduced in an ad hoc or per situ manner, this can result in new features that push the API away from its core contract without the documentation or planning necessary to integrate these shifts into the core business and operational precepts.

Mitigating API Drift

Thankfully, there are some actionable steps that any organization can implement to mitigate API drift quite effectively.

First and foremost, employ consistent guardrails in your development lifecycle. Regularly reviewing your code against the API contract, either through automated testing or manual review, will help to ensure continual development aligns with the mission and the form factor of the API. Validating the specification against the API — and the specification against the business case — will go a long way towards ensuring that continual development effectively syncs against the core purpose of the service.

Versioning can also help this effort tremendously. Versioning sets apart breaking changes and new features from the core codebase, and as such, is a forcing function to ensure adequate coverage and documentation. When a version requires documentation and explanation, it’s harder to sidestep this process. That said, a culture of adequate documentation alongside this versioning effort is paramount.

Documentation review and change management is also a huge step in the right direction. The concept of documentation review is to regularly review and update the API documentation itself throughout the API development, and this action will significantly improve outcomes by itself. When paired with effective change management, in which design reviews, impact assessments, and other approaches are taken to guide change, an effective control system will be created that can ensure you are developing against the contract rather than in competition with it.

Final Thoughts

The good news is that API drift is easy to manage if you adopt some general guardrails and processes. The bad news is that it is infinitely easier to not do this, and accordingly, resolving these issues takes consistent and intentional effort. The best practices outlined herein can help significantly deal with these issues, but you must implement them now and be steadfast in this implementation.

What do you think of these best practices? Have we missed any other solutions to the problem of API drift? Let us know in the comments below!