Tighten Evolution Strategy

Context and Motivation

An API runs in production. Its compatibility and extensibility characteristics are expressed in the form of one or more evolution patterns, Experimental Preview, Aggressive Obsolescence, Limited Lifetime Guarantee, Two in Production. These patterns differ in the way and the frequency the provider is able to introduce breaking changes.¹

As an API provider, I want to extend the stability, lifetime and support guarantees for an API endpoint with its elements (e.g., operations, messages) so that existing and new clients can use the API longer and rely on the stability of its API Description.

According to “Interface Evolution Patterns — Balancing Compatibility and Extensibility across Service Life Cycles” [Lübke et al. 2019], Evolution patterns are confronted with desired qualities such as:

1. “Enhancing compatibility and developer experience
2. Allowing the provider and the client to follow different life cycles, e.g., a provider can roll out a new API version without breaking existing clients
3. Minimizing changes to the client forced by API changes
4. Making it possible for the provider to improve and extend the API and change it to accommodate new requirements
5. Guaranteeing that API changes do not lead to semantic ‘misunderstandings’ between client and provider
6. Minimizing the maintenance effort to support old clients.”

Stakeholder Concerns

Different stakeholders exist both on the API provider and the API client side, including product owners/managers, integration architects, API and API client code developers and maintainers, testers, and auditors. The API contracts (API Descriptions, Service Level Agreements) might be owned by one or more of these roles. While a detailed discussion of these stakeholders groups/types is out of scope of IRC, it is worth noting that client- and provider-side stakeholders might have opposing interests; API pricing is an example.

#cost

Effort comes with cost. Operational and capital expenditure, including budget for training, evolution (including documentation), and operations, increases as lifetime guarantees are extended.

#flexibility

The more lifetime guarantees are given and the longer the committed time spans are, the less freedom remains to maintainers with respect to changing product vision, roadmap, and corresponding API designs (for instance, the role of an API in the organization).

#maintainability

During their lifetimes, API versions have to be supported and evolved. Security patches are applied, bugs are fixed, and so on. The effort for such activities might not be proportional to the guaranteed lifetime but certainly is related to it. Related (sub-)concerns are compatibility and extensibility [Zimmermann et al. 2022].

#stability

Stability has two different meanings according to the arc 42 quality model. Stability at runtime means being free from severe errors that cause interruption of system function. Stability in development means that existing parts can remain largely unchanged when adding new features. API clients welcome both reliability and dependability.

Initial Position Sketch

Clients are not clear enough about what they can expect from the provider. See Figure 1 for the initial position sketch.

Figure 1: Tighten Evolution Strategy: Initial Position Sketch. Two clients are using an API, but the provider has not given any lifetime or support guarantees.

This refactoring targets API endpoints and their API Descriptions (including business and technical contract between a provider and its clients). It may also affect any Service Level Agreement (SLA) that accompanies an API Description.

Smells

Client community smaller than expected

The provider receives less client traffic than expected and hoped for. Client feedback indicates that the API functionality is appreciated, but API usage is not considered a viable design option due to missing stability and support guarantees.

Lack of trust and confidence

Client developers decide not to use the API because it comes across as unstable and subject to change often; a risk that it might disappear is perceived.

Missing decisions/documentation about evolution policies

Unanswered design questions about the evolution strategy lead to an underspecification in the API Description and/or complementary Service Level Agreement. This lack of clarity makes it difficult to plan ahead for API client developers and product managers of software that serves such clients.

Instructions

The following transitions and combinations of evolution patterns make sense to increase guarantees:

• Make explicit that current API version is an Experimental Preview.
• Transition from Experimental Preview to Aggressive Obsolescence.
• Become more assertive and switch from Aggressive Obsolescence to Limited Lifetime Guarantee.
• Change from any other life cycle pattern to Two in Production.
• Refine from Two in Production to its N in Production variant (with N equal to or greater than three).

Figure 2 illustrates these common transitions in a state diagram. Note that it is possible to give guarantees such as Limited Lifetime Guarantee or Two in Production without having offered an Experimental Preview before; we do not show these two transitions in the figure for the sake of clarity.

Figure 2: Transitions to tighten lifetime guarantees of an API.

Target Solution Sketch (Evolution Outline)

The life cycle guarantee that is targeted is not an architectural construct but a documentation element. The sketch in Figure 3 shows that evolution patterns can also be combined (here: Two in Production with a Limited Lifetime Guarantee).

Figure 3: Tighten Evolution Strategy: Target Solution Sketch. The API Provider offers the API endpoint as Two in Production with a Limited Lifetime Guarantee.

Some Interface Definition Languages (IDLs) have dedicated support for such Metadata Elements (for instance, OpenAPI Specification and MDSL); in other documentation formats, lifecycle and version information usually goes to more general description fields or comments.

Example(s)

When applying this refactoring, an MDSL contract changes from:

API provider SampleAPIProvider1 version 1.0.0
  offers SomeDemoContract 
    at endpoint location "http://www.testdomain.io/path/subpath"
    via protocol HTTP 
  binding 
    resource SampleAPIProvider1Resource
  provider governance AGGRESSIVE_OBSOLESCENCE

to:

API provider SampleAPIProvider1 version 2.0.0
  offers SomeDemoContract 
    at endpoint location "http://www.testdomain.io/path/subpath"
    via protocol HTTP
  binding 
    resource SampleAPIProvider1Resource
  provider governance LIMITED_LIFETIME_GUARANTEE

The Evolution Patterns web pages feature real-world known uses of the patterns (and their introduction over time).

Hints and Pitfalls to Avoid

Apply this refactoring when existing clients complain, for instance about a lack of planning horizon and effort to catch up with API updates.

When applying the refactoring, perform a cost-benefit analysis. Communicate changes early, clearly, and continuously. Minimize incompatible changes and apply Semantic Versioning when releasing.

Do not over-commit but find a balance between conflicting concerns, for instance those of external stakeholders and those of the API development team.

Related Content

This refactoring is the opposite of Relax Evolution Strategy.

Introduce Version Mediator can be applied if lifetime guarantees are weakened and some clients are unable to migrate to a newer, incompatible version. Introduce Version Identifier might have to be applied before Tighten Evolution Strategy so that clients can learn about versions and their (in-)compatibilities.

MDSL specifications have an optional Service Level Agreement section and a provider governance element in the API provider part. This is explained on the MDSL website.

References

Lübke, Daniel, Olaf Zimmermann, Cesare Pautasso, Uwe Zdun, and Mirko Stocker. 2019. “[Interface Evolution Patterns: Balancing Compatibility and Extensibility across Service Life Cycles]{.nocase}.” In Proceedings of the 24th European Conference on Pattern Languages of Programs. EuroPLop ’19. New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3361149.3361164.

Zimmermann, Olaf, Mirko Stocker, Daniel Lübke, Uwe Zdun, and Cesare Pautasso. 2022. Patterns for API Design: Simplifying Integration with Loosely Coupled Message Exchanges. Addison-Wesley Signature Series (Vernon). Addison-Wesley Professional.

1. Alternatively (but only justified in rare cases), the provider might decide not to do so at all and guarantee an Eternal Lifetime Guarantee. ↩