Encapsulate Context Representation

also known as: Add Context Metadata to Payload, Extract QoS Information from Headers

Context and Motivation

An API endpoint and its operations have been defined. API client and provider must exchange context information about their interaction, such as the client’s location, Quality-of-Service (QoS) control data, or data used to authenticate, authorize, and bill clients. API client-provider interactions might be part of conversations within and across API endpoints, possibly involving external systems as well.

As a conversation participant, I want to consolidate all context information in a single place and keep it close to the domain data so that clients and providers can prepare and process it along with that data. This also allows switching protocols if that is required to satisfy requirements and constraints that change over time.

Stakeholder Concerns

#developer-experience and #learnability

Accessing protocol headers is different from accessing message payload; different local APIs and/or platform-specific libraries have to be used. Consolidating information in the payload reduces the learning and implementation effort.

#interoperability and #modifiability

Less protocol-specific functionality means fewer changes are required when one protocol is replaced by another.

#security and #auditability

In multiprotocol scenarios, end-to-end security guarantees can only be given and enforced when the information in protocol headers is aggregated and correlated somehow. System and process assurance auditors appreciate if all relevant compliance information can be found in a single place (that is adequately protected) [Julisch et al. 2011].

Initial Position Sketch

Technical metadata such as API Keys, session IDs, or other QoS properties (for example, correlation identifiers, priority levels, time-to-live information, transactional policies, bandwidth requirements, packet-loss tolerance, or latency constraints) travel exclusively in the form of protocol headers. The initial position sketch in Figure 1 shows a response message with a payload and several protocol headers.

Figure 1: Encapsulate Context Representation: Initial Position Sketch. API client and provider exchange a message that contains context information as Metadata Elements in the protocol header.

The refactoring targets request and/or response messages of one or more operations. Operations that form a conversation may or may not appear in the same API endpoint.

Design Smells

Tight coupling to a communication protocol

Most of the network and communication protocols define their own header formats and fields; HTTP is an example. Some of these protocols support custom headers, and others do not. Using protocol-specific headers locks the communication participant in; this can be positive or negative, depending on context and requirements.

Quality-of-Service (QoS) fragmentation and dispersion

Several protocols might be used in conversations, such as HTTP, gRPC, and asynchronous messaging (AMQP). API clients and providers must go to multiple places to gather or produce all required context information, which can be error-prone, time-consuming, and cause technical debt.

Instructions

1. Design a data structure, the Context Representation, to represent the context information.
2. Add this data structure to the request and/or response operations of the targeted operations, add DTOs where necessary.
3. Implement a client-provider message exchange that produces and consumes instances of the new data structure.
4. Update the API Description with information about the syntax and semantics of the new message payload part. Provide data usage examples in the documentation, including valid and invalid values (or value ranges).
5. Inform clients about their new options and/or liabilities to work with the new Context Representation.

Note that it might be required to keep the context information in its current place, e.g., in protocol headers, for backward compatibility reasons. In this case, the refactoring allows clients to choose between the old and new ways of providing context information.

Target Solution Sketch (Evolution Outline)

Applying these steps leads to the solution sketched in Figure 2.

Figure 2: Encapsulate Context Representation: Target Solution Sketch. In addition to protocol-specific metadata transported in protocol headers, application-level Metadata Elements are bundled and included in the payload of the response message.

Example(s)

The following MDSL code snippet shows an API endpoint with an operation that expects context information in the headers:

data type KeyValuePair {
  "name": ID<string>,
  "property": D<string>
}+

endpoint type SampleService 
  exposes 
    operation sampleOperationInitial
      expecting 
        headers {
          "apiKey":ID<int>, 
          "sessionId":D<int>?, 
          "otherQosProperties":
            KeyValuePair* 
        }
        payload 
          "regularRequestPayload":D<string>
      delivering 
        payload "someUnspecifiedResponseData"

After the context information has been encapsulated, the apiKey, sessionId, and otherQosProperties from the header have moved. They now appear in a Data Transfer Object called RequestContext that is part of the request payload:

data type RequestContext {
  "apiKey":ID<int>, 
  "sessionId":D<int>?, 
  "otherQosProperties": KeyValuePair*
}

data type KeyValuePair {
  "name": ID<string>,
  "property": D<string> }+

endpoint type SampleService 
  exposes     
    operation sampleOperationTarget
      expecting 
        payload { 
         <<Context_Representation>> 
           "requestContext": RequestContext,
         <<Data_Element>> 
           "regularRequestPayload":D<string>
        }
      delivering 
        payload "someUnspecifiedResponseData"

The OpenAPI version of the source and target sketch is available here.

The online example for the Context Representation pattern shows a CustomerInformationHolderService whose getCustomerAttributes operation contains an explicit Context Representation in its request message payload.

Hints and Pitfalls to Avoid

Before and when encapsulating context information, make sure to:

• Decide whether the context has a local or global scope with respect to operation invocations in one or more API endpoints. The pattern variants discussed in Zimmermann et al. [2022] provide detailed information about this decision.
• Decide whether request or response messages should contain a message payload-level Context Representation (or both). Contextualizing requests is more common; for instance, think about client location, API user data, Wish List items, as well as credentials used to authenticate, authorize, and bill clients. Response contexts can also be observed in practice.
• Strive for a reusable data structure design across operations (and endpoints, if possible). Prefer de-jure or de-facto industry standards over own creations to define the inner structure of the QoS information in the Context Representation if possible. For example, RFC 7807 [Nottingham and Wilde 2016] defines a standard way to carry problem details in HTTP response messages.

It might be required but not possible to encrypt the data in protocol headers; this would be a reason why this refactoring is eligible. Suppose the payload is encrypted but contains context information used for message routing (for instance, in an API Gateway [Richardson 2018]). In that case, the refactoring might cause undesired decrypt/encrypt steps in the intermediary.

Related Content

Steps 1 and 2 of this refactoring can be seen as an instance of Introduce Data Transfer Object.

See the API Patterns website for a detailed description of the Context Representation pattern.

Undoing the content encapsulation is possible, but our Interface Refactoring Catalog does not contain an explicit inverse refactoring at present.

References

Julisch, Klaus, Christophe Suter, Thomas Woitalla, and Olaf Zimmermann. 2011. “Compliance by Design - Bridging the Chasm Between Auditors and IT Architects.” Computers and Security 30 (September): 410–26. https://doi.org/10.1016/j.cose.2011.03.005.

Nottingham, Mark, and Erik Wilde. 2016. “[Problem Details for HTTP APIs]{.nocase}.” Request for Comments. RFC 7807; RFC Editor. https://doi.org/10.17487/RFC7807.

Richardson, Chris. 2018. Microservices Patterns. Manning.

Zimmermann, Olaf, Mirko Stocker, Daniel Lübke, Uwe Zdun, and Cesare Pautasso. 2022. Patterns for API Design: Simplifying Integration with Loosely Coupled Message Exchanges. Addison-Wesley Signature Series (Vernon). Addison-Wesley Professional.