Externalize Context Representation

also known as: Add Context Metadata to Payload, Extract QoS Information from Headers

Context and Motivation

Context Representation is a pattern that may serve as a refactoring target. The context of this pattern is:

An API endpoint and its operations have been defined. Context information has to be exchanged between API client and provider. Examples of such context information are client location and other API user data, the preferences forming a Wish List, or Quality-of-Service (QoS) information such as credentials that are used to authenticate, authorize, and bill clients. Such credentials may be API Keys or JWT claims.

It then states:

Interactions between API client and API provider might be part of conversations, consisting of multiple related operation calls. API providers can in turn also act as API clients that consume services provided by other APIs (in their implementations) to create operation call chains. Parts of the context information are local to single operations, others are shared and handed over from operation invocation to operation invocation in conversations or call chains.

As conversation participant, I want to consolidate all technical metadata in a single place and keep it close to the domain data so that clients and providers can prepare and process it jointly and so that protocols can be switched if that is required to satisfy requirements and constraints that change over time.

Stakeholder Concerns (including Quality Attributes and Design Forces)

#developer-experience and #learnability

Accessing protocol headers is different from accessing payload; different local APIs and/or platform-specific libraries have to be used. Consolidating information in the payload reduces the learning effort.

#interoperability and #modifiability

The less protocol-dependent functionality is used, the fewer misunderstandings between communicating parties can occur, and the fewer changes are required when a protocol is replaced by another.

#security and #auditability

In multi-protocol scenarios and long-running client-provider invocation sequences, end-to-end security guarantees can only be given and enforced when the information in protocol headers is aggregated and correlated somehow. System and process assurance audits appreciate if all relevant compliance information can be found in a single place (that is protected properly).

Initial Position Sketch

Quality-of-Service (QoS) metadata travels in the form of protocol headers:

data type KeyValuePair {
    "name": ID<string>,
    "property": D<string> }+

endpoint type SampleService 
  exposes 
    operation sampleOperationInitial
      expecting 
        headers {
          "apiKey":ID<int>, 
          "sessionId":D<int>?, 
          "qosPropertiesThatShouldNotGoToProtocolHeader":KeyValuePair* }
        payload 
          "regularRequestPayload":D<string>
      delivering 
        payload "someUnspecifiedResponseData"

The refactoring targets request and/or response messages of operations. These operations may or may not appear in the same endpoint within an API.

Design Smells

Tight coupling to a communication protocol

Most networking and communication protocols define their own header formats; HTTP is an example. Some of these protocols support custom headers, others do not. Using protocol-specific headers locks the communication participant in; this can be positive or negative, depending on context and requirements.

Quality-of-Service (QoS) fragmentation and scattering

One or more protocols might be used in the same system. API clients and providers have to go to multiple places to gather or produce all required non-functional metadata. This might be error-prone and time-consuming and cause technical debt.

Instructions (Steps)

1. Design the data type to represent context information. Provide data usage examples, including valid and invalid values (or value ranges).
2. Add this Context Representation data structure to the request and/or response operations of the targeted operations.
3. Implement a client-provider message exchange that produces and consumes instances of the new data structure.
4. Update the API Description with information about the syntax and semantics of the new payload part.
5. Inform clients about their new options and/or liabilities to work with the new Context Representation.

Target Solution Sketch (Evolution Outline)

The following UML class diagram shows where an explicit Context Representation can be placed:

The apiKey, sessionId, and qosPropertiesThatShouldNotGoToProtocolHeader from the header in the “Initial Position Sketch” above have moved. They now appear in a Data Transfer Object called RequestContext that is part of the request payload:

data type RequestContext {
    "apiKey":ID<int>, 
    "sessionId":D<int>?, 
    "qosPropertiesThatShouldNotGoToProtocolHeader":KeyValuePair* }

data type KeyValuePair {
    "name": ID<string>,
    "property": D<string> }+

endpoint type SampleService 
  exposes     
    operation sampleOperationTarget
      expecting 
        payload { 
         <<Context_Representation>> "requestContext": RequestContext,
         <<Data_Element>> "otherPayload":D<string>
        }
      delivering 
        payload "someUnspecifiedResponseData"

The OpenAPI version of the source and target sketch is available here.

Example(s)

The example in the Context Representation pattern shows a CustomerInformationHolderService whose getCustomerAttributes operation contains an explicit Context Representation in its request message payload.

Hints and Pitfalls to Avoid

Before and when externalizing context information, make sure to:

• Decide whether the context has a local or a global scope (with respect to operation invocations in one or more API endpoints). See discussion of pattern variants for information about this decision.
• Decide whether request or response messages should contain payload-level Context Representation (or both). Contextualizing requests is more common; for instance, think about client location, API user data, Wish List items, as well as credentials used to authenticate, authorize, and bill clients. But response contexts can also be observed in practice sometimes.
• Strive for a reusable data structure design across operations (and endpoints, if possible). Prefer de-jure or de-facto industry standards over own creations to define the inner structure of the QoS information in the Context Representation if possible (for instance, legally and from a licensing perspective).

Note that it might not be possible to encrypt protocol headers; this would be a reason why this refactoring has to be applied. If the payload is encrypted but contains context information that is used for message routing (for instance, in an API Gateway), the refactoring might cause undesired decrypt/encrypt steps in the intermediary.

Related Content

Steps 1 and 2 of this refactoring can be seen as an instance of Introduce Data Transfer Object.

See the MAP website for a detailed description of the Context Representation pattern.

Undoing the content externalization is possible, but our Interface Refactoring Catalog does not contain an explicit inverse refactoring at present.