Inline Information Holder

also known as: Inline Resource Representation

Context and Motivation

An API provides several endpoints that give clients access to data-centric Information Holder Resources. The resources are related and reference each other via hyperlinks. For example, operational data such as an order in an e-commerce shop may reference a master data Information Holder describing the products.

Clients of the API are interested in the data of several of these linked Information Holder Resources. To access this distributed data, the clients have to send multiple requests.

As the API provider, I want to reduce indirection by embedding referenced information holder resources so that clients have to issue fewer requests when working with linked data.

Stakeholder Concerns (including Quality Attributes and Design Forces)

#performance

Both API clients and providers are interested in keeping the latency and bandwidth usage low and using as few resources as possible.

#usability and #developer-experience

An API that offers clients all the data they require with as few requests as possible is easier to use than an API where the client has to issue many requests and requires a complex state management.

Initial Position Sketch

A response message (2) of an API operation contains a link to a secondary resource that the client has to retrieve using a follow-up request (3):

Inline Information Holder: Initial Position Sketch

In terms of the API specification, the response Data Transfer Object (DTO) contains a link to the referenced resource (notation: OpenAPI):

paths:
  '/resource/{id}':
    get:
      summary: Get a single resource representation.
      produces:
        - '*/*'
      parameters:
        - name: id
          in: path
          description: the resource's unique id
          required: true
          type: string
definitions:
  ResponseDto:
    type: object
    properties:
      _links:
        $ref: '#/definitions/Links'
      referencedResourceId:
        type: string

Targets:

• an operation in an endpoint and its response message

Design Smells

Underfetching

Clients have to issue many requests to get the data they require, harming performance.

Leaky encapsulation

The implementation data model is leaking through the API. For example, the relational database schema has been exposed with one endpoint per table, and now clients have to resolve the foreign key relationships themselves.

Instructions (Steps)

Instead of providing clients with a hyperlink to fetch the related data, the API operation’s response message includes the referenced data:

1. To transfer the data, insert a new attribute to the DTO. When the API implementation is following the “code first” approach, directly add this DTO to the code and re-generate the specification; in “API first”, start with updating the API design specification and then re-generate the code.
2. From the implementation of the linked Information Holder Resource, copy the code to retrieve the additional entity or value.
3. Paste the code into the endpoint implementation and add the entity/value to the DTO.
4. If present, e.g. when using Hypertext as the Engine of Application State (HATEOAS), remove the link to the inlined resource. Only perform this removal if backward compatibility is not needed.
5. Adjust the tests to the new response structure.
6. Clean up the implementation code if necessary (observing the Rule of Three of refactoring Fowler [2018]), for example by moving duplicated code to a common implementation.
7. If under your control, adjust API clients to remove the now obsolete API calls.
8. Adjust API Description, version number, sample code, tutorials, etc. as needed.

Target Solution Sketch (Evolution Outline)

After the refactoring, the linked information is included in the initial response (2), saving the client an additional request:

Inline Information Holder: Target Solution Sketch

The implementation effort on the client also decreases: state management is less complex when fewer requests are needed to fetch required data. These benefits are countered by an increased message size, leading to increased transfer times and higher processing and database retrieval effort for the endpoint, which might not be needed by clients after all.

In terms of the API specification, the response DTO now contains the additional data (see the two lines at the bottom marked with +++).

paths:
  '/resource/{id}':
    get:
      summary: Get a single resource representation.
      produces:
        - '*/*'
      parameters:
        - name: id
          in: path
          description: the resource's unique id
          required: true
          type: string
definitions:
  ResponseDto:
    type: object
    properties:
      _links:
        $ref: '#/definitions/Links'
      referencedResourceId:
        type: string
 +++  referencedResource:
 +++    $ref: '#/definitions/ReferencedResourceDto'

If the refactoring only adds additional information to the response message, the refactoring is backward-compatible. If older clients still want to issue a second request, the referencedResourceId can be kept in the response. Otherwise, the reference link can be removed or marked as deprecated.

Example(s)

The Policy Management backend microservice of Lakeside Mutual, a fictitious insurance company, contains an endpoint to retrieve the details of a specific policy, along with a reference to the customer through their customerId:

curl http://localhost/policies/fvo5pkqerr
{
  "policyId" : "fvo5pkqerr",
  "customerId" : "rgpp0wkpec",
  "creationDate" : "2021-07-07T13:40:52.201+00:00",
  "policyPeriod" : {
    "startDate" : "2018-02-04T23:00:00.000+00:00",
    "endDate" : "2018-02-09T23:00:00.000+00:00"
  },
  "policyType" : "Health Insurance",
  "deductible" : {
    "amount" : 1500.00,
    "currency" : "CHF"
  },
  ...
}

We start the refactoring by adding a new attribute to the DTO:

public class PolicyDto extends RepresentationModel {
    private String policyId;
--- private String customerId;
+++ private CustomerDto customer;
    private Date creationDate;
...

To provide the value for the customer, the endpoint implementation uses the customerService to look-up the customer and adds it to the response:

@ApiOperation(value = "Get a single policy.")
@GetMapping(value = "/{policyId}")
public ResponseEntity<PolicyDto> getPolicy(
    @ApiParam(value = "the policy's unique id", required = true) @PathVariable PolicyId policyId) {
    logger.debug("Fetching policy with id '{}'", policyId.getId());
    Optional<PolicyAggregateRoot> optPolicy = policyRepository.findById(policyId);
    if(!optPolicy.isPresent()) {
      final String errorMessage = "Failed to find a policy with id '{}'";
       logger.warn(errorMessage, policyId.getId());
        throw new PolicyNotFoundException(errorMessage);
    }
    PolicyAggregateRoot policy = optPolicy.get();
    PolicyDto response = PolicyDto.fromDomainObject(policy);
+++ CustomerDto customer = customerService.getCustomer(policy.getCustomerId());
+++ response.setCustomer(customer);
    return ResponseEntity.ok(response);
}

The customer data is now inlined in the response. Note that in this example, the customerId was removed, so this makes the refactored operation backward incompatible.

curl http://localhost/policies/fvo5pkqerr
{
  "policyId" : "fvo5pkqerr",
  "customer" : {
    "customerId" : "rgpp0wkpec",
    "firstname" : "Max",
    "lastname" : "Mustermann",
    "birthday" : "1989-12-31T23:00:00.000+00:00",
    "streetAddress" : "Oberseestrasse 10",
    "postalCode" : "8640",
    "city" : "Rapperswil",
    "email" : "admin@example.com",
    "phoneNumber" : "055 222 4111",
    "moveHistory" : [ ],
  },
  "creationDate" : "2021-07-07T13:40:52.201+00:00",
  "policyPeriod" : {
    "startDate" : "2018-02-04T23:00:00.000+00:00",
    "endDate" : "2018-02-09T23:00:00.000+00:00"
  },
  "policyType" : "Health Insurance",
  "deductible" : {
    "amount" : 1500.00,
    "currency" : "CHF"
  },
  ...
}

Hints and Pitfalls to Avoid

The referenced Information Holder should be part of the same service. Otherwise, performing the refactoring might introduce undesired dependencies between the backend services.

An endpoint may now work with more backends or databases than before, which might not be desired from a separation of concerns standpoint (think about role- or attribute-based authorization).

See the Embedded Entity and Linked Information Holder patterns for a deeper discussion of the benefits and liabilities of each pattern.

Related Content

Extract Information Holder inverses this refactoring.

The Wish List and Wish Template patterns offer an alternative solution to the problem of how an API client can inform the API provider at runtime about the data it is interested in.

The Backends For Frontends pattern by Sam Newman is another approach to tailoring a backend to the specific needs of a client.

References

Fowler, Martin. 2018. Refactoring: Improving the Design of Existing Code. 2nd ed. Addison-Wesley Signature Series (Fowler). Boston, MA: Addison-Wesley.