Inline Information Holder

also known as: Inline Resource Representation, Embed Entity

Context and Motivation

An API provides several endpoints that give clients access to data-centric Information Holder Resources. The resources are related and refer to each other, for instance, via hyperlinks. For example, operational data such as an order in an e-commerce shop may reference a Master Data Holder describing the products.

Clients of the API are interested in the data of several of these linked Information Holder Resources. To access this distributed data, the clients have to send multiple requests.

As the API provider, I want to reduce indirection by embedding data that is available from one or more referenced Information Holder Resources so that my clients have to issue fewer requests when working with linked data.

Stakeholder Concerns

#performance, #green-software

Both API clients as well as providers are interested in keeping the latency and bandwidth usage low and using as few resources as possible.

#usability, #developer-experience

An API that provides clients all the required data with as few requests as possible may be easier to use than an API where the client has to issue many requests and requires complex state management on the client side to keep track of multiple API calls. See the blog post API Design Review Checklist: Questions Concerning the Developer Experience (DX) for hints on improving the developer experience.

#offline-support

When the connection is unstable or if one wants to build offline functionality into an app, one large request is often better than many small ones.

Initial Position Sketch

The initial response shown in Figure 1 contains a Link Element that refers to another information holder of the API. The client has to follow the link to retrieve the data of the referenced resource.

Figure 1: Inline Information Holder: Initial Position Sketch: A response message (2) of an API operation that is requested (1) contains a link to a secondary resource that the client has to retrieve using a follow-up request (3, 4).

In terms of the API specification, the response Data Transfer Object (DTO) PolicyDto contains the customerId of the referenced customer and a link to it (notation: OpenAPI Specification, simplified for brevity):

paths:
  '/policies/{policyId}':
    get:
      summary: Get a single policy.
      parameters:
        - name: policyId
          in: path
          description: the policy's unique id
          required: true
          type: string
      responses:
        '200':
          description: A single policy.
          schema:
            $ref: '#/definitions/PolicyDto'
definitions:
  PolicyDto:
    type: object
    properties:
      ...
      customerId:
        type: string
      link:
        type: string

The response message uses a DTO to transfer the data.

This refactoring targets an operation in an endpoint and its response message.

Design Smells

Underfetching

Clients have to issue many requests to get the required data, harming performance.

Leaky encapsulation

The implementation data model is leaking through the API. For example, a relational database has been exposed via an API with an endpoint for each table in the database, and now clients must resolve the foreign key relationships between tables.

Instructions

Instead of providing clients with a hyperlink to fetch the related data, the response message of the API operation includes the referenced data:

1. Decide which linked data to inline/embed into the message. For a discussion of the tradeoffs involved, see the Embedded Entity and Linked Information Holder patterns [Zimmermann et al. 2022].
2. To transfer the data, insert a new attribute to the DTO.
3. Retrieve the additional entity or value from the repository and add it to the DTO instance.
4. If present, e.g., when using Hypertext as the Engine of Application State (HATEOAS), remove the superfluous link to the resource whose data is now inlined. Only perform this removal if backward compatibility is not needed.
5. Adjust the tests to the new response structure.
6. Clean up the implementation code if necessary (observing the “Rule of Three” of refactoring¹), for example, by moving duplicated code to a common location.
7. Adjust API clients under your control to remove obsolete API calls, but find and use the inlined data instead.
8. Adjust API Description, version number, sample code, tutorials, etc., as needed.

The link to the referenced resource can be kept in the response message to maintain backward compatibility. In this case, old clients can still follow the link, and updated clients can use the inlined data directly.

Target Solution Sketch (Evolution Outline)

After the refactoring, the linked information is included in the initial response, saving the client an additional request. This solution is sketched in Figure 2.

Figure 2: Inline Information Holder: Target Solution Sketch: The client requests (1) a resource through the API. The API implementation responds with a rich response message (2) that contains all the data.

The implementation effort on the client also decreases: state management is less complex when fewer requests are needed to fetch required data. These benefits are countered by increased message size, leading to longer transfer times and higher processing and database retrieval effort for the endpoint, which might not be needed by clients after all.

Regarding the API specification, the response DTO now contains the additional data (see the lines at the bottom marked with +++). The link to the referenced resource can be removed if backward compatibility is not needed (see the lines marked with ---).

paths:
  '/policies/{policyId}':
    get:
      summary: Get a single policy.
      parameters:
        - name: policyId
          in: path
          description: the policy's unique id
          required: true
          type: string
      responses:
        '200':
          description: A single policy.
          schema:
            $ref: '#/definitions/PolicyDto'
definitions:
  PolicyDto:
    type: object
    properties:
      ...
      customerId:
        type: string
 +++  customer:
 +++    type: object
 +++      properties:
 +++        customerId:
 +++          type: string
 +++        firstname:
 +++          type: string
 +++        lastname:
 +++          type: string
 ---  link:
 ---    type: string

Example(s)

The Policy Management backend microservice of Lakeside Mutual, a fictitious insurance company, contains an endpoint to retrieve the details of a specific policy, along with a reference to the customer through their customerId. The following listing shows two requests made using the curl command line tool. It sends an HTTP GET request to the specified URL. The response to this request is a JSON object.

curl http://localhost/policies/fvo5pkqerr

{
  "policyId" : "fvo5pkqerr",
  "customerId" : "rgpp0wkpec",
  "creationDate" : "2021-07-07T13:40:52.201+00:00",
  "policyPeriod" : {
    "startDate" : "2018-02-04T23:00:00.000+00:00",
    "endDate" : "2018-02-09T23:00:00.000+00:00"
  },
  ...
}

curl http://localhost/customers/rgpp0wkpec

{
  "customerId" : "rgpp0wkpec",
  "firstname" : "Max",
  "lastname" : "Mustermann",
  ...
}

We start the refactoring by adding a new attribute to the DTO:

public class PolicyDto extends RepresentationModel {
    private String policyId;
--- private String customerId;
+++ private CustomerDto customer;
    private Date creationDate;
...

Depending on the backward compatibility requirements, the customerId can be kept in the DTO. Otherwise, it can be removed, as shown above. To fetch the data for the customer, the endpoint implementation uses the customerService, a Java class residing in the business logic layer of the sample application, to look up the customer and add it to the response DTO:

@ApiOperation(value = "Get a single policy.")
@GetMapping(value = "/{policyId}")
public ResponseEntity<PolicyDto> getPolicy(
    @ApiParam(value = "the policy's unique id")
    @PathVariable PolicyId policyId) {
    logger.debug("Fetching policy with id '{}'", 
      policyId.getId());
    Optional<PolicyAggregateRoot> optPolicy = 
      policyRepository.findById(policyId);
    PolicyAggregateRoot policy = optPolicy.get();
    PolicyDto response = PolicyDto.fromDomainObject(policy);
+++ CustomerDto customer = customerService.
+++   getCustomer(policy.getCustomerId());
+++ response.setCustomer(customer);
    return ResponseEntity.ok(response);
}

Note that we use the Java Web framework Spring Boot in this example. The annotations @GetMapping and @PathVariable are used to instruct Spring that this is an HTTP endpoint that expects a PolicyId in its path. The annotations prefixed with @Api are used to generate an OpenAPI Specification file from the source code and serve as further documentation.

The customer data is now part of the response message. The client can access the data without issuing a second request:

curl http://localhost/policies/fvo5pkqerr

{
  "policyId" : "fvo5pkqerr",
  "customer" : {
    "customerId" : "rgpp0wkpec",
    "firstname" : "Max",
    "lastname" : "Mustermann",
    ...
  },
  "creationDate" : "2021-07-07T13:40:52.201+00:00",
  "policyPeriod" : {
    "startDate" : "2018-02-04T23:00:00.000+00:00",
    "endDate" : "2018-02-09T23:00:00.000+00:00"
  },
  ...
}

Hints and Pitfalls to Avoid

The referenced information holder should be part of the same API endpoint. Otherwise, performing the refactoring might introduce undesired dependencies between backend services.

An API endpoint may now interact with more backends or databases than before. This additional dependency might not be desired from a separation of concerns standpoint, for instance, when considering role- or attribute-based authorization [Kapferer and Jost 2017].

See the Embedded Entity and Linked Information Holder patterns for a deeper discussion of the benefits and liabilities of each pattern.

Related Content

Extract Information Holder inverses this refactoring.

The Wish List and Wish Template patterns both offer alternative solutions to the problem of how an API client can inform the API provider at runtime about the data it is interested in, known as response shaping.

The Backends For Frontends pattern by Sam Newman is another approach to tailoring a backend to the specific needs of a client.

As also mentioned in the inverse refactoring Extract Information Holder, Context Mapper [Kapferer and Zimmermann 2020] implements these refactorings on domain-level (DDD). While Extract Information Holder corresponds to Split Aggregate by Entity, Inline Information Holder would be established with Merge Aggregates in Context Mapper and DDD.

References

Fowler, Martin. 2018. Refactoring: Improving the Design of Existing Code. 2nd ed. Addison-Wesley Signature Series (Fowler). Boston, MA: Addison-Wesley.

Kapferer, Stefan, and Samuel Jost. 2017. “[Attributbasierte Autorisierung in einer Branchenlösung für das Versicherungswesen - Analyse, Konzept und prototypische Umsetzung]{.nocase}.” Bachelor Thesis, https://eprints.ost.ch/id/eprint/602/: University of Applied Sciences of Eastern Switzerland (HSR FHO).

Kapferer, Stefan, and Olaf Zimmermann. 2020. “Domain-Driven Service Design.” In Service-Oriented Computing, edited by Schahram Dustdar, 189–208. Springer International Publishing. https://doi.org/10.1007/978-3-030-64846-6_11.

Zimmermann, Olaf, Mirko Stocker, Daniel Lübke, Uwe Zdun, and Cesare Pautasso. 2022. Patterns for API Design: Simplifying Integration with Loosely Coupled Message Exchanges. Addison-Wesley Signature Series (Vernon). Addison-Wesley Professional.

1. The Rule of Three states that when you copy and paste code for the third time, you should extract it into a method [Fowler 2018]. Not to be confused with the Rule of Three of the Patterns community: call it a pattern if there are at least three known uses (https://wiki.c2.com/?RuleOfThree). ↩