Bundle Requests

Context and Motivation

API clients make many small requests (in terms of request message size) in quick succession, and individual responses are returned for one or more of these requests. These high-velocity request-response sequences have a negative impact on scalability and throughput.

As an API provider, I want to reduce the number of requests and responses that API clients have to send and receive so that bandwidth and network capacity are saved and unnecessary message processing is avoided. To do so, I want to allow clients to group multiple domain-level requests in a single, structured API call.

Stakeholder Concerns

#developer-experience

Bulk/batch uploads might be prepared by iterating through a data set and then calling API operations multiple times, once for each entry in the set. It might be easier for client developers to prepare a single technical request message (for instance, a JSON array) that contains several business-, domain-, or application-level requests (each represented as a JSON object). Such design frees the clients from having to manage the state (running, succeeded, failed) of all individual requests.

#performance

Transferring many small requests may stress network and communication endpoints preparing requests and consuming them (for instance, on the HTTP or the TCP/IP level).

Initial Position Sketch

The API client currently sends multiple requests to a provider. Figure 1 sketches this initial position.

Figure 1: Bundle Requests: Initial Position Sketch. A client sends three requests to the API provider. Each request comprises several Data Elements and can have its own structure. Note that the responses are not shown for the sake of clarity.

The targeted design elements are one or more API operations and the request and response messages of these operations.

Design Smells

Atomicity and consistency management issues

Some clients might want to make sure that either all or none of the requests and corresponding responses complete successfully. This can be much harder if they arrive one by one; most remote APIs do not offer global system transactions (for instance, distributed two-phase commits) – for good reasons: operations overhead, testing effort, and technical risk over time. “Software Architecture: The Hard Parts” covers the consistency design concern, its relationship with other concerns (communication, coordination) as well as related tradeoffs [Ford et al. 2021].

High latency/poor response time

Responses take a long time to be returned to clients because many small requests cause high workload for the communications infrastructure and protocol endpoint(s) on the provider side.

Instructions

The refactoring can be applied to an existing operation. It causes a breaking change:

1. Apply the Request Bundle Pattern to an existing operation: “Define a Request Bundle as a data container that assembles multiple independent requests in a single request message. Add metadata such as identifiers of individual requests (bundle elements) and a bundle element counter.” (source: “Patterns for API Design” [Zimmermann et al. 2022])
2. Adjust the API Description, tests, and all other supporting/supplemental artifacts accordingly.

Alternatively, it can be offered as/in a new operation in the same endpoint:

1. Copy an existing operation.
2. Apply the Request Bundle Pattern to the new operation, as already described above.
3. Document and test the new operation.

Optionally, the bundles can be annotated with additional Metadata Elements. A bundle-level Error Report may be included in the response.

Special emphasis should be put on the monitoring to assess the positive or negative effect of the refactoring. Average response times and message sizes should be logged, and the performance of the initial API design be compared with that of the refactored one.

Target Solution Sketch (Evolution Outline)

After the refactoring, both the request message and, optionally, the response message of the refactored operation contain collections of individual requests. See Figure 2 for this solution sketch.

Figure 2: Bundle Requests: Target Solution Sketch. The client bundles the three individual requests into a Request Bundle message. The provider uses a Splitter component to disassemble the bundle and forward the requests (1, 2, 3) to their final destination. Note that only the request bundling is shown in the figure; the corresponding responses are left out. The repositories supporting the resources are not shown either, but continue to exist.

A variant of the Request Bundle pattern is shown in the HTTP resource Customer Information Holder in Lakeside Mutual, a sample microservice application we built to demonstrate API design and patterns.

Example(s)

In the following example, a Request Bundle is added to a Customer Relationship Management endpoint (notation: MDSL):

API description RequestBundleExample

data type YesOrNo D<bool>
data type QueryExpression D<string> // detailed query syntax not shown 
data type CustomerDTO {"name":D<string>, "contacts":D<string>*}

endpoint type CustomerRelationshipManagement 
 serves as MASTER_DATA_HOLDER
exposes 
  operation createCustomer 
   with responsibility STATE_CREATION_OPERATION
    expecting payload CustomerDTO 
    delivering payload "id":ID<int>
    
  operation updateCustomer
    expecting payload CustomerDTO 
    delivering payload "ok":YesOrNo

  operation lookupCustomers 
   with responsibility RETRIEVAL_OPERATION
    expecting payload "criteria":QueryExpression 
    delivering payload "results":CustomerDTO*

Note that the response of the new operation sendBulkRequest contains a tree of objects (just like the request):

API description RequestBundleExample

data type YesOrNo D<bool>
data type QueryExpression D<string> // detailed query syntax not shown 
data type CustomerDTO {"name":D<string>, "contacts":D<string>*}

data type CustomerOperation {
     "createCustomer":CustomerDTO
    |"updateCustomer":CustomerDTO
    |"lookupCustomers":QueryExpression 
}

data type CustomerOperationResponse {
     "id":ID<int>
    |"ok":YesOrNo
    |"results":CustomerDTO*
}

endpoint type CustomerRelationshipManagement 
 serves as MASTER_DATA_HOLDER
exposes 
  operation sendBulkRequest 
    expecting payload <<Request_Bundle>> 
      "operations":CustomerOperation+ 
    delivering payload 
      "results":CustomerOperationResponse+

This MDSL contract is also available here and its OpenAPI Specification pendant here.

The MDSL Tools prototype contains an implementation of this refactoring.

Hints and Pitfalls to Avoid

Consider the following:

• Apply the Splitter pattern from “Enterprise Integration Patterns” to unbundle the incoming request message.
• Leverage parallel programming concepts in the provider implementation. An advanced option to do so is the Map-Reduce pattern.
• Decide whether the responses should also be bundled (see solution and discussion of the consequences of the Request Bundle pattern for detailed design information).
• Measure and compare: Do the performance gains justify the extra effort and complexity? Be willing to undo if the new design turns out to be difficult to teach and maintain and the performance gains do not outweigh these negative consequences.
• When implemented by a new endpoint, make sure that this endpoint is covered by the same security measures as the original endpoint.
• Do not prescribe a minimum bundle size, but allow clients to decide how many requests they want to bundle together. Otherwise, they might never be able to send their requests. Client developers can prepare bundles incrementally, for example, by accumulating requests while processing other tasks. Common scenarios include processing files in batches (such as monthly sales figures in Backend Integration) or collecting user input selections (like checked boxes in a UI form) before sending them as a bundle.

Carefully think about error handling when applying the Request Bundle pattern:

• If one of the requests in the bundle fails, how should the provider respond? Should an error be returned for the entire bundle or just for the failed request? Consider using an Error Report to communicate errors in a structured way.
• What happens if one of the bundled requests fails and the client simply retries the entire bundle? This can lead to unintended consequences, such as duplicate processing of requests that were already successful. An idempotent design of the request processing can help mitigate this risk. See the Idempotent Receiver pattern by Hohpe and Woolf [2003] for more information.

A deeper discussion of the pros and cons of the Request Bundle pattern is available on the “Patterns for API Design” website and in [Zimmermann et al. 2022].

Related Content

A blog post on API design by James Higginbotham titled “API Design Guidance: Bulk and Batch Import” clarifies the difference between batch and bulk processing and gives related advice.

Merge Operations can be used in preparation of Bundle Requests if the requests that should be bundled currently target different operations. Merging these into a single operation enables the introduction of a Request Bundle.

Introduce Pagination is an alternative to improve performance, focussing on responses rather than requests.

As an alternative solution to expose batch processing in an API, you may want to consider using a job scheduler rather than bundled API operations.

Note that we do not feature an “Unbundle Requests” refactoring in our catalog.

References

Ford, N., M. Richards, P. Sadalage, and Z. Dehghani. 2021. Software Architecture: The Hard Parts. O’Reilly Media.

Hohpe, Gregor, and Bobby Woolf. 2003. Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions. Addison-Wesley.

Zimmermann, Olaf, Mirko Stocker, Daniel Lübke, Uwe Zdun, and Cesare Pautasso. 2022. Patterns for API Design: Simplifying Integration with Loosely Coupled Message Exchanges. Addison-Wesley Signature Series (Vernon). Addison-Wesley Professional.