Bundle Requests

also known as: Introduce Request Bundle

Context and Motivation

The description of Request Bundle sets the pattern context as follows: API “clients make many small requests (in terms of request message size), and individual responses are returned for one or more of these requests. These sequences have a negative impact on scalability and throughput.” (source: Microservice API Patterns (MAP) website)

As an API provider, I want to allow my clients to reduce the number of requests and responses to save bandwidth and network capacity and to avoid unnecessary message processing.

Stakeholder Concerns

#performance

Transferring many small requests may stress network and communication endpoints preparing requests and consuming them (for instance, on HTTP or TCP/IP level).

#developer-experience

Bulk/batch uploads might be prepared by iterating through a data set and then sending API operation calls for each entry in the set. It might be easier to prepare a single technical request message (for instance, a JSON array) that contains several business-, domain-, or application-level requests instead of having to manage the state (running, succeeded, failed) of all individual requests on the client side.

Initial Position Sketch

The API provider currently sends multiple requests (responses not shown in figure):

Bundle Requests: Initial Position Sketch

Targets:

• An API operation and its request and response messages.

Design Smells

High latency/poor response time

Responses take a long time to be returned to clients because many small requests cause high workload for the communications infrastructure and protocol endpoint(s) on the provider side.

Atomicity and consistency management issues

Some clients might want to make sure that either all or none of the requests and corresponding responses complete successfully. This can be much harder if they arrive one by one; most remote APIs do not offer global transactions (for instance, distributed two-phase commits) nowadays — for good reasons such as operation overhead, testing effort, and technical risk over time.

Instructions

The refactoring can be applied to an existing operation (as a breaking change):

1. Apply the Request Bundle Pattern to an existing operation: “Define a Request Bundle as a data container that assembles multiple individual requests in a single request message. Add metadata such as number and identifiers of individual requests.” (source: Microservice API Patterns (MAP))
2. Adjust the API Description, tests, and all other supporting/supplemental artifacts accordingly.

Alternatively, it can be offered as/in a new operation in the same endpoint:

1. Copy an existing operation.
2. Apply the Request Bundle Pattern to the new operation, as already described above.
3. Document and test the new operation.

Special emphasis should be put on the monitoring to assess the positive or negative effect of the refactoring. Average response times and message sizes should logged, and the performance of the initial API design be compared with that of the refactored one.

Target Solution Sketch (Evolution Outline)

After the refactoring, both the request message and, optionally, the response message of the refactored operation contain collections of individual requests (note that only the request bundling is shown in the figure; the corresponding responses are left out):

Bundle Requests: Target Solution Sketch

Optionally, the bundles can be annotated with additional Metadata Elements. A bundle-level Error Report may be included in the response.

A variant of the Request Bundle pattern is shown in the HTTP resource Customer Information Holder in Lakeside Mutual.

Example(s)

In this example, we will add a Request Bundle to a Customer Relationship Management endpoint:

API description RequestBundleExample

data type YesOrNo D<bool>
data type QueryExpression D<string> // detailed query syntax not shown for brevity
data type CustomerDTO {"name":D<string>, "contacts":D<string>*}

endpoint type CustomerRelationshipManagement serves as MASTER_DATA_HOLDER
exposes 
  operation createCustomer with responsibility STATE_CREATION_OPERATION
    expecting payload CustomerDTO 
    delivering payload "id":ID<int>
    
  operation updateCustomer
    expecting payload CustomerDTO 
    delivering payload "ok":YesOrNo

  operation lookupCustomers with responsibility RETRIEVAL_OPERATION
    expecting payload "criteria":QueryExpression 
    delivering payload "results":CustomerDTO*

Note that the response of the new operation sendBulkRequest contains a tree of objects (just like the request):

API description RequestBundleExample

data type YesOrNo D<bool>
data type QueryExpression D<string> // detailed query syntax not shown for brevity 
data type CustomerDTO {"name":D<string>, "contacts":D<string>*}

data type CustomerOperation {
     "createCustomer":CustomerDTO
    |"updateCustomer":CustomerDTO
    |"lookupCustomers":QueryExpression 
}

data type CustomerOperationResponse {
     "id":ID<int>
    |"ok":YesOrNo
    |"results":CustomerDTO*
}

endpoint type CustomerRelationshipManagement serves as MASTER_DATA_HOLDER
exposes 
  operation sendBulkRequest 
    expecting payload <<Request_Bundle>> "operations":CustomerOperation+ 
    delivering payload "results":CustomerOperationResponse+

This MDSL contract is also available here and its OpenAPI Specification pendant here.

Hints and Pitfalls to Avoid

Consider the following:

• Apply the Splitter pattern to unbundle the incoming request message.
• Leverage parallel programming concepts in the provider implementation. An advanced option is the Map-Reduce pattern.
• Decide whether the responses should also be bundled (see solution and discussion of the consequences of the Request Bundle pattern).
• Measure and compare: Do the performance gains justify the extra effort and complexity?
• Be willing to undo if the new design turns out to be difficult to teach and maintain and the performance gains do not outweigh these negative consequences.
• When implemented by a new endpoint, make sure that it is covered by the same security measures as the original endpoint.

As an alternative to exposing batch processing in an API, you may want to consider using a job scheduler rather than bundled API operations.

A deeper discussion of the pros and cons of the Request Bundle pattern is available on the MAP website.

Related Content

Merge Operations can be used in preparation of Bundle Requests if the requests that should be bundled currently target different operations. Merging these into a single operation enables the introduction of a Request Bundle. Note that we do not feature an “Unbundle Requests” refactoring in our catalog at present.

Introduce Pagination is an alternative to improve performance, focussing on responses rather than requests.

The blog post on API design by James Higginbotham titled “API Design Guidance: Bulk and Batch Import” clarifies the difference between batch and bulk processing and gives related advice.

The (still emerging) MDSL tool prototypes contain an implementation of this refactoring.