Merge Operations

Updated:

 Note: This is a preview release and subject to change. Feedback welcome! Contact Information and Background (PDF)

also known as: Colocate Responsibilities, Consolidate Operations

Context and Motivation

An API endpoint contains two operations that have rather similar, possibly overlapping responsibilities.

As an API designer, I want to remove an API operation from an endpoint and let another operation in that endpoint take over its responsibilities so that there are fewer operations to maintain and evolve and the inner cohesion of the endpoint improves.

Stakeholder Concerns (including Quality Attributes and Design Forces)

#understandability (a.k.a. #explainability, #learnability)
Fewer operations might be easier to understand and maintain — if they still each have one and only one dedicated responsibility. Move Operation has more explanations.
#cohesion
Cohesive design elements (here: API operations in an endpoint) belong together naturally because they share certain properties. In an API design context, the security rules that apply are an example of such a property. Refer to the Wikipedia entry and the Systems Engineering Body of Knowledge (SEBoK) for general introductions of the term.
#coupling
In our context here, coupling is a measure of how closely connected and dependent on each other endpoints and their operations are. The coupling may concern the data exchanged and/or the operation call sequencing. See Wikipedia entry and summary of Loose Coupling in pattern form for general explanations.

Initial Position Sketch

The refactoring is applicable if the current API looks as follows:

endpoint type Endpoint1BeforeMerge
exposes 
  operation operation1 
    expecting payload "RequestMessage1" 
    delivering payload "ResponseMessage1"
    
   operation operation2 
    expecting payload "RequestMessage2" 
    delivering payload "ResponseMessage2"

As the MDSL snippet shows, this refactoring targets a single endpoint and two of its operations. The following figure visualizes this initial position:

Note that there is no order/sequence of calls implied. The API offers two operations, which clients may or may not call in any particular order.

Smells / Drivers

Single responsibility spread
The Single Responsibility Principle is violated. For instance, API clients serving a particular stakeholder have to work with multiple operations to satisfy their information needs. Endpoint role and/or operation responsibilities are rather diffuse.
High coupling
Two or more operations perform narrowly focussed, rather low-level activities. Clients have to understand and combine all of these activities to achieve higher goals, leading to a degraded developer experience and coordination needs. This causes these operations to be coupled with each other implicitly.
REST principle(s) violated
“Uniform interface” is an important design constraint imposed by the REST style that many HTTP APIs employ. REST mandates the use of the standard HTTP verbs (POST, GET, PUT, PATCH, and DELETE), which are associated with additional constraints/properties. For instance, GET and PUT requests must be idempotent so that they can be cached [Allamaraju 2010]. Sometimes, mismatches between the API semantics and the REST constraints can be observed; sometimes, the REST constraints limit extensibility (for instance, when a resource, identified by a URI, runs out of verbs).

Instructions (Steps)

This refactoring requires careful planning and execution:

  1. Merge the data structures used in the two request messages. A straightforward approach is to wrap the original data in a new DTO (see Introduce Data Transfer Object) in the consolidated message.
  2. Add a boolean flag to the request message to distinguish the merged operations if needed, for instance to dispatch the request to the right API implementation logic.
  3. Merge the response message data structures as well. A DTO can be used here again.
  4. Consolidate the implementation code, deciding how to route incoming requests and how to prepare the consolidated response.
  5. Add two tests that work with the boolean flag introduced in Step 2 (if any). Test the new API and compare old and new behavior.
  6. Update supporting artifacts (API description, examples, etc.).
  7. Inform the API user community about the change.

The changes introduced in Steps 1 to 4 are not backwards-compatible per se. Steps 5 to 7 apply to most refacorings in our catalog; we refer to them as TELL (Test, Explain, Let Know and Learn).

Target Solution Sketch (Evolution Outline)

The API contract from the Initial Position Sketch above still contains one endpoint. But only one operation is present now:

data type ConsolidatedRequestMessage {"RequestMessage1", "RequestMessage2"}
data type ConsolidatedResponseMessage {"ResponseMessage1",  "ResponseMessage2"}

endpoint type Endpoint1AfterMerge
exposes 
  operation operation1and2Merged 
    expecting payload "RequestMessage12":ConsolidatedRequestMessage
    delivering payload "ResponseMessage12":ConsolidatedResponseMessage

The following figure visualizes the resulting API design:

Example(s)

In the following example of the user administration endpoint of an API implemented in Spring Boot, there are two operations to change the e-mail address and username, respectively. Both use the same endpoint /users/{id}, but the developer decided to use different HTTP verbs (POST and PATCH) to implement the two operations:

@PostMapping("/users/{id}")
public ResponseEntity<User> changeEmail(@Valid @RequestBody ChangeEmailDTO changeEmailDTO)  {
    log.debug("REST request to change email : {}", changeEmailDTO);
    ...
}

@PatchMapping("/users/{id}")
public ResponseEntity<User> changeUsername(@Valid @RequestBody ChangeUsernameDTO changeUsernameDTO) {
    log.debug("REST request to change username : {}", changeUsernameDTO);
    ...
}

Keep in mind that the API client does not see these method names, but the POST and PATCH verbs only:

curl -X POST  api/users/123 -d '{ … }'
curl -X PATCH api/users/123 -d '{ … }'

The HTTP verb usage is the only difference from the perspective of the API client; the fixed amount of available HTTP verbs limits future extensibility given (maybe passwords should also be changeable?). Hence, it is decided to merge the two operations and to create a composite request message DTO:

class ChangeUserDetailsDTO {
    ChangeEmailDTO changeEmail;
    ChangeUsernameDTO changeUsername;
}

@PatchMapping("/users/{id}")
public ResponseEntity<User> changeUserDetails(@Valid @RequestBody ChangeUserDetailsDTO changeUserDetailsDTO) {
    log.debug("REST request to change user details : {}", changeUserDetailsDTO);
    ...
}

Further operations to change other aspects of the user can now be implemented by extending the DTO, without having to introduce new operations or even endpoints. The nature of the change is determined by the DTO content.

Hints and Pitfalls to Avoid

Merging operations is more challenging than merging endpoints (which merely group operations under a unique address such as a URI):

  • The operations to be merged must appear in the same endpoint. Apply Move Operation first if needed.
  • Do not break HTTP verb semantics when merging (in HTTP resource APIs). For instance, idempotence might get lost if a replacing PUT and an updating PATCH are merged.
  • When merging the request and response messages, decide whether and how to list them. Some data exchange formats have first-class concepts for choices. If this is not the case, optionality of list items combined with feature flags/toggles can be used.

Implementing this refactoring in a backwards compatible way is not trivial because of the changes imposed on request and response messages. One tactic could be to provide a new operation for the merged functionality, but keeping the original ones in place. The original ones can then forward incoming requests to the new operation, wrapping and un-wrapping request and response messages.

See Merge Endpoints for security considerations; they are less likely to occur here (depending on how the API endpoint and operations have been identified) but still relevant.

This refactoring reverts Split Operation. If the two operations do not reside in the same endpoint, an upfront Move Operation refactoring can prepare its application.

The Bundle Requests refactoring “merges” multiple request messages sent to the same operation into a single request. Merge Operations can be used in preparation of Bundle Requests if the requests that should be bundled currently target different operations.

The Service Cutter tool and method suggest sixteen coupling criteria [Gysel et al. 2016]. These criteria primarily apply when merging endpoints, but are also worth considering when merging operations.

The operation ExecuteTest in the Abrantix Extension API (v2) can be seen as the result of merging three operations (that in this case remain available in the API).

References

Allamaraju, Subbu. 2010. RESTful Web Services Cookbook. O’Reilly.

Gysel, Michael, Lukas Kölbener, Wolfgang Giersche, and Olaf Zimmermann. 2016. “Service Cutter: A Systematic Approach to Service Decomposition.” In Service-Oriented and Cloud Computing - 5th IFIP WG 2.14 European Conference, ESOCC 2016, Vienna, Austria, September 5-7, 2016, Proceedings, edited by Marco Aiello, Einar Broch Johnsen, Schahram Dustdar, and Ilche Georgievski, 9846:185–200. Lecture Notes in Computer Science. Springer. https://link.springer.com/chapter/10.1007/978-3-319-44482-6_12.