Merge Operations

also known as: Colocate Responsibilities, Consolidate Operations

Context and Motivation

An API endpoint contains two operations with similar, possibly overlapping responsibilities. Typically, this was not the intention of the original API designers, but the result of an API evolution process. For instance, the API provider might have added a new operation to the endpoint instead of adjusting an existing one.

As an API designer, I want to remove an API operation from an endpoint and let another operation in that endpoint take over its responsibilities so that there are fewer operations to maintain and evolve and the inner cohesion of the endpoint improves.

Stakeholder Concerns (including Quality Attributes and Design Forces)

#understandability, #explainability, #learnability

There are many reasons to change an API (not just refactorings) [Stocker and Zimmermann 2021]. APIs should be changed as much as required and as little as possible, and ripple effects be avoided. One of the first steps in related maintenance tasks is to determine which parts of the system should be changed. An API whose endpoints have clearly identified roles helps developers to quickly understand the API.

#cohesion

Cohesive design elements (here: API operations in an endpoint) belong together naturally because they share certain properties. In an API design context, the security rules that apply are an example of such a property. Cohesion is desirable because it makes the system easier to understand and maintain. ISO/IEC/IEEE 24765 [Standardization et al., n.d.] defines cohesion as “manner and degree to which the tasks performed by a single software module are related to one another.”

#coupling

In our context, coupling is a measure of how closely connected and dependent on each other endpoints and their operations are. The coupling may concern the data exchanged and/or the operation call sequencing. See Wikipedia entry for Coupling and Loose Coupling pattern for general explanations.

Initial Position Sketch

The refactoring is applicable if the current API exposes an endpoint with at least two operations, as shown by the following MDSL snippet:

endpoint type Endpoint1BeforeMerge
exposes 
  operation operation1 
    expecting payload "RequestMessage1" 
    delivering payload "ResponseMessage1"
    
   operation operation2 
    expecting payload "RequestMessage2" 
    delivering payload "ResponseMessage2"

As the snippet shows, this refactoring targets a single endpoint and two of its operations. Figure 1 visualizes this initial position. The API offers two operations, which clients may or may not call in any particular order.

Figure 1: Merge Operations: Initial Position Sketch. The API provider offers two distinct operations (Op1, Op2). The client can invoke one (Request 1, Response 2) or the other (Request 3, Response 4). Some metadata and other data elements are exchanged.

Design Smells

Responsibility spread

Endpoint roles and/or operation responsibilities are rather diffuse; the Single Responsibility Principle is violated. For instance, API clients serving a particular stakeholder have to call multiple operations to satisfy their information needs. Another example would be that a choreographed or orchestrated business process implementation has to consult too many distributed operations to fulfill its job.

High coupling

Two or more operations perform narrowly focused, rather low-level activities. Clients have to understand and combine all of these activities to achieve higher goals, leading to a degraded developer experience and coordination needs. This causes these operations to be coupled with each other implicitly.

REST principle(s) violated

The “Uniform interface” is an important design constraint imposed by the REST style that many HTTP APIs employ. REST mandates using the standard HTTP verbs (POST, GET, PUT, PATCH, DELETE, etc.), which are associated with additional constraints. For instance, GET and PUT requests must be idempotent to be cachable [Allamaraju 2010]. Sometimes, mismatches between the API semantics and the REST constraints can be observed; sometimes, the REST constraints limit extensibility (for instance, when a resource identified by a single URI runs out of verbs) [Serbout et al. 2022].

Instructions (Steps)

This refactoring requires careful planning and execution:

1. Merge the data structures used in the two request messages if they differ. A straightforward approach is to combine and wrap the original message contents in a new DTO (see Introduce Data Transfer Object) in the consolidated message.
2. (Optional) Add a boolean flag to the request message to distinguish the merged operations, for instance, to dispatch the request to the proper API implementation logic. This step is optional because it might be possible to inspect the merged request message to select the implementation logic (see the example later).
3. Merge the response message data structures as well. A DTO can be used to do so.
4. Consolidate the implementation code, deciding how to route incoming requests and how to prepare the consolidated response.
5. Add at least two tests if the boolean flag introduced in Step 2 is present: one sets it to false, and the other sets it to true. Test the new API and compare old and new behavior.
6. Update supporting artifacts such as API Description and usage examples. Show how to use the boolean flag (if introduced) and explain how to migrate from the old to the new API.
7. Inform the API user community about the change and its rationale. Make the news item self-contained or provide direct links to the updated API Description and usage examples; avoid general statements such as “We have changed our API in an incompatible way. Please consult the documentation to learn how.”

The changes introduced in Steps 1 to 4 are not backward-compatible per se. Steps 5 to 7 apply to most refactorings in our catalog; we refer to them as TELL (Test, Explain, Let Know, and Learn).

Target Solution Sketch (Evolution Outline)

The API contract from the Initial Position Sketch above still contains one endpoint. But only one operation is present now (note that { , } is the MDSL notation for Parameter Trees [Zimmermann et al. 2017], an abstraction of JSON objects):

data type ConsolidatedRequestMessage {
    "RequestMessage1", "RequestMessage2"
}
data type ConsolidatedResponseMessage {
    "ResponseMessage1",  "ResponseMessage2"
}

endpoint type Endpoint1AfterMerge
exposes 
  operation operation1and2Merged 
    expecting payload 
        "RequestMessage12":ConsolidatedRequestMessage
    delivering payload 
        "ResponseMessage12":ConsolidatedResponseMessage

Figure 2 visualizes the resulting API design that uses a Content-Based Router to select the operation to execute.

Figure 2: Merge Operations: Target Solution Sketch. The client sends a request (1) that includes one or more data elements. This figure shows two data elements, but they might not all be mandatory. The provider uses a Content-Based Router to execute operations (Op1, Op2) depending on the content of the message and returns a response, for example, some metadata (2).

Example(s)

In the following example of the user administration endpoint of an API implemented in Spring Boot, there are two operations to change the e-mail address and username, respectively. Both use the same endpoint /users/{id}, but the developer decided to use different HTTP verbs (POST and PATCH) to implement the two operations:

@PostMapping("/users/{id}")
public ResponseEntity<User> changeEmail(
    @RequestBody ChangeEmailDTO changeEmailDTO)  {
    log.debug("REST request to change email : {}", 
        changeEmailDTO);
    ...
}

@PatchMapping("/users/{id}")
public ResponseEntity<User> changeUsername(
    @RequestBody ChangeUsernameDTO changeUsernameDTO) {
    log.debug("REST request to change username : {}", 
        changeUsernameDTO);
    ...
}

Keep in mind that the API client does not see these method names but the POST and PATCH verbs only. Using different HTTP verbs simply to distinguish between two operations violates REST principles. POST is meant for creating resources, not updating them, as done in changeEmail. These endpoints can then be used as follows:

curl -X POST  api/users/123 -d '{ … }'
curl -X PATCH api/users/123 -d '{ … }'

The HTTP verb used is the only difference from the perspective of the API client; the fixed amount of available HTTP verbs limits future extensibility given (maybe passwords should also be changeable?). Hence, it is decided to merge the two operations and create a composite request message DTO:

class ChangeUserDetailsDTO {
    ChangeEmailDTO changeEmail;
    ChangeUsernameDTO changeUsername;
}

@PatchMapping("/users/{id}")
public ResponseEntity<User> changeUserDetails(
    @RequestBody ChangeUserDetailsDTO changeUserDetailsDTO) {
    if (changeUserDetailsDTO.changeEmail != null) {
        log.debug("REST request to change email : {}", 
            changeUserDetailsDTO.changeEmail);
        ...
    }
    if (changeUserDetailsDTO.changeUsername != null) {
        log.debug("REST request to change username : {}", 
            changeUserDetailsDTO.changeUsername);
        ...
    }
    ...
}

Further operations changing other properties of the user can now be implemented by extending the DTO without introducing new operations or even endpoints. The DTO content determines the nature of the change; no boolean parameter was needed in this example. Clients can now also initiate several changes in a single request.

Hints and Pitfalls to Avoid

Merging operations is more challenging than merging endpoints (which usually merely group operations under a unique address such as a parent URI):

• The operations to be merged must appear in the same endpoint. Apply Move Operation first if needed.
• Do not break HTTP verb semantics when merging (in HTTP resource APIs). For instance, idempotence might get lost if a replacing PUT and an updating PATCH are merged.
• When merging the request and response messages, decide where and how the merged messages embed the original message elements. Some data exchange formats have first-class concepts for choices. If this is not the case, the optionality of list items combined with feature flags/toggles can be used.
• The complexity of the implementation logic and the tests increases when merging operations: The implementation logic must distinguish between the merged operations. The tests must cover all possible combinations of the merged operations.

Implementing this refactoring in a backward-compatible way is not trivial because of the changes imposed on request and response messages. One tactic could be to provide a new operation for the merged functionality and keep the original ones in place. The original ones can then forward incoming requests to the new operation, wrapping and un-wrapping request and response messages.

See the Merge Endpoints refactoring for Confidentiality, Integrity, and Availability (CIA) considerations; inconsistent or inappropriate CIA settings (authentication, authorization) are less likely to result from this refactoring (depending on how the API endpoint and operations have been identified) but are still worth considering.

Related Content

This refactoring reverts Split Operation. If the two operations do not reside in the same endpoint, an upfront Move Operation refactoring can prepare its application.

The Service Cutter tool and method suggest sixteen coupling criteria [Gysel et al. 2016]. These criteria primarily apply when merging endpoints but are also worth considering when merging operations.

References

Allamaraju, Subbu. 2010. RESTful Web Services Cookbook. O’Reilly.

Gysel, Michael, Lukas Kölbener, Wolfgang Giersche, and Olaf Zimmermann. 2016. “Service Cutter: A Systematic Approach to Service Decomposition.” In Service-Oriented and Cloud Computing - 5th IFIP WG 2.14 European Conference, ESOCC 2016, Vienna, Austria, September 5-7, 2016, Proceedings, edited by Marco Aiello, Einar Broch Johnsen, Schahram Dustdar, and Ilche Georgievski, 9846:185–200. Lecture Notes in Computer Science. Springer. https://link.springer.com/chapter/10.1007/978-3-319-44482-6_12.

Serbout, Souhaila, Cesare Pautasso, Uwe Zdun, and Olaf Zimmermann. 2022. “From OpenAPI Fragments to API Pattern Primitives and Design Smells.” In 26th European Conference on Pattern Languages of Programs. EuroPLoP’21. New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3489449.3489998.

Standardization, International Organization for, International Electrotechnical Commission, Institute of Electrical, and Electronics Engineers. n.d. ISO/IEC/IEEE 24765: 2017(e): ISO/IEC/IEEE International Standard - Systems and Software Engineering–Vocabulary. IEEE Std. IEEE.

Stocker, Mirko, and Olaf Zimmermann. 2021. “From Code Refactoring to API Refactoring: Agile Service Design and Evolution.” In Service-Oriented Computing, edited by Johanna Barzen, 174–93. Cham: Springer International Publishing.

Zimmermann, Olaf, Mirko Stocker, Daniel Lübke, and Uwe Zdun. 2017. “Interface Representation Patterns - Crafting and Consuming Message-Based Remote APIs.” In 22nd European Conference on Pattern Languages of Programs (EuroPLoP 2017), 1–36. https://doi.org/10.1145/3147704.3147734.