Spring Reactive — Bird’s eye-view

Everything you need to know to plan migration to Spring Reactive

Are you a product manager, a technical program manager, or a tech lead of a team of engineers who want to make your application reactive? Or, are you a seasoned programmer looking to get into Spring Reactive?

Read on to find out why Spring Reactive is getting all the hype and how you can conquer this reactive wave with ease. Regardless of your experience in Java or Spring or backend programming, Spring Reactive is a must-have tool in your arsenal for future growth (as of 2021). The real challenge is orchestrating the migration to Spring Reactive and timing it right. This blog will give you a high-level understanding and walk you through all the elements to get it right.

Why reactive? — With the dawn of the Internet of Things, the number of wearables, devices, systems, and interfaces has increased. With the increase in IoT technologies, the number of requests to an application has been skyrocketted in recent days. A mobile app or a microservice of 2021, has to not just respond to web requests, but to mobile, watch, tablets, voice assistants, and the list goes on.

The increased need to serve a wide number of user requests at lightning speed is as essential as optimizing for computational complexity. If you think about optimizing programming practices and computing resources for computational complexity as vertical expansion, then scaling for a higher number of users, on-demand, at exponential speed, without compromising performance and customer experience is horizontal expansion.

Reactive programming is the best tool that can enable this horizontal expansion in your application — the ability to serve large number of users, with minimal resources, minimum time, and highest velocity(atleast with the current technologies out there. I will not be surprised if there is something better in the future).

Before we deep dive into Reactive, let’s see why the existing programming frameworks and architectures are not geared to do this.

What are the problems Traditional Non-Reactive Micro Service Architecture — I love microservices. It made enterprise digital transformations feel like a cakewalk — a wide deep cake, but still easy enough from the monolithic predecessor. For a microservice avid fan like me, you know why this is a rather controversial statement to say, especially when multiple global organizations from myriad industries have experienced tremendous speed to market, adaptiveness to business uncertainties, and internal engineering excellence by adopting a microservices culture.

However, there is one problem often overlooked and the cost to overlook this is increasing as IoT devices fill the market — Most large enterprise microservices tend to be built with Spring MVC which is inherently a blocking synchronous paradigm.

So why is this synchronous blocking programming paradigm a problem? — Two reasons:

1. Slow customer request fulfillment
2. Wasted computing resources

Let me explain here:

We all know how chatty microservices are. When you make a digital transaction in real life, there are multiple systems (composed of many microservices) that orchestrate behind the scenes. One service talks to at least 2–3 other services to fulfill its responsibility.

To elaborate the chattiness further and put it into a real-world context — let’s say you make a payment on a banking app through your apple watch. At the backend, let’s say the transfer of money digitally is orchestrated by about 4–5 systems ( in real life this tends to be ~14 microservices). So these 4–5 microservices have to talk to each other to fulfill the function in real life.

In this scenario, let’s say there is a microservice A that responds to user requests to wire money. Let’s assume Microservice A’s job is just to authenticates the user, validates the account. Microservice A calls another Microservice B to initiate a transfer of money process that is to run nightly. All that Microservice B is to batch that process onto an existing queue of transfer requests which is orchestrated by another service C. Microservice A has to wait until Microservice B responds to report successfully transaction to the user. If B is slow or down, the delay propagates to the calling service A. I’d like to call this chattiness-delay — ironic right, in the current age of viral videos and gossip wildfires.

Extrapolating that to real-world digital fulfillment scenarios, we may end up responding to a customer request slower than expected and the reason may completely not be in the control of microservice A. Until B responds, A sits idle with wasted resources (primarily threads, in addition to others).

Wasted Resources increases Operational Cost

This inherent way of operating — waiting on other resources — enters the application in a blocking state. This blocking state programming paradigm introduces delays that compound as the number of user requests increases. In such a scenario, most often, the threads in your Java Virtual Machine, which are the ones waiting, have run out but the CPU is idle because the threads are just waiting for responses from external services and processes.

In short, your computing resources (threads, in this case) are all just waiting and not performing tasks until the external services are complete.

The waiter analogy

Think of it like this — A waiter at your restaurant is waiting only on you until you finish eating instead of waiting on other customers simultaneously. Wouldn’t that be hilarious? Yet that happens in the Java-Spring-Microservices world. A thread, the Java counterpart for a waiter, waits indefinitely until the called service responds.

Reactive programming is a lot similar to how waiters operate in the real world. With that, let’s pivot to the three shifts required to guide your migration journey.

Foundational Element 1: Mental Shift

If you come from a Spring MVC world, you are used to the imperative programming style. In an imperative approach, the developer tells the computer, exactly how to execute something. This is called algorithmic programming. The underlying processes and threads that support such a style are built in one thread/worker per customer request. This programming style requires constant monitoring for incoming instructions to be able to carry out an instruction. Hence a thread is exclusively assigned to a customer request until it is fulfilled.

A JVM can have a maximum of 256 threads. So if your application is running on 2 EC2 with 8 containers per EC2 and one JVM per container, at any given time, your application can handle 2 * 8 * 256 =4096 customer requests at a time without any communication bumps. Anything above this number will result in delays.

Reactive programming follows a declarative approach as opposed to an imperative approach. It is built in such a way that it solves one waiter waiting on one customer problem. When a client request reaches a microservice, the service allocates it a thread to perform operations/tasks only until the thread is needed. When the thread has done everything it can and is in a blocked state aka waiting for another service to respond, it frees itself up for another client request (just like a waiter would operate at a restaurant). In this way, threads within a service run in an asynchronous fashion.

Foundational Element 2: Technology stack shift:

Most often when we think of using Reactive Spring, we are thinking of upgrading an existing MVC application or a team to reactive. In this case, the daunting question is — How do we migrate?

Successful migration depends on 2 factors —

1. Talent up-skilling
2. Migration that doesn’t compromise the product release velocity.

Changes to the codebase go hand in hand with the team’s learning pace and growth. Begin by socializing the high-level application strategy with the team. Architect the migration considering new incoming product features and existing feature development timelines. This will enable to migrate the codebase incrementally. Identify responsibilities for areas of research and assign learning and engineering tasks.

Foundational Element 3: Engineering shift

To understand the fundamental mechanics of how Spring Reactive works, you have to understand one important concept — Reactive Streams. Reactive streams make asynchronicity possible. In our previous banking example, when Microservice A calls Microservice B and if A and B are built reactive, then the communication between A and B will be facilitated through Reactive streams that can be a Flux or a Mono.

Woah Woah Woah, hold on. We have a lot to unpack there :)

Before we get into Flux and Mono, which is really the crux of Spring Reactive, let’s take a step back to understand what Reactive Streams are.

Imagine a conveyor belt at the airport. The conveyor belt is a good real-world analogy for the reactive stream. Every time there is luggage (data), it gets on the conveyor belt. You see, no passenger is waiting in line to take the luggage one by one. In synchronous world’s example of a conveyor belt, passengers are waiting in line for their turn. In the real world, that is not the case. Real world conveyor belt is asynchronous, similar to Reactive streams. If all the passengers are threads of one big consumer; as and when the data is available, a thread picks it up and continues its processing. This way the threads are able to do their own things just like how passengers can read their book or get food while waiting for their luggage.

Now that we understood what Reactive Streams are, there is a popular type of Reactive Stream — Publisher. Flux and Mono are really an implementation of the conveyor belt. What does this mean?

Imagine a conveyor belt only publishing one data at a time — That type of reactive stream is a mono. Similarly, a belt sending a lot of luggage at a time — that is a Flux.

You may ask me, why do we need a conveyor belt (Mono) that only sends one data? It is a complicated answer and I will dedicate it to its own blog. I will recommend learning more about Flux and Mono implementations if you are a programmer. I tried to link a blog for you, but I realized there is no one comprehensive blog about it, so I will write one real soon. Stay tuned and don’t forget to follow me (shameless advertisement) to get a notification when it comes out.

Finally, the last element to be cautious of. I kept it to the end because this is a really important part of your migration journey.

Foundational Element 4: Customer traffic shift: It is crucial to have a strategy to migrate customers to the new app way earlier in the migration journey. Create a migration plan early on, even before architecting your application with reactive. This will help make informed decisions of your cloud-based solution architecture. If you are migrating your existing application to Reactive, you are essentially pointing your existing customers to the same customer experience but with new underlying mechanics. Consider how to shift customer traffic, back out plan in case of failures, beta testing strategy with a feature flag for phased rollout.

As all good things come with their limitations, there are cases when Reactive may not be the first choice or best choice.

When not to use Reactive:

1. If your single concern is just about serving an occasional spike in user request volume and velocity, do not default to using reactive. Experiment with existing public cloud offerings to enable that.
2. If your application is already utilizing low memory, making it reactive will not produce any significant benefit, because, as we will explore in later paragraphs, Reactive helps optimize high memory utilization applications, not applications that are doing well in memory, but need CPU optimizations. As I explained earlier, optimizing for CPU utilization, is a different beast to solve and Reactive programming is not always the best answer
3. If your downstream application responds slow. Being reactive, only helps so much here. There are other factors outside the realm of your application, increasing the response time
4. If you are not microservice-based. Reactive in a monolithic world adds complexity and introduces unexpected bugs. A couple of years back, moving from monolithic to modular architecture was all about reducing the time to market to reach customers faster. Now, switching to microservices-based architecture has also become about ensuring consistent and highly performant customer experience throughout the lifecycle.