by Chukwuemeka Ajima
The circuit breaker pattern is a very well known pattern in software engineering, microservices architecture especially. In microservices, numerous services are usually dependent on one another and these services are usually distributed on the same or different remote servers and as a system, all the services need to always be up with a very low request-response latency to ensure the smooth operation of the entire system. But in practice, things don’t always turn out the intended way.
Assuming one service that many other services depend on goes down or starts exhibiting high request-response latency, then all the services that are dependent on it fail or start experiencing longer response time, the circuit breaker solves this problem of preventing cascading failures across the microservice.
A simple retry logic might not be very efficient and can even worsen the situation due to excess service request of the retry logic. The circuit breaker pattern helps to prevent such a catastrophic cascading failure across multiple systems.
The circuit breaker pattern allows you to build a fault-tolerant and resilient system that can survive gracefully when key services are either unavailable or have high latency.
The circuit breaker pattern helps to prevent such a catastrophic cascading failure across multiple systems. The circuit breaker pattern allows you to build a fault-tolerant and resilient system that can survive gracefully when key services are either unavailable or have high latency.
Everything fails at some point
Anything can fail, it’s a known fact, but how to manage the failure is what matters. Circuit breaker helps to gracefully manage this failure in such a way that it doesn’t cause more damages to the system. The concept behind the circuit breaker pattern is very simple and straightforward.
The circuit breaker is just a wrapper that wraps a service or services (usually that which many other services depend on) with a monitor that tracks failure. The circuit breaker has three distinct states and can only be in one of the states at any given time:
- Closed: If everything is okay, the circuit breaker remains in a closed state and allows all requests to pass through to the underlying service(s). When the number of failures exceeds a predetermined threshold, the breaker trips and it goes into the Open state.
- Open: In this state, the circuit is completely open and blocks all requests from reaching the unavailable service.
- Half Open: After a timeout period, the circuit switches to a half-open state to test if the underlying problem still exists. If a single call fails in this half-open state, the breaker is once again tripped. If it succeeds, the circuit breaker resets back to the normally closed state.
Closed State
When the circuit breaker is in the closed state all calls go through to the supplier microservice or services without any failure or latency.
Open State
If the wrapped (supplier microservice) is experiencing slowness or high response time, the circuit breaker receives timeouts for any requests to that service. Once the number of timeouts reaches a predetermined threshold, it trips the circuit breaker to the open state.
In the open state, the circuit breaker returns an error or reroutes the request to an available service, database or cache for all calls to the service without making the calls to the supplier microservice. This behaviour allows the supplier microservice to recover by reducing its load.
Half Open
The circuit breaker uses a monitoring logic called the half-open state to monitor when the supplier microservice comes back online. It uses this mechanism to make a trial call to the supplier microservice within a chosen time interval to check if it has fully recovered and is ready to accept and fulfil requests.
If the call to the supplier microservice times out or errors, the circuit breaker remains in the open state. If the call returns success, then the circuit switches to the closed state. The circuit breaker then returns all external calls to the service with an error during the half-open state.
Implementation
There are various implementations of circuit breakers in different languages, also different libraries exist that implement. he circuit breaker pattern in languages like Java and JavaScript. Hystrix is a Java implementation developed by Netflix for its internal use and later open-sourced. Similarly, Opossum is the JavaScript (Node.js) implementation of the circuit breaker pattern. Also, it is important to bear in mind that a custom (built from scratch) circuit breaker pattern.
Also, it is important to bear in mind that a custom (built from scratch) circuit breaker class/function can be easily implemented and customized to project-specific needs or service specifications.
Conclusion
The circuit breaker pattern is very useful as it facilitates service discovery, monitoring and logging processes. can also route requests based on the current state to ensure service availability by adding logic to ensure a truly resilient and fault-tolerant system.
For example, if the product service of an e-commerce microservice is not available, then the circuit breaker is open, however, the logic can then route requests (during the open state) to a cache server or a backup microservice pending when the main product service comes up and all requests routed back to the product service to resume normal operation, with this, a list of backup servers can be configured to assume operation when the primary microservice is offline or experiencing high latency.
The circuit breaker can also be extended to act as load balancer reducing the load on a specific service and waiting for the service to upscale or fully recover to low latency before it can resume full operation.
Note
In the [Part Two] of this article we will see how to implement a custom circuit breaker class, with backup services and we will also simulate a service downtime on the primary (supplier service) to see how the circuit breaker routes the requests to the first available backup (secondary) service.
Further reading
● Martin Fowlers Circuit Breaker Pattern
