Microservices: Protocols and Design Patterns

The modern distributed systems has been shifting from monolithic codebases toward microservices. While this move brings adaptability and scalability, it introduces significant complexity in how services communicate and maintain data consistency. Choosing the right communication protocol, gateway architecture, and transaction model is a critical architectural decision that determines whether a system thrives or buckles under production load.

Protocols

In microservices, getting different parts of the application to talk to each other, there are three dominant protocols; REST, gRPC, and GraphQL. Choosing between those should be based on specific payload sizes and call patterns.

REST

Representational State Transfer (REST) is the most popular architecture, which is designed for network-based (client-server) applications. It is native to the web. It relies on standard HTTP verbs (GET, POST, PUT, DELETE) to represent state transitions.

• Strengths: Excellent browser support, native alignment with internet routing, and robust caching.
• Weaknesses: Over-fetching (receiving too much data), and under-fetching (requiring multiple calls), which may cause chatty APIs. Also, the need for versioning prefixes like /v1/ and /v2/.

gRPC

Created by Google, gRPC is a service-oriented framework built on HTTP/2 (transport) and Protocol Buffers (Protobuf). It uses binary wire serialization, which is significantly more efficient than text-based JSON.

• Strengths: Low latency, multiplexed connections (many requests over one TCP connection), and strongly typed contracts.
• Weaknesses: Limited browser support and opaque binary payloads (difficult to inspect).

GraphQL

Created by Facebook, GraphQL is a query language. It lets clients control the data shape, allowing them to request only the required fields through a single endpoint.

• Strengths: No over-fetching, and supports schema evolution without global versions.
• Weaknesses: N+1 query problem (executing independent database lookups for nested items) and security risks (e.g., deeply nested malicious queries that can exhaust server resources).

Comparison

Takeaways

• Choose REST when building public-facing APIs, standard web applications, or need simple caching layer compatibility.
• Choose gRPC when latency, payload size, and internal microservice-to-microservice communication efficiency are top priorities.
• Choose GraphQL when frontend apps need to fetch deeply nested or specific data from multiple backends (in a single round trip).

Patterns

As systems transition from shared databases to independent service datastores, ensuring ACID (atomicity, consistency, isolation, durability) across the service boundaries becomes hard. It is no longer an option to rely on a single database transaction to keep everything in sync. So, we manage eventual consistency using distributed patterns, such as Sagas, Outbox, and Event Sourcing.

Sagas

A Saga manages a complex, long-running processes by breaking a large transaction into a sequence of smaller, local transactions. If one step fails, the Saga triggers compensating transactions to undo previous changes.

Coordination

• Choreography: Decentralized; services exchange events and trigger own local transactions.
• Orchestration: Centralized; a coordinator directs the workflow and failure recovery.

Outbox

One of the most common pitfalls in microservices is the Dual Write Problem: successfully updating a database but failing to publish an event to a message broker (e.g., due to a network crash). Outbox Pattern solves this by saving the event to an outbox table in the same database transaction. A separate process then uses Change Data Capture (CDC) to read the database logs and push events to the message broker, guaranteeing at-least-once delivery.

Event Sourcing

This pattern is about storing object state as an append-only log of events rather than the current state. Check out the Martin Fowler's talk on youtube. Replaying these events allows to rebuild the state at any point in time.

In a State-Based system (like SQL), you store the current state of an object (e.g., ID=101, Name="Alice" and Email="[email protected]"). When Alice updates her email, you overwrite her old email. The history is lost unless you have manually built separate audit tables.

In an Event Sourced system, you do not store the current state. You store an append-only log of every business event that has occurred (e.g., UserCreated, UserNameChanged, UserEmailChanged). To find Alice's current state, you start from a blank slate and replay all events associated with her ID.

Edge Patterns and Resiliency

To shield microservices, there are some edge patterns and fault-tolerance strategies.

Multiple Client Types

When multiple client types (web, iOS, Android) need data from many services, you have two main choices:

• BFF (Backend for Frontend): Create client-specific backend components to optimize payloads for each device.
• GraphQL Federation: Use a declarative, bottom-up model where downstream services act as "subgraphs" that a central router merges into a unified "supergraph". This grants teams autonomy while presenting a single API to clients.

Protocol Transcoding

To maintain the performance of gRPC internally while supporting standard web clients, gateways can perform inline transcoding. Some tools can map RESTful HTTP/JSON parameters directly to gRPC method calls, allowing a single backend to serve both worlds.

Resiliency Patterns

• Circuit Breakers: Tripping to an "Open" state when a downstream service fails, immediately rejecting requests to protect upstream resources.
• gRPC Deadlines: Unlike static timeouts, deadlines propagate an absolute point in time (e.g., T+5 seconds) down the entire call chain. If the budget is exhausted, downstream services abort early to stop "zombie requests" from wasting CPU.
• Retries with Jitter: Using exponential backoff with randomized variations to prevent "retry storms" that can overwhelm struggling services.

Conclusion

There is no "one size fits all" protocol. The most effective systems utilize a multi-protocol architecture.

• Use gRPC for high-throughput, low-latency communication between internal services.
• Expose REST at the public edge to maximize compatibility and leverage existing internet caching infrastructure.
• Implement GraphQL Federation when frontend teams need to aggregate data from dozens of shifting sub-services.
• Enforce data integrity, such as via the Outbox pattern and orchestrated Sagas for complex workflows.