Service calls that survive a restart
Replacing direct HTTP between services with a broker, so one failure stops being everyone's failure.
The problem
With services calling each other over HTTP, one service going down stopped the chain. Worse, after a restart there was no record of what had been requested, so nothing could be replayed.
What I did
- Introduced RabbitMQ as the message broker between services, giving every request a queue it can wait in rather than a connection it can lose.
- Implemented the RPC request-reply pattern, opening a reply channel per request so callers still get their response back.
- Leaned on FIFO ordering and queue durability as the failure mechanism, instead of retry logic scattered through each caller.
Inter-service calls became queued and replayable, so a restart delays work instead of losing it — the property that made scaling the system out actually safe.
The outcome
Read more
I wrote this one up in full: RabbitMQ Implementation for Microservices Communication on Node.js.
