Messages persistence implementation missed when message bus service down

[raja-maragani]

Description

REMReM publish is publishing Eiffel messages into message bus(RabbitMQ).
Reliability perspective RabbitMQ defined: Data safety is a joint responsibility of RabbitMQ nodes, publishers and consumers.

If RabbitMQ service unavailability due to various reason the REMReM publish service will not fail & same the messages in REMReM local disk and publish messages when RabbitMQ service available.

Motivation

REMReM client no need to handle Message bus failures and resend messages. REMReM publish monitor thread check the messages bus available or not if available messages will publish.

Exemplification

Benefits

1. REMReM clients no need to implement the persistence logic them self.
2. The client will have the flexibility to persist message based on request param persistIfMBDown=true/false, default true.
3. REMReM Publish endpoint response client knows the message persisted or published

Possible Drawbacks

REMReM publish endpoints /producer/msg & /generateAndPublish backward-compatible will miss due to this change

REMReM publish Request and Response changes

1. REMReM publish endpoint a new request parameter will add (persistIfMBDown=true/false, default true)
2. REMReM publish endpoint response will change when message persisted.

[magnusbaeck]

Implementing a persisted queue that's reliable and performs well isn't entirely easy. Logstash made that journey a while back. I'd like to make two points:

• REMReM clients must always have retry logic to cope with failures. The two REMReM components are currently stateless so they're easy to scale and distribute, but we can never clear clients of responsibility.
• If you want to reduce the impact of an unavailable RabbitMQ, why not deploy a local RabbitMQ instance that acts as the first relay? If run close to REMReM Publish (same machine, same pod, whatever abstraction applies) it's arguably less likely to be unavailable than a full-blown cluster running somewhere else. To publish messages to the rest of the organization the shovel plugin can be used.

[raja-maragani]

Asynchronous communication model, we are following the message bus to persistence, and if we maintain persistence to replace the message bus it is very tricky to track our communication end to end.

Eiffel Principles wise we need to discuss, we keep this Item in the block and proceed with some discussion around here and proceed.

[magnusbaeck]

Asynchronous communication model, we are following the message bus to persistence, and if we maintain persistence to replace the message bus it is very tricky to track our communication end to end.

I'm afraid I don't understand what you mean. Could you please elaborate?

[SantoshNC68]

@raja-maragani can you please elaborate for @magnusbaeck 's understanding.