Need Some General Info Regarding Client Design & partitioning #6437
Unanswered
YashasAnand
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I currently have a single connection per application set up on my clients. Thus all Produce & consume happen on this same connection. Does having a Producer per subject helpful in anyway or is it an anti-pattern?
What is the recommended replication factor for streams in a 3 node nats k8s cluster? In kafka we have min.isr & is set to 2, it says minimum 2 replicas should ack a request to be successful, does this mean i need to have replication of 3 to achive min.isr of 2 in NATS? I observe replication overhead is quite high at high rps rates. Considering we have multiple streams is it recommended to replicate all streams with replication of 3?
As @jnmoyne states in Event Synchronization with NATS nats.go#970,
"If you have a very high message rate you want to capture in a stream and you are reaching the limit of performance a single stream can handle and you want to scale that stream then you would create a stream per partition (and one durable per partition stream)."
We have a use case where we dont require strict ordering of messages, my 1st question is , lets say if i have 5 partitioned streams & 1 durable consumer per stream , how should the producing application produce to this stream, do we have any example blog on this? Is there some stream mapping functionality similar to subject transforms?
My 2nd question is as per @Jarema in #6100, having many streams can cause RAFT issues especially at higher replication factors. So can anyone tell me what kind of trade offs am i looking at here for implemnting stream per partition?
Is a stream equivalent to partition in kafka?
Beta Was this translation helpful? Give feedback.
All reactions