-
Notifications
You must be signed in to change notification settings - Fork 410
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Global Concurrency Setting for Bull-MQ Workers #2461
Comments
The concurrency setting can only be set on each worker, so if you have two workers then it will add the concurrency factor of every worker, in the example above the total concurrency would be 2. |
Thank you for your response. In our system, certain critical processes require sequential execution, and the current behavior of Bull-MQ poses a challenge in maintaining this sequence when the server is running on multiple instances. While we appreciate the concurrency settings on individual workers, having a global concurrency setting would simplify the management of sequential processes across all instances of the server. This is particularly crucial for processes that depend on a specific order of execution. Having a global concurrency setting would allow us to ensure that, regardless of the number of instances, these processes are executed sequentially, meeting our system requirements more effectively. We believe this enhancement would greatly enhance the flexibility and usability of Bull-MQ in scenarios like ours. Thank you for your consideration. |
@faisal-merchlink It is possible to achieve what you are asking with BullMQ Pro. You need to use the groups feature and set the maximum concurrency to 1, that will effectively imply that maximum one job for that group would be executed independently on the number of workers: https://docs.bullmq.io/bullmq-pro/groups/concurrency |
This PR should address this issue: #2465 |
Hi @faisal-merchlink for global concurrency, It's already addressed in this pr #2496 |
I hope this message finds you well. Currently, we are facing a challenge in our system where the need for sequential execution of certain processes conflicts with Bull-Mq's default behaviour, particularly when the server is running on multiple instances. This situation has prompted us to seek an enhancement in Bull-Mq to better support our use case.
The first instance of the server is adding jobs to the queue, and both the first and second instances of the server workers are executing these jobs sequentially. However, I want to have a global concurrency setting for the workers.
The text was updated successfully, but these errors were encountered: