-
Notifications
You must be signed in to change notification settings - Fork 410
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(queue): add global concurrency #2496
Conversation
c760525
to
486aa07
Compare
486aa07
to
dccea30
Compare
Our team would greatly benefit from this feature as we have multiple backends that each spawn a worker but only one scheduled job should be executed at the same time. Looking forward to this! <3 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This LGTM, thanks to the way we handle paused queues we get this almost for free. I think it will be quite important though to explain the documentation of this feature that if you choose a concurrency level in your workers it will not override the global one, it will just be the maximum jobs a given worker can process in parallel but never more than the global one.
I really appreciate for implementing global concurrency feature. 🙇♂️ |
What does this paragraph in the document mean?「It is not possible to achieve a global concurrency of at most 1 job at a time if you use more than one worker.」What if we setGlobalConcurrency to 1 when using multiple workers ? |
hi @faller, we missed to change that statement now that we have global concurrency. And yes, you can set global concurrency as 1 when using múltiple workers |
Thank you for your excellent work |
hi @roggervalf , If I set a queue with 10 global concurrency, when there are 2 wokers, should I set worker level concurrency at the same time or keep it default ? |
hi @faller, as from our docs https://docs.bullmq.io/guide/queues/global-concurrency, global concurrency is the maximum value that any worker can handle. Worker concurrency can be set at any time independently. Also the default local concurrency value for a worker is 1. If you want to take advantage of your global concurrency, you should add a local concurrency with a value no more than 10. |
Thank you for your answer |
ref #2465