Optimization Suggestion for Job Retrieval in Nexrender with Redis

[naitik30]

First, I would like to express my gratitude for creating such an amazing library. It has been working great for our needs.
Currently, we are using Nexrender with network rendering and a Redis database setup to render short videos (approximately 30 seconds each). This setup is functioning well, but we are facing performance challenges when there are a large number of jobs in the pending queue. Specifically, when we have around 30k-40k jobs, it takes approximately 1-2 minutes for the Nexrender server to assign jobs to workers.
From my understanding, the Nexrender server checks for jobs by running a query that fetches all jobs, identifies the queued ones, and assigns them to available workers. Given the high volume of jobs, this process is becoming time-consuming.
To optimize this, would it be possible to modify the Nexrender server or the Nexrender-Redis package to include a route that allows the server to fetch and assign a single queued job directly, rather than querying all jobs at once? This could significantly reduce the delay in job assignment and improve overall performance.
Thank you for your time and for considering this improvement. I look forward to hearing your thoughts on this suggestion.