Within collect we are using Load Balencer. There is a setting for max jobs per LB queue but due to the way Collect service pages allocates the jobs, we are not seeing multiplue jobs in a LB queue. We are using manual SDS jobs, but i suspect the same is true for all collect job and also applies to no-load balancer solutions.
At the moment, the Service page goes down the list of LB queues adding a single job to each then sleeps. If the jobs run fast, then the queue empties quickly and the LB does nothing until the Service page runs again and adds another job. This means that our collect process is comprised of about 95% wait time. As a reuslt we have had to change the service page intervals from several minutes to 10 seconds and even now, about 30% of the time appears to be wait time.
Ideally, this process would run in a loop until all jobs are in the LB queues, or the LB is fully subscribed. This way the Service page run frequency can be longer.
This idea relates to a custom web application, not core product. The details have been passed onto the relevant team in Syniti.