All ideas are welcome. Just because the Idea doesn't make it into the product immediately does not make it a bad idea.
Within collect we are using Load Balencer. There is a setting for max jobs per LB queue but due to the way Collect service pages allocates the jobs, we are not seeing multiplue jobs in a LB queue. We are using manual SDS jobs, but i suspect the same is true for all collect job and also applies to no-load balancer solutions.
At the moment, the Service page goes down the list of LB queues adding a single job to each then sleeps. If the jobs run fast, then the queue empties quickly and the LB does nothing until the Service page runs again and adds another job. This means that our collect process is comprised of about 95% wait time. As a reuslt we have had to change the service page intervals from several minutes to 10 seconds and even now, about 30% of the time appears to be wait time.
Ideally, this process would run in a loop until all jobs are in the LB queues, or the LB is fully subscribed. This way the Service page run frequency can be longer.
Phil Towe
This idea relates to a custom web application, not core product. The details have been passed onto the relevant team in Syniti.
Andy, this refers to the way all collect jobs are run, i think. In our case we are using Load balancer, but even when we are not, there is a massive amount of wait time.
I think its the way the core product works.
Attachments Open full size
Andy, I do not think it is limited to the Load Balancer, but I put that in as it maybe. I think it applies to all collect jobs, but you will know better.
Basically, the way jobs are processed at the moment, it appears ADM spend a lot of time in a wait state. The jobs get called one at a time with a sleep in the middle instead of all "available slots" being used, and then the launcher sleeping until the next jobs are launched.
Attachments Open full size
Hello Phil - Are you referring to psaLoadBalance? If so, this not part of our core product, so this feedback with need to be referred to the owners of that application.
Attachments Open full size