Maximum parallel replications for pull

[Joseph94m]

Is your feature request related to a problem? Please describe.
It's annoying and causes an avalanche effect when my pull based replications for the same Job start before the previous one is over.

It starts off by taking 2 minutes, then it slowly starts piling up until it reaches hours...

Describe the solution you'd like
The ability to specify, in the replication configuration:

• Maximum of replications waiting for an inProgress replication to finish
• Maximum replications that can be inProgress

Describe the main design/architecture of your solution

What do you guys think before we discuss implementation

[stonezdj]

The cron string is 0 */2 * * * *, it means that the replication is start every 2 minutes, you should always keep the schedule interval longer than a single job complete time.

[Joseph94m]

Hello, that's the fallback approach indeed.
But you cannot always guarantee a replication's time due to occasional bulk insertions in the source registry and network variability.

[stonezdj]

In your case, most of the InProgress replication job is actually Pending, you could check the job in the job service dashboard, they should in the job queue. you could adjust the job service worker count to control the max Parallel job.

[bupd]

But at this point we can clearly see. It creates huge load on system and lot of wasted bandwidth.
It is better to have a replication skip option.

[bupd]

The cron string is 0 */2 * * * *, it means that the replication is start every 2 minutes, you should always keep the schedule interval longer than a single job complete time.

The workaround of setting a longer replication interval, like once a day, fails to address the need for timely synchronization across registries. For users who rely on Harbor to maintain identical registries at different locations, frequent replication (e.g., every 5 minutes) is necessary to ensure minimal discrepancies between registries. By suggesting a longer interval, users may end up with outdated or inconsistent images, undermining the core functionality of replication.

Thanks.