Commit b00d37ee authored by James Fargher's avatar James Fargher

Support restoring repository backups in parallel

Parallel restore was recently added to downstream gitaly-backup. Now we
can configure it properly on the gitlab side.

Changelog: changed
parent 5a614293
......@@ -347,18 +347,21 @@ sudo -u git -H GITLAB_ASSUME_YES=1 bundle exec rake gitlab:backup:restore RAILS_
#### Back up Git repositories concurrently
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37158) in GitLab 13.3.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37158) in GitLab 13.3.
> - [Concurrent restore introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69330) in GitLab 14.3
When using [multiple repository storages](../administration/repository_storage_paths.md),
repositories can be backed up concurrently to help fully use CPU time. The
repositories can be backed up or restored concurrently to help fully use CPU time. The
following variables are available to modify the default behavior of the Rake
task:
- `GITLAB_BACKUP_MAX_CONCURRENCY`: The maximum number of projects to back up at
the same time. Defaults to `1`.
the same time. Defaults to the number of logical CPUs (in GitLab 14.1 and
earlier, defaults to `1`).
- `GITLAB_BACKUP_MAX_STORAGE_CONCURRENCY`: The maximum number of projects to
back up at the same time on each storage. This allows the repository backups
to be spread across storages. Defaults to `1`.
to be spread across storages. Defaults to `2` (in GitLab 14.1 and earlier,
defaults to `1`).
For example, for Omnibus GitLab installations with 4 repository storages:
......
......@@ -22,8 +22,8 @@ module Backup
end
args = []
args += ['-parallel', @parallel.to_s] if type == :create && @parallel
args += ['-parallel-storage', @parallel_storage.to_s] if type == :create && @parallel_storage
args += ['-parallel', @parallel.to_s] if @parallel
args += ['-parallel-storage', @parallel_storage.to_s] if @parallel_storage
@stdin, stdout, @thread = Open3.popen2(ENV, bin_path, command, '-path', backup_repos_path, *args)
......
......@@ -131,8 +131,19 @@ RSpec.describe Backup::GitalyBackup do
context 'parallel option set' do
let(:parallel) { 3 }
it 'does not pass parallel option through' do
expect(Open3).to receive(:popen2).with(ENV, anything, 'restore', '-path', anything).and_call_original
it 'passes parallel option through' do
expect(Open3).to receive(:popen2).with(ENV, anything, 'restore', '-path', anything, '-parallel', '3').and_call_original
subject.start(:restore)
subject.wait
end
end
context 'parallel_storage option set' do
let(:parallel_storage) { 3 }
it 'passes parallel option through' do
expect(Open3).to receive(:popen2).with(ENV, anything, 'restore', '-path', anything, '-parallel-storage', '3').and_call_original
subject.start(:restore)
subject.wait
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment