Commit 9bb0ca4f authored by Harsh Chouraria's avatar Harsh Chouraria Committed by Evan Read

Clarify repository data moves for Gitaly Cluster

The approaches to copy/move repository data using tar, rsync, etc.
will not work for Gitaly Cluster targets/destinations because they
only go to a single host and because they bypass Praefect.

This change attempts to make more explicit on what approaches
would work for Gitaly Cluster (only backup and restore) and
what will not.
parent 3e98649b
...@@ -184,8 +184,8 @@ Each of the approaches we list can or does overwrite data in the target director ...@@ -184,8 +184,8 @@ Each of the approaches we list can or does overwrite data in the target director
### Recommended approach in all cases ### Recommended approach in all cases
The GitLab [backup and restore capability](../../raketasks/backup_restore.md) should be used. Git For either Gitaly or Gitaly Cluster targets, the GitLab [backup and restore capability](../../raketasks/backup_restore.md)
repositories are accessed, managed, and stored on GitLab servers by Gitaly as a database. Data loss should be used. Git repositories are accessed, managed, and stored on GitLab servers by Gitaly as a database. Data loss
can result from directly accessing and copying Gitaly's files using tools like `rsync`. can result from directly accessing and copying Gitaly's files using tools like `rsync`.
- From GitLab 13.3, backup performance can be improved by - From GitLab 13.3, backup performance can be improved by
...@@ -193,13 +193,15 @@ can result from directly accessing and copying Gitaly's files using tools like ` ...@@ -193,13 +193,15 @@ can result from directly accessing and copying Gitaly's files using tools like `
- Backups can be created of just the repositories using the - Backups can be created of just the repositories using the
[skip feature](../../raketasks/backup_restore.md#excluding-specific-directories-from-the-backup). [skip feature](../../raketasks/backup_restore.md#excluding-specific-directories-from-the-backup).
No other method works for Gitaly Cluster targets.
### Target directory is empty: use a `tar` pipe ### Target directory is empty: use a `tar` pipe
If the target directory `/mnt/gitlab/repositories` is empty the For Gitaly targets (use [recommended approach](#recommended-approach-in-all-cases) for Gitaly Cluster targets), if the
simplest thing to do is to use a `tar` pipe. This method has low target directory `/mnt/gitlab/repositories` is empty the simplest thing to do is to use a `tar` pipe. This method has
overhead and `tar` is almost always already installed on your system. low overhead and `tar` is almost always already installed on your system.
However, it is not possible to resume an interrupted `tar` pipe: if
that happens then all data must be copied again. However, it is not possible to resume an interrupted `tar` pipe; if that happens then all data must be copied again.
```shell ```shell
sudo -u git sh -c 'tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\ sudo -u git sh -c 'tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\
...@@ -210,9 +212,9 @@ If you want to see progress, replace `-xf` with `-xvf`. ...@@ -210,9 +212,9 @@ If you want to see progress, replace `-xf` with `-xvf`.
#### `tar` pipe to another server #### `tar` pipe to another server
You can also use a `tar` pipe to copy data to another server. If your For Gitaly targets (use [recommended approach](#recommended-approach-in-all-cases) for Gitaly Cluster targets), you can
`git` user has SSH access to the new server as `git@newserver`, you also use a `tar` pipe to copy data to another server. If your `git` user has SSH access to the new server as
can pipe the data through SSH. `git@newserver`, you can pipe the data through SSH.
```shell ```shell
sudo -u git sh -c 'tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\ sudo -u git sh -c 'tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\
...@@ -228,11 +230,11 @@ WARNING: ...@@ -228,11 +230,11 @@ WARNING:
Using `rsync` to migrate Git data can cause data loss and repository corruption. Using `rsync` to migrate Git data can cause data loss and repository corruption.
[These instructions are being reviewed](https://gitlab.com/gitlab-org/gitlab/-/issues/270422). [These instructions are being reviewed](https://gitlab.com/gitlab-org/gitlab/-/issues/270422).
If the target directory already contains a partial / outdated copy If the target directory already contains a partial or outdated copy of the repositories it may be wasteful to copy all
of the repositories it may be wasteful to copy all the data again the data again with `tar`. In this scenario it is better to use `rsync` for Gitaly targets (use
with `tar`. In this scenario it is better to use `rsync`. This utility [recommended approach](#recommended-approach-in-all-cases) for Gitaly Cluster targets).
is either already installed on your system, or installable
by using `apt` or `yum`. This utility is either already installed on your system, or installable using `apt` or `yum`.
```shell ```shell
sudo -u git sh -c 'rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ sudo -u git sh -c 'rsync -a --delete /var/opt/gitlab/git-data/repositories/. \
...@@ -249,8 +251,9 @@ WARNING: ...@@ -249,8 +251,9 @@ WARNING:
Using `rsync` to migrate Git data can cause data loss and repository corruption. Using `rsync` to migrate Git data can cause data loss and repository corruption.
[These instructions are being reviewed](https://gitlab.com/gitlab-org/gitlab/-/issues/270422). [These instructions are being reviewed](https://gitlab.com/gitlab-org/gitlab/-/issues/270422).
If the `git` user on your source system has SSH access to the target For Gitaly targets (use [recommended approach](#recommended-approach-in-all-cases) for Gitaly Cluster targets), if the
server you can send the repositories over the network with `rsync`. `git` user on your source system has SSH access to the target server you can send the repositories over the network with
`rsync`.
```shell ```shell
sudo -u git sh -c 'rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ sudo -u git sh -c 'rsync -a --delete /var/opt/gitlab/git-data/repositories/. \
...@@ -269,17 +272,18 @@ Every time you start an `rsync` job it must: ...@@ -269,17 +272,18 @@ Every time you start an `rsync` job it must:
- Inspect all files in the target directory. - Inspect all files in the target directory.
- Decide whether or not to copy files. - Decide whether or not to copy files.
If the source or target directory If the source or target directory has many contents, this startup phase of `rsync` can become a burden for your GitLab
has many contents, this startup phase of `rsync` can become a burden server. You can reduce the workload of `rsync` by dividing its work into smaller pieces, and sync one repository at a
for your GitLab server. You can reduce the workload of `rsync` by dividing its time.
work in smaller pieces, and sync one repository at a time.
In addition to `rsync` we use [GNU Parallel](http://www.gnu.org/software/parallel/). In addition to `rsync` we use [GNU Parallel](http://www.gnu.org/software/parallel/).
This utility is not included in GitLab, so you must install it yourself with `apt` This utility is not included in GitLab, so you must install it yourself with `apt`
or `yum`. or `yum`.
This process does not clean up repositories at the target location that no This process:
longer exist at the source.
- Doesn't clean up repositories at the target location that no longer exist at the source.
- Only works for Gitaly targets. Use [recommended approach](#recommended-approach-in-all-cases) for Gitaly Cluster targets.
#### Parallel `rsync` for all repositories known to GitLab #### Parallel `rsync` for all repositories known to GitLab
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment