@@ -1016,6 +1016,15 @@ When updating the `gitaly['listen_addr']` or `gitaly['prometheus_listen_addr']`
When this occurs, performing a `sudo gitlab-ctl restart` will resolve the issue. This will no longer be necessary after [this issue](https://gitlab.com/gitlab-org/gitaly/issues/2521) is resolved.
### Permission denied errors appearing in Gitaly logs when accessing repositories from a standalone Gitaly node
If this error occurs even though file permissions are correct, it's likely that
@@ -573,6 +573,127 @@ Particular attention should be shown to:
Congratulations! You have configured a highly available Praefect cluster.
### Failover
There are two ways to do a failover from one internal Gitaly node to another as the primary. Manually, or automatically.
As an example, in this `config.toml` we have one virtual storage named "default" with two internal Gitaly nodes behind it.
One is deemed the "primary". This means that read and write traffic will go to `internal_storage_0`, and writes
will get replicated to `internal_storage_1`:
```toml
socket_path="/path/to/Praefect.socket"
# failover_enabled will enable automatic failover
failover_enabled=false
[logging]
format="json"
level="info"
[[virtual_storage]]
name="default"
[[virtual_storage.node]]
name="internal_storage_0"
address="tcp://localhost:9999"
primary=true
token="supersecret"
[[virtual_storage.node]]
name="internal_storage_1"
address="tcp://localhost:9998"
token="supersecret"
```
#### Manual failover
In order to failover from using one internal Gitaly node to using another, a manual failover step can be used. Unless `failover_enabled` is set to `true`
in the `config.toml`, the only way to fail over from one primary to using another node as the primary is to do a manual failover.
1. Move `primary = true` from the current `[[virtual_storage.node]]` to another node in `/etc/gitlab/gitlab.rb`:
```ruby
praefect['virtual_storages']={
'praefect'=>{
'gitaly-1'=>{
'address'=>'tcp://GITALY_HOST:8075',
'token'=>'PRAEFECT_INTERNAL_TOKEN',
# no longer the primary
},
'gitaly-2'=>{
'address'=>'tcp://GITALY_HOST:8075',
'token'=>'PRAEFECT_INTERNAL_TOKEN',
# this is the new primary
'primary'=>true
},
'gitaly-3'=>{
'address'=>'tcp://GITALY_HOST:8075',
'token'=>'PRAEFECT_INTERNAL_TOKEN',
}
}
}
```
1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
On a restart, Praefect will send write traffic to `internal_storage_1`. `internal_storage_0` is the new secondary now,
and replication jobs will be created to replicate repository data to `internal_storage_0`**from**`internal_storage_1`
#### Automatic failover
When automatic failover is enabled, Praefect will do automatic detection of the health of internal Gitaly nodes. If the
primary has a certain amount of healthchecks fail, it will decide to promote one of the secondaries to be primary, and
demote the primary to be a secondary.
1. To enable automatic failover, edit `/etc/gitlab/gitlab.rb`:
```ruby
# failover_enabled turns on automatic failover
praefect['failover_enabled']=true
praefect['virtual_storages']={
'praefect'=>{
'gitaly-1'=>{
'address'=>'tcp://GITALY_HOST:8075',
'token'=>'PRAEFECT_INTERNAL_TOKEN',
'primary'=>true
},
'gitaly-2'=>{
'address'=>'tcp://GITALY_HOST:8075',
'token'=>'PRAEFECT_INTERNAL_TOKEN'
},
'gitaly-3'=>{
'address'=>'tcp://GITALY_HOST:8075',
'token'=>'PRAEFECT_INTERNAL_TOKEN'
}
}
}
```
1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
Below is the picture when Praefect starts up with the config.toml above:
| [PgBouncer](../../development/architecture.md#pgbouncer) | Database Pool Manager | [PgBouncer HA configuration](pgbouncer.md)**(PREMIUM ONLY)** |
| [Redis](../../development/architecture.md#redis)[^3] with Redis Sentinel | Key/Value store for shared data with HA watcher service | [Redis HA configuration](redis.md) |
| [Gitaly](../../development/architecture.md#gitaly)[^2][^5] [^7] | Recommended high-level storage for Git repository data | [Gitaly HA configuration](gitaly.md) |
| [Sidekiq](../../development/architecture.md#sidekiq) | Asynchronous/Background jobs | |
| [Sidekiq](../../development/architecture.md#sidekiq) | Asynchronous/Background jobs | [Sidekiq configuration](sidekiq.md) |
@@ -353,10 +353,6 @@ configuring a different storage driver. By default the GitLab Container Registry
is configured to use the filesystem driver, which makes use of [storage path](#container-registry-storage-path)
configuration.
NOTE: **Note:** Enabling a storage driver other than `filesystem` would mean
that your Docker client needs to be able to access the storage backend directly.
In that case, you must use an address that resolves and is accessible outside GitLab server. The Docker client will continue to authenticate via GitLab but data transfer will be direct to and from the storage backend.
The different supported drivers are:
| Driver | Description |
...
...
@@ -425,6 +421,55 @@ storage:
NOTE: **Note:**
`your-s3-bucket` should only be the name of a bucket that exists, and can't include subdirectories.
### Disable redirect for storage driver
By default, users accessing a registry configured with a remote backend are redirected to the default backend for the storage driver. For example, registries can be configured using the `s3` storage driver, which redirects requests to a remote S3 bucket to alleviate load on the GitLab server.
However, this behaviour is undesirable for registries used by internal hosts that usually can't access public servers. To disable redirects, set the `disable` flag to true as follows. This makes all traffic to always go through the Registry service. This results in improved security (less surface attack as the storage backend is not publicly accessible), but worse performance (all traffic is redirected via the service).
**Omnibus GitLab installations**
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
registry['storage'] = {
's3' => {
'accesskey' => 's3-access-key',
'secretkey' => 's3-secret-key-for-access-key',
'bucket' => 'your-s3-bucket',
'region' => 'your-s3-region',
'regionendpoint' => 'your-s3-regionendpoint'
},
'redirect' => {
'disable' => true
}
}
```
1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
**Installations from source**
1. Add the `redirect` flag to your registry configuration YML file:
```yml
storage:
s3:
accesskey: 'AKIAKIAKI'
secretkey: 'secret123'
bucket: 'gitlab-registry-bucket-AKIAKIAKI'
region: 'your-s3-region'
regionendpoint: 'your-s3-regionendpoint'
redirect:
disable: true
cache:
blobdescriptor: inmemory
delete:
enabled: true
```
1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect.
### Storage limitations
Currently, there is no storage limitation, which means a user can upload an
msgid "ClusterIntegration|Set a prefix for your namespaces. If not set, defaults to your project path. If modified, existing environments will use their current namespaces until the cluster cache is cleared."
msgstr ""
msgid "ClusterIntegration|Set the global mode for the WAF in this cluster. This can be overridden at the environmental level."
