Skip to content

G
gitlab-ce
Commit 735e1f2c authored by Marcel Amirault's avatar Marcel Amirault Committed by Evan Read
Browse files
Options

Fix whitespace in many administration docs 

Many code blocks are 4spaced, and they render in GitLab
without coloring as a result, even though they are
fenced with a language label. If in a list, other items
will render as being in a code block too, even if not
meant to. This fixes all these issues for many admin
docs in /high_availability and /auth (part 1)
parent e105232c
Hide whitespace changes
Inline Side-by-side
Showing with 1247 additions and 1232 deletions
doc/administration/auth/authentiq.md
View file @ 735e1f2c
......@@ -8,47 +8,48 @@ Authentiq will generate a Client ID and the accompanying Client Secret for you t
1. On your GitLab server, open the configuration file:
For omnibus installation
```sh
sudo editor /etc/gitlab/gitlab.rb
```
For omnibus installation
For installations from source:
```sh
sudo editor /etc/gitlab/gitlab.rb
```
```sh
sudo -u git -H editor /home/git/gitlab/config/gitlab.yml
```
For installations from source:
```sh
sudo -u git -H editor /home/git/gitlab/config/gitlab.yml
```
1. See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration) for initial settings to enable single sign-on and add Authentiq as an OAuth provider.
1. Add the provider configuration for Authentiq:
For Omnibus packages:
```ruby
gitlab_rails['omniauth_providers'] = [
{
"name" => "authentiq",
"app_id" => "YOUR_CLIENT_ID",
"app_secret" => "YOUR_CLIENT_SECRET",
"args" => {
"scope": 'aq:name email~rs address aq:push'
}
}
]
```
For installations from source:
```yaml
- { name: 'authentiq',
app_id: 'YOUR_CLIENT_ID',
app_secret: 'YOUR_CLIENT_SECRET',
args: {
scope: 'aq:name email~rs address aq:push'
}
}
```
For Omnibus packages:
```ruby
gitlab_rails['omniauth_providers'] = [
{
"name" => "authentiq",
"app_id" => "YOUR_CLIENT_ID",
"app_secret" => "YOUR_CLIENT_SECRET",
"args" => {
"scope": 'aq:name email~rs address aq:push'
}
}
]
```
For installations from source:
```yaml
- { name: 'authentiq',
app_id: 'YOUR_CLIENT_ID',
app_secret: 'YOUR_CLIENT_SECRET',
args: {
scope: 'aq:name email~rs address aq:push'
}
}
```
1. The `scope` is set to request the user's name, email (required and signed), and permission to send push notifications to sign in on subsequent visits.
See [OmniAuth Authentiq strategy](https://github.com/AuthentiqID/omniauth-authentiq/wiki/Scopes,-callback-url-configuration-and-responses) for more information on scopes and modifiers.
......
doc/administration/auth/crowd.md
View file @ 735e1f2c
......@@ -6,55 +6,56 @@
1. Go through the 'Add application' steps, entering the appropriate details.
The screenshot below shows an example configuration.
![Example Crowd application configuration](img/crowd_application.png)
![Example Crowd application configuration](img/crowd_application.png)
## Configure GitLab
1. On your GitLab server, open the configuration file.
**Omnibus:**
**Omnibus:**
```sh
sudo editor /etc/gitlab/gitlab.rb
```
```sh
sudo editor /etc/gitlab/gitlab.rb
```
**Source:**
**Source:**
```sh
cd /home/git/gitlab
```sh
cd /home/git/gitlab
sudo -u git -H editor config/gitlab.yml
```
sudo -u git -H editor config/gitlab.yml
```
1. See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration)
for initial settings.
1. Add the provider configuration:
**Omnibus:**
```ruby
gitlab_rails['omniauth_providers'] = [
{
"name" => "crowd",
"args" => {
"crowd_server_url" => "CROWD_SERVER_URL",
"application_name" => "YOUR_APP_NAME",
"application_password" => "YOUR_APP_PASSWORD"
}
}
]
```
**Source:**
```
- { name: 'crowd',
args: {
crowd_server_url: 'CROWD_SERVER_URL',
application_name: 'YOUR_APP_NAME',
application_password: 'YOUR_APP_PASSWORD' } }
```
**Omnibus:**
```ruby
gitlab_rails['omniauth_providers'] = [
{
"name" => "crowd",
"args" => {
"crowd_server_url" => "CROWD_SERVER_URL",
"application_name" => "YOUR_APP_NAME",
"application_password" => "YOUR_APP_PASSWORD"
}
}
]
```
**Source:**
```
- { name: 'crowd',
args: {
crowd_server_url: 'CROWD_SERVER_URL',
application_name: 'YOUR_APP_NAME',
application_password: 'YOUR_APP_PASSWORD' } }
```
1. Change `CROWD_SERVER_URL` to the URL of your Crowd server.
1. Change `YOUR_APP_NAME` to the application name from Crowd applications page.
1. Change `YOUR_APP_PASSWORD` to the application password you've set.
......@@ -77,4 +78,4 @@ could not authorize you from Crowd because invalid credentials
Please make sure the Crowd users who need to login to GitLab are authorized to [the application](#configure-a-new-crowd-application) in the step of **Authorisation**. This could be verified by try "Authentication test" for Crowd as of 2.11.
![Example Crowd application authorisation configuration](img/crowd_application_authorisation.png)
\ No newline at end of file
![Example Crowd application authorisation configuration](img/crowd_application_authorisation.png)
doc/administration/auth/google_secure_ldap.md
View file @ 735e1f2c
......@@ -66,7 +66,7 @@ values obtained during the LDAP client configuration earlier:
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
```ruby
gitlab_rails['ldap_enabled'] = true
gitlab_rails['ldap_servers'] = YAML.load <<-EOS # remember to close this block with 'EOS' below
main: # 'main' is the GitLab 'provider ID' of this LDAP server
......@@ -127,7 +127,7 @@ values obtained during the LDAP client configuration earlier:
AcZSFJQjdg5BTyvdEDhaYUKGdRw=
-----END PRIVATE KEY-----
EOS
```
```
1. Save the file and [reconfigure] GitLab for the changes to take effect.
......@@ -137,7 +137,7 @@ values obtained during the LDAP client configuration earlier:
1. Edit `config/gitlab.yml`:
```yaml
```yaml
ldap:
enabled: true
servers:
......
doc/administration/auth/jwt.md
View file @ 735e1f2c
......@@ -3,65 +3,65 @@
To enable the JWT OmniAuth provider, you must register your application with JWT.
JWT will provide you with a secret key for you to use.
1. On your GitLab server, open the configuration file.
1. On your GitLab server, open the configuration file.
For Omnibus GitLab:
For Omnibus GitLab:
```sh
sudo editor /etc/gitlab/gitlab.rb
```
```sh
sudo editor /etc/gitlab/gitlab.rb
```
For installations from source:
For installations from source:
```sh
cd /home/git/gitlab
sudo -u git -H editor config/gitlab.yml
```
```sh
cd /home/git/gitlab
sudo -u git -H editor config/gitlab.yml
```
1. See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration) for initial settings.
1. Add the provider configuration.
1. See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration) for initial settings.
1. Add the provider configuration.
For Omnibus GitLab:
For Omnibus GitLab:
```ruby
gitlab_rails['omniauth_providers'] = [
{ name: 'jwt',
args: {
secret: 'YOUR_APP_SECRET',
algorithm: 'HS256', # Supported algorithms: 'RS256', 'RS384', 'RS512', 'ES256', 'ES384', 'ES512', 'HS256', 'HS384', 'HS512'
uid_claim: 'email',
required_claims: ['name', 'email'],
info_maps: { name: 'name', email: 'email' },
auth_url: 'https://example.com/',
valid_within: 3600 # 1 hour
}
}
]
```
```ruby
gitlab_rails['omniauth_providers'] = [
{ name: 'jwt',
args: {
secret: 'YOUR_APP_SECRET',
algorithm: 'HS256', # Supported algorithms: 'RS256', 'RS384', 'RS512', 'ES256', 'ES384', 'ES512', 'HS256', 'HS384', 'HS512'
uid_claim: 'email',
required_claims: ['name', 'email'],
info_maps: { name: 'name', email: 'email' },
auth_url: 'https://example.com/',
valid_within: 3600 # 1 hour
}
}
]
```
For installation from source:
For installation from source:
```
- { name: 'jwt',
args: {
secret: 'YOUR_APP_SECRET',
algorithm: 'HS256', # Supported algorithms: 'RS256', 'RS384', 'RS512', 'ES256', 'ES384', 'ES512', 'HS256', 'HS384', 'HS512'
uid_claim: 'email',
required_claims: ['name', 'email'],
info_map: { name: 'name', email: 'email' },
auth_url: 'https://example.com/',
valid_within: 3600 # 1 hour
}
}
```
```
- { name: 'jwt',
args: {
secret: 'YOUR_APP_SECRET',
algorithm: 'HS256', # Supported algorithms: 'RS256', 'RS384', 'RS512', 'ES256', 'ES384', 'ES512', 'HS256', 'HS384', 'HS512'
uid_claim: 'email',
required_claims: ['name', 'email'],
info_map: { name: 'name', email: 'email' },
auth_url: 'https://example.com/',
valid_within: 3600 # 1 hour
}
}
```
NOTE: **Note:** For more information on each configuration option refer to
the [OmniAuth JWT usage documentation](https://github.com/mbleigh/omniauth-jwt#usage).
NOTE: **Note:** For more information on each configuration option refer to
the [OmniAuth JWT usage documentation](https://github.com/mbleigh/omniauth-jwt#usage).
1. Change `YOUR_APP_SECRET` to the client secret and set `auth_url` to your redirect URL.
1. Save the configuration file.
1. [Reconfigure][] or [restart GitLab][] for the changes to take effect if you
installed GitLab via Omnibus or from source respectively.
1. Change `YOUR_APP_SECRET` to the client secret and set `auth_url` to your redirect URL.
1. Save the configuration file.
1. [Reconfigure][] or [restart GitLab][] for the changes to take effect if you
installed GitLab via Omnibus or from source respectively.
On the sign in page there should now be a JWT icon below the regular sign in form.
Click the icon to begin the authentication process. JWT will ask the user to
......
doc/administration/auth/ldap-ee.md
View file @ 735e1f2c
......@@ -85,19 +85,19 @@ following.
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['ldap_servers'] = YAML.load <<-EOS
main:
## snip...
##
## Base where we can search for groups
##
## Ex. ou=groups,dc=gitlab,dc=example
##
##
group_base: ou=groups,dc=example,dc=com
EOS
```
```ruby
gitlab_rails['ldap_servers'] = YAML.load <<-EOS
main:
## snip...
##
## Base where we can search for groups
##
## Ex. ou=groups,dc=gitlab,dc=example
##
##
group_base: ou=groups,dc=example,dc=com
EOS
```
1. [Reconfigure GitLab][reconfigure] for the changes to take effect.
......@@ -105,14 +105,14 @@ following.
1. Edit `/home/git/gitlab/config/gitlab.yml`:
```yaml
production:
ldap:
servers:
main:
# snip...
group_base: ou=groups,dc=example,dc=com
```
```yaml
production:
ldap:
servers:
main:
# snip...
group_base: ou=groups,dc=example,dc=com
```
1. [Restart GitLab][restart] for the changes to take effect.
......@@ -140,30 +140,30 @@ group, as opposed to the full DN.
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['ldap_servers'] = YAML.load <<-EOS
main:
## snip...
##
## Base where we can search for groups
##
## Ex. ou=groups,dc=gitlab,dc=example
##
##
group_base: ou=groups,dc=example,dc=com
##
## The CN of a group containing GitLab administrators
##
## Ex. administrators
##
## Note: Not `cn=administrators` or the full DN
##
##
admin_group: my_admin_group
EOS
```
```ruby
gitlab_rails['ldap_servers'] = YAML.load <<-EOS
main:
## snip...
##
## Base where we can search for groups
##
## Ex. ou=groups,dc=gitlab,dc=example
##
##
group_base: ou=groups,dc=example,dc=com
##
## The CN of a group containing GitLab administrators
##
## Ex. administrators
##
## Note: Not `cn=administrators` or the full DN
##
##
admin_group: my_admin_group
EOS
```
1. [Reconfigure GitLab][reconfigure] for the changes to take effect.
......@@ -171,15 +171,15 @@ group, as opposed to the full DN.
1. Edit `/home/git/gitlab/config/gitlab.yml`:
```yaml
production:
ldap:
servers:
main:
# snip...
group_base: ou=groups,dc=example,dc=com
admin_group: my_admin_group
```
```yaml
production:
ldap:
servers:
main:
# snip...
group_base: ou=groups,dc=example,dc=com
admin_group: my_admin_group
```
1. [Restart GitLab][restart] for the changes to take effect.
......@@ -191,7 +191,6 @@ to lock down user abilities to invite new members to a group. When enabled follo
1. Only administrator can manage memberships of any group including access levels.
2. Users are not allowed to share project with other groups or invite members to a project created in a group.
## Adjusting LDAP user sync schedule
> Introduced in GitLab Enterprise Edition Starter.
......@@ -211,9 +210,9 @@ sync to run once every 12 hours at the top of the hour.
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['ldap_sync_worker_cron'] = "0 */12 * * *"
```
```ruby
gitlab_rails['ldap_sync_worker_cron'] = "0 */12 * * *"
```
1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
......@@ -221,11 +220,11 @@ sync to run once every 12 hours at the top of the hour.
1. Edit `config/gitlab.yaml`:
```yaml
cron_jobs:
ldap_sync_worker_cron:
"0 */12 * * *"
```
```yaml
cron_jobs:
ldap_sync_worker_cron:
"0 */12 * * *"
```
1. [Restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect.
......@@ -252,9 +251,9 @@ sync to run once every 2 hours at the top of the hour.
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['ldap_group_sync_worker_cron'] = "0 */2 * * * *"
```
```ruby
gitlab_rails['ldap_group_sync_worker_cron'] = "0 */2 * * * *"
```
1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
......@@ -262,11 +261,11 @@ sync to run once every 2 hours at the top of the hour.
1. Edit `config/gitlab.yaml`:
```yaml
cron_jobs:
ldap_group_sync_worker_cron:
"*/30 * * * *"
```
```yaml
cron_jobs:
ldap_group_sync_worker_cron:
"*/30 * * * *"
```
1. [Restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect.
......@@ -283,20 +282,20 @@ task.
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['ldap_servers'] = YAML.load <<-EOS
main:
## snip...
##
## An array of CNs of groups containing users that should be considered external
##
## Ex. ['interns', 'contractors']
##
## Note: Not `cn=interns` or the full DN
##
external_groups: ['interns', 'contractors']
EOS
```
```ruby
gitlab_rails['ldap_servers'] = YAML.load <<-EOS
main:
## snip...
##
## An array of CNs of groups containing users that should be considered external
##
## Ex. ['interns', 'contractors']
##
## Note: Not `cn=interns` or the full DN
##
external_groups: ['interns', 'contractors']
EOS
```
1. [Reconfigure GitLab][reconfigure] for the changes to take effect.
......@@ -304,14 +303,14 @@ task.
1. Edit `config/gitlab.yaml`:
```yaml
production:
ldap:
servers:
main:
# snip...
external_groups: ['interns', 'contractors']
```
```yaml
production:
ldap:
servers:
main:
# snip...
external_groups: ['interns', 'contractors']
```
1. [Restart GitLab][restart] for the changes to take effect.
......@@ -436,66 +435,71 @@ step of the sync.
1. Start a Rails console
```bash
# For Omnibus installations
sudo gitlab-rails console
```bash
# For Omnibus installations
sudo gitlab-rails console
# For installations from source
sudo -u git -H bundle exec rails console production
```
# For installations from source
sudo -u git -H bundle exec rails console production
```
1. Set the log level to debug (only for this session):
```ruby
Rails.logger.level = Logger::DEBUG
```
```ruby
Rails.logger.level = Logger::DEBUG
```
1. Choose a GitLab group to test with. This group should have an LDAP group link
already configured. If the output is `nil`, the group could not be found.
If a bunch of group attributes are output, your group was found successfully.
```ruby
group = Group.find_by(name: 'my_group')
```ruby
group = Group.find_by(name: 'my_group')
# Output
=> #<Group:0x007fe825196558 id: 1234, name: "my_group"...>
```
# Output
=> #<Group:0x007fe825196558 id: 1234, name: "my_group"...>
```
1. Run a group sync for this particular group.
```ruby
EE::Gitlab::Auth::LDAP::Sync::Group.execute_all_providers(group)
```
```ruby
EE::Gitlab::Auth::LDAP::Sync::Group.execute_all_providers(group)
```
1. Look through the output of the sync. See [example log output](#example-log-output)
below for more information about the output.
1. If you still aren't able to see why the user isn't being added, query the
LDAP group directly to see what members are listed. Still in the Rails console,
run the following query:
```ruby
adapter = Gitlab::Auth::LDAP::Adapter.new('ldapmain') # If `main` is the LDAP provider
ldap_group = EE::Gitlab::Auth::LDAP::Group.find_by_cn('group_cn_here', adapter)
```ruby
adapter = Gitlab::Auth::LDAP::Adapter.new('ldapmain') # If `main` is the LDAP provider
ldap_group = EE::Gitlab::Auth::LDAP::Group.find_by_cn('group_cn_here', adapter)
# Output
=> #<EE::Gitlab::Auth::LDAP::Group:0x007fcbdd0bb6d8
```
# Output
=> #<EE::Gitlab::Auth::LDAP::Group:0x007fcbdd0bb6d8
```
1. Query the LDAP group's member DNs and see if the user's DN is in the list.
One of the DNs here should match the 'Identifier' from the LDAP identity
checked earlier. If it doesn't, the user does not appear to be in the LDAP
group.
```ruby
ldap_group.member_dns
```ruby
ldap_group.member_dns
# Output
=> ["uid=john,ou=people,dc=example,dc=com", "uid=mary,ou=people,dc=example,dc=com"]
```
# Output
=> ["uid=john,ou=people,dc=example,dc=com", "uid=mary,ou=people,dc=example,dc=com"]
```
1. Some LDAP servers don't store members by DN. Rather, they use UIDs instead.
If you didn't see results from the last query, try querying by UIDs instead.
```ruby
ldap_group.member_uids
```ruby
ldap_group.member_uids
# Output
=> ['john','mary']
```
# Output
=> ['john','mary']
```
#### Example log output
......
doc/administration/auth/ldap.md
View file @ 735e1f2c
......@@ -398,30 +398,30 @@ The `user_filter` DN can contain special characters. For example:
- A comma:
```
OU=GitLab, Inc,DC=gitlab,DC=com
```
```
OU=GitLab, Inc,DC=gitlab,DC=com
```
- Open and close brackets:
```
OU=Gitlab (Inc),DC=gitlab,DC=com
```
```
OU=Gitlab (Inc),DC=gitlab,DC=com
```
These characters must be escaped as documented in
[RFC 4515](https://tools.ietf.org/search/rfc4515).
These characters must be escaped as documented in
[RFC 4515](https://tools.ietf.org/search/rfc4515).
- Escape commas with `\2C`. For example:
```
OU=GitLab\2C Inc,DC=gitlab,DC=com
```
```
OU=GitLab\2C Inc,DC=gitlab,DC=com
```
- Escape open and close brackets with `\28` and `\29`, respectively. For example:
```
OU=Gitlab \28Inc\29,DC=gitlab,DC=com
```
```
OU=Gitlab \28Inc\29,DC=gitlab,DC=com
```
## Enabling LDAP sign-in for existing GitLab users
......@@ -445,13 +445,13 @@ the configuration option `lowercase_usernames`. By default, this configuration o
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['ldap_servers'] = YAML.load <<-EOS
main:
# snip...
lowercase_usernames: true
EOS
```
```ruby
gitlab_rails['ldap_servers'] = YAML.load <<-EOS
main:
# snip...
lowercase_usernames: true
EOS
```
1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
......@@ -459,14 +459,14 @@ the configuration option `lowercase_usernames`. By default, this configuration o
1. Edit `config/gitlab.yaml`:
```yaml
production:
ldap:
servers:
main:
# snip...
lowercase_usernames: true
```
```yaml
production:
ldap:
servers:
main:
# snip...
lowercase_usernames: true
```
1. [Restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect.
......@@ -519,13 +519,13 @@ ldapsearch -H ldaps://$host:$port -D "$bind_dn" -y bind_dn_password.txt -b "$ba
- Run the following check command to make sure that the LDAP settings are
correct and GitLab can see your users:
```bash
# For Omnibus installations
sudo gitlab-rake gitlab:ldap:check
```bash
# For Omnibus installations
sudo gitlab-rake gitlab:ldap:check
# For installations from source
sudo -u git -H bundle exec rake gitlab:ldap:check RAILS_ENV=production
```
# For installations from source
sudo -u git -H bundle exec rake gitlab:ldap:check RAILS_ENV=production
```
### Connection Refused
......
doc/administration/auth/oidc.md
View file @ 735e1f2c
......@@ -5,76 +5,76 @@ GitLab can use [OpenID Connect](https://openid.net/specs/openid-connect-core-1_0
To enable the OpenID Connect OmniAuth provider, you must register your application with an OpenID Connect provider.
The OpenID Connect will provide you with a client details and secret for you to use.
1. On your GitLab server, open the configuration file.
For Omnibus GitLab:
```sh
sudo editor /etc/gitlab/gitlab.rb
```
For installations from source:
```sh
cd /home/git/gitlab
sudo -u git -H editor config/gitlab.yml
```
See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration) for initial settings.
1. Add the provider configuration.
For Omnibus GitLab:
```ruby
gitlab_rails['omniauth_providers'] = [
{ 'name' => 'openid_connect',
'label' => '<your_oidc_label>',
'args' => {
"name' => 'openid_connect',
'scope' => ['openid','profile'],
'response_type' => 'code',
'issuer' => '<your_oidc_url>',
'discovery' => true,
'client_auth_method' => 'query',
'uid_field' => '<uid_field>',
'client_options' => {
'identifier' => '<your_oidc_client_id>',
'secret' => '<your_oidc_client_secret>',
'redirect_uri' => '<your_gitlab_url>/users/auth/openid_connect/callback'
}
}
}
]
```
For installation from source:
```yaml
- { name: 'openid_connect',
label: '<your_oidc_label>',
args: {
name: 'openid_connect',
scope: ['openid','profile'],
response_type: 'code',
issuer: '<your_oidc_url>',
discovery: true,
client_auth_method: 'query',
uid_field: '<uid_field>',
client_options: {
identifier: '<your_oidc_client_id>',
secret: '<your_oidc_client_secret>',
redirect_uri: '<your_gitlab_url>/users/auth/openid_connect/callback'
}
}
}
```
> **Note:**
>
> - For more information on each configuration option refer to
the [OmniAuth OpenID Connect usage documentation](https://github.com/m0n9oose/omniauth_openid_connect#usage) and
the [OpenID Connect Core 1.0 specification](https://openid.net/specs/openid-connect-core-1_0.html).
1. On your GitLab server, open the configuration file.
For Omnibus GitLab:
```sh
sudo editor /etc/gitlab/gitlab.rb
```
For installations from source:
```sh
cd /home/git/gitlab
sudo -u git -H editor config/gitlab.yml
```
See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration) for initial settings.
1. Add the provider configuration.
For Omnibus GitLab:
```ruby
gitlab_rails['omniauth_providers'] = [
{ 'name' => 'openid_connect',
'label' => '<your_oidc_label>',
'args' => {
"name' => 'openid_connect',
'scope' => ['openid','profile'],
'response_type' => 'code',
'issuer' => '<your_oidc_url>',
'discovery' => true,
'client_auth_method' => 'query',
'uid_field' => '<uid_field>',
'client_options' => {
'identifier' => '<your_oidc_client_id>',
'secret' => '<your_oidc_client_secret>',
'redirect_uri' => '<your_gitlab_url>/users/auth/openid_connect/callback'
}
}
}
]
```
For installation from source:
```yaml
- { name: 'openid_connect',
label: '<your_oidc_label>',
args: {
name: 'openid_connect',
scope: ['openid','profile'],
response_type: 'code',
issuer: '<your_oidc_url>',
discovery: true,
client_auth_method: 'query',
uid_field: '<uid_field>',
client_options: {
identifier: '<your_oidc_client_id>',
secret: '<your_oidc_client_secret>',
redirect_uri: '<your_gitlab_url>/users/auth/openid_connect/callback'
}
}
}
```
> **Note:**
>
> - For more information on each configuration option refer to
the [OmniAuth OpenID Connect usage documentation](https://github.com/m0n9oose/omniauth_openid_connect#usage) and
the [OpenID Connect Core 1.0 specification](https://openid.net/specs/openid-connect-core-1_0.html).
1. For the configuration above, change the values for the provider to match your OpenID Connect client setup. Use the following as a guide:
- `<your_oidc_label>` is the label that will be displayed on the login page.
......
doc/administration/auth/okta.md
View file @ 735e1f2c
......@@ -16,7 +16,7 @@ The following documentation enables Okta as a SAML provider.
1. Next, you'll need the to fill in the SAML general config. Here's an example
image.
![Okta admin panel view](img/okta_admin_panel.png)
![Okta admin panel view](img/okta_admin_panel.png)
1. The last part of the configuration is the feedback section where you can
just say you're a customer and creating an app for internal use.
......@@ -24,7 +24,7 @@ The following documentation enables Okta as a SAML provider.
profile. Click on the SAML 2.0 config instructions button which should
look like the following:
![Okta SAML settings](img/okta_saml_settings.png)
![Okta SAML settings](img/okta_saml_settings.png)
1. On the screen that comes up take note of the
**Identity Provider Single Sign-On URL** which you'll use for the
......@@ -38,112 +38,112 @@ Now that the Okta app is configured, it's time to enable it in GitLab.
## Configure GitLab
1. On your GitLab server, open the configuration file:
**For Omnibus GitLab installations**
```sh
sudo editor /etc/gitlab/gitlab.rb
```
**For installations from source**
```sh
cd /home/git/gitlab
sudo -u git -H editor config/gitlab.yml
```
1. See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration)
for initial settings.
1. To allow your users to use Okta to sign up without having to manually create
an account first, don't forget to add the following values to your
configuration:
**For Omnibus GitLab installations**
```ruby
gitlab_rails['omniauth_allow_single_sign_on'] = ['saml']
gitlab_rails['omniauth_block_auto_created_users'] = false
```
**For installations from source**
```yaml
allow_single_sign_on: ["saml"]
block_auto_created_users: false
```
1. You can also automatically link Okta users with existing GitLab users if
their email addresses match by adding the following setting:
**For Omnibus GitLab installations**
```ruby
gitlab_rails['omniauth_auto_link_saml_user'] = true
```
**For installations from source**
```yaml
auto_link_saml_user: true
```
1. Add the provider configuration.
>**Notes:**
>
>- Change the value for `assertion_consumer_service_url` to match the HTTPS endpoint
of GitLab (append `users/auth/saml/callback` to the HTTPS URL of your GitLab
installation to generate the correct value).
>
>- To get the `idp_cert_fingerprint` fingerprint, first download the
certificate from the Okta app you registered and then run:
`openssl x509 -in okta.cert -noout -fingerprint`. Substitute `okta.cert`
with the location of your certificate.
>
>- Change the value of `idp_sso_target_url`, with the value of the
**Identity Provider Single Sign-On URL** from the step when you
configured the Okta app.
>
>- Change the value of `issuer` to the value of the **Audience Restriction** from your Okta app configuration. This will identify GitLab
to the IdP.
>
>- Leave `name_identifier_format` as-is.
**For Omnibus GitLab installations**
```ruby
gitlab_rails['omniauth_providers'] = [
{
name: 'saml',
args: {
assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml/callback',
idp_cert_fingerprint: '43:51:43:a1:b5:fc:8b:b7:0a:3a:a9:b1:0f:66:73:a8',
idp_sso_target_url: 'https://gitlab.oktapreview.com/app/gitlabdev773716_gitlabsaml_1/exk8odl81tBrjpD4B0h7/sso/saml',
issuer: 'https://gitlab.example.com',
name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient'
},
label: 'Okta' # optional label for SAML login button, defaults to "Saml"
}
]
```
**For installations from source**
```yaml
- {
name: 'saml',
args: {
assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml/callback',
idp_cert_fingerprint: '43:51:43:a1:b5:fc:8b:b7:0a:3a:a9:b1:0f:66:73:a8',
idp_sso_target_url: 'https://gitlab.oktapreview.com/app/gitlabdev773716_gitlabsaml_1/exk8odl81tBrjpD4B0h7/sso/saml',
issuer: 'https://gitlab.example.com',
name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient'
},
label: 'Okta' # optional label for SAML login button, defaults to "Saml"
}
```
1. On your GitLab server, open the configuration file:
**For Omnibus GitLab installations**
```sh
sudo editor /etc/gitlab/gitlab.rb
```
**For installations from source**
```sh
cd /home/git/gitlab
sudo -u git -H editor config/gitlab.yml
```
1. See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration)
for initial settings.
1. To allow your users to use Okta to sign up without having to manually create
an account first, don't forget to add the following values to your
configuration:
**For Omnibus GitLab installations**
```ruby
gitlab_rails['omniauth_allow_single_sign_on'] = ['saml']
gitlab_rails['omniauth_block_auto_created_users'] = false
```
**For installations from source**
```yaml
allow_single_sign_on: ["saml"]
block_auto_created_users: false
```
1. You can also automatically link Okta users with existing GitLab users if
their email addresses match by adding the following setting:
**For Omnibus GitLab installations**
```ruby
gitlab_rails['omniauth_auto_link_saml_user'] = true
```
**For installations from source**
```yaml
auto_link_saml_user: true
```
1. Add the provider configuration.
>**Notes:**
>
>- Change the value for `assertion_consumer_service_url` to match the HTTPS endpoint
of GitLab (append `users/auth/saml/callback` to the HTTPS URL of your GitLab
installation to generate the correct value).
>
>- To get the `idp_cert_fingerprint` fingerprint, first download the
certificate from the Okta app you registered and then run:
`openssl x509 -in okta.cert -noout -fingerprint`. Substitute `okta.cert`
with the location of your certificate.
>
>- Change the value of `idp_sso_target_url`, with the value of the
**Identity Provider Single Sign-On URL** from the step when you
configured the Okta app.
>
>- Change the value of `issuer` to the value of the **Audience Restriction** from your Okta app configuration. This will identify GitLab
to the IdP.
>
>- Leave `name_identifier_format` as-is.
**For Omnibus GitLab installations**
```ruby
gitlab_rails['omniauth_providers'] = [
{
name: 'saml',
args: {
assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml/callback',
idp_cert_fingerprint: '43:51:43:a1:b5:fc:8b:b7:0a:3a:a9:b1:0f:66:73:a8',
idp_sso_target_url: 'https://gitlab.oktapreview.com/app/gitlabdev773716_gitlabsaml_1/exk8odl81tBrjpD4B0h7/sso/saml',
issuer: 'https://gitlab.example.com',
name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient'
},
label: 'Okta' # optional label for SAML login button, defaults to "Saml"
}
]
```
**For installations from source**
```yaml
- {
name: 'saml',
args: {
assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml/callback',
idp_cert_fingerprint: '43:51:43:a1:b5:fc:8b:b7:0a:3a:a9:b1:0f:66:73:a8',
idp_sso_target_url: 'https://gitlab.oktapreview.com/app/gitlabdev773716_gitlabsaml_1/exk8odl81tBrjpD4B0h7/sso/saml',
issuer: 'https://gitlab.example.com',
name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient'
},
label: 'Okta' # optional label for SAML login button, defaults to "Saml"
}
```
1. [Reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) or [restart](../restart_gitlab.md#installations-from-source) GitLab for Omnibus and installations
from source respectively for the changes to take effect.
......
doc/administration/auth/smartcard.md
View file @ 735e1f2c
......@@ -56,11 +56,11 @@ attribute. As a prerequisite, you must use an LDAP server that:
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['smartcard_enabled'] = true
gitlab_rails['smartcard_ca_file'] = "/etc/ssl/certs/CA.pem"
gitlab_rails['smartcard_client_certificate_required_port'] = 3444
```
```ruby
gitlab_rails['smartcard_enabled'] = true
gitlab_rails['smartcard_ca_file'] = "/etc/ssl/certs/CA.pem"
gitlab_rails['smartcard_client_certificate_required_port'] = 3444
```
1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure)
GitLab for the changes to take effect.
......@@ -154,15 +154,15 @@ attribute. As a prerequisite, you must use an LDAP server that:
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['ldap_servers'] = YAML.load <<-EOS
main:
# snip...
# Enable smartcard authentication against the LDAP server. Valid values
# are "false", "optional", and "required".
smartcard_auth: optional
EOS
```
```ruby
gitlab_rails['ldap_servers'] = YAML.load <<-EOS
main:
# snip...
# Enable smartcard authentication against the LDAP server. Valid values
# are "false", "optional", and "required".
smartcard_auth: optional
EOS
```
1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure)
GitLab for the changes to take effect.
......@@ -171,16 +171,16 @@ attribute. As a prerequisite, you must use an LDAP server that:
1. Edit `config/gitlab.yml`:
```yaml
production:
ldap:
servers:
main:
# snip...
# Enable smartcard authentication against the LDAP server. Valid values
# are "false", "optional", and "required".
smartcard_auth: optional
```
```yaml
production:
ldap:
servers:
main:
# snip...
# Enable smartcard authentication against the LDAP server. Valid values
# are "false", "optional", and "required".
smartcard_auth: optional
```
1. Save the file and [restart](../restart_gitlab.md#installations-from-source)
GitLab for the changes to take effect.
......@@ -191,9 +191,9 @@ attribute. As a prerequisite, you must use an LDAP server that:
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['smartcard_required_for_git_access'] = true
```
```ruby
gitlab_rails['smartcard_required_for_git_access'] = true
```
1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure)
GitLab for the changes to take effect.
......@@ -202,13 +202,13 @@ attribute. As a prerequisite, you must use an LDAP server that:
1. Edit `config/gitlab.yml`:
```yaml
## Smartcard authentication settings
smartcard:
# snip...
# Browser session with smartcard sign-in is required for Git access
required_for_git_access: true
```
```yaml
## Smartcard authentication settings
smartcard:
# snip...
# Browser session with smartcard sign-in is required for Git access
required_for_git_access: true
```
1. Save the file and [restart](../restart_gitlab.md#installations-from-source)
GitLab for the changes to take effect.
doc/administration/high_availability/consul.md
View file @ 735e1f2c
......@@ -26,27 +26,27 @@ On each Consul node perform the following:
1. Edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section:
```ruby
# Disable all components except Consul
roles ['consul_role']
# START user configuration
# Replace placeholders:
#
# Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
# with the addresses gathered for CONSUL_SERVER_NODES
consul['configuration'] = {
server: true,
retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z)
}
# Disable auto migrations
gitlab_rails['auto_migrate'] = false
#
# END user configuration
```
> `consul_role` was introduced with GitLab 10.3
```ruby
# Disable all components except Consul
roles ['consul_role']
# START user configuration
# Replace placeholders:
#
# Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
# with the addresses gathered for CONSUL_SERVER_NODES
consul['configuration'] = {
server: true,
retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z)
}
# Disable auto migrations
gitlab_rails['auto_migrate'] = false
#
# END user configuration
```
> `consul_role` was introduced with GitLab 10.3
1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes
to take effect.
......@@ -77,6 +77,7 @@ check the [Troubleshooting section](#troubleshooting) before proceeding.
### Checking cluster membership
To see which nodes are part of the cluster, run the following on any member in the cluster
```
# /opt/gitlab/embedded/bin/consul members
Node Address Status Type Build Protocol DC
......@@ -112,18 +113,18 @@ You will see messages like the following in `gitlab-ctl tail consul` output if y
2017-09-25_19:53:41.74356 2017/09/25 19:53:41 [ERR] agent: failed to sync remote state: No cluster leader
```
To fix this:
1. Pick an address on each node that all of the other nodes can reach this node through.
1. Update your `/etc/gitlab/gitlab.rb`
```ruby
consul['configuration'] = {
...
bind_addr: 'IP ADDRESS'
}
```
```ruby
consul['configuration'] = {
...
bind_addr: 'IP ADDRESS'
}
```
1. Run `gitlab-ctl reconfigure`
If you still see the errors, you may have to [erase the consul database and reinitialize](#recreate-from-scratch) on the affected node.
......@@ -144,12 +145,13 @@ To fix this:
1. Pick an address on the node that all of the other nodes can reach this node through.
1. Update your `/etc/gitlab/gitlab.rb`
```ruby
consul['configuration'] = {
...
bind_addr: 'IP ADDRESS'
}
```
```ruby
consul['configuration'] = {
...
bind_addr: 'IP ADDRESS'
}
```
1. Run `gitlab-ctl reconfigure`
### Outage recovery
......@@ -157,6 +159,7 @@ To fix this:
If you lost enough server agents in the cluster to break quorum, then the cluster is considered failed, and it will not function without manual intervenetion.
#### Recreate from scratch
By default, GitLab does not store anything in the consul cluster that cannot be recreated. To erase the consul database and reinitialize
```
......@@ -168,4 +171,5 @@ By default, GitLab does not store anything in the consul cluster that cannot be
After this, the cluster should start back up, and the server agents rejoin. Shortly after that, the client agents should rejoin as well.
#### Recover a failed cluster
If you have taken advantage of consul to store other data, and want to restore the failed cluster, please follow the [Consul guide](https://www.consul.io/docs/guides/outage.html) to recover a failed cluster.
doc/administration/high_availability/database.md
View file @ 735e1f2c
......@@ -33,15 +33,15 @@ deploy the bundled PostgreSQL.
1. SSH into the PostgreSQL server.
1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab
package you want using **steps 1 and 2** from the GitLab downloads page.
- Do not complete any other steps on the download page.
- Do not complete any other steps on the download page.
1. Generate a password hash for PostgreSQL. This assumes you will use the default
username of `gitlab` (recommended). The command will request a password
and confirmation. Use the value that is output by this command in the next
step as the value of `POSTGRESQL_PASSWORD_HASH`.
```sh
sudo gitlab-ctl pg-password-md5 gitlab
```
```sh
sudo gitlab-ctl pg-password-md5 gitlab
```
1. Edit `/etc/gitlab/gitlab.rb` and add the contents below, updating placeholder
values appropriately.
......@@ -51,32 +51,32 @@ deploy the bundled PostgreSQL.
addresses of the GitLab application servers that will connect to the
database. Example: `%w(123.123.123.123/32 123.123.123.234/32)`
```ruby
# Disable all components except PostgreSQL
roles ['postgres_role']
repmgr['enable'] = false
consul['enable'] = false
prometheus['enable'] = false
alertmanager['enable'] = false
pgbouncer_exporter['enable'] = false
redis_exporter['enable'] = false
gitlab_monitor['enable'] = false
postgresql['listen_address'] = '0.0.0.0'
postgresql['port'] = 5432
# Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value
postgresql['sql_user_password'] = 'POSTGRESQL_PASSWORD_HASH'
# Replace XXX.XXX.XXX.XXX/YY with Network Address
# ????
postgresql['trust_auth_cidr_addresses'] = %w(APPLICATION_SERVER_IP_BLOCKS)
# Disable automatic database migrations
gitlab_rails['auto_migrate'] = false
```
```ruby
# Disable all components except PostgreSQL
roles ['postgres_role']
repmgr['enable'] = false
consul['enable'] = false
prometheus['enable'] = false
alertmanager['enable'] = false
pgbouncer_exporter['enable'] = false
redis_exporter['enable'] = false
gitlab_monitor['enable'] = false
postgresql['listen_address'] = '0.0.0.0'
postgresql['port'] = 5432
# Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value
postgresql['sql_user_password'] = 'POSTGRESQL_PASSWORD_HASH'
# Replace XXX.XXX.XXX.XXX/YY with Network Address
# ????
postgresql['trust_auth_cidr_addresses'] = %w(APPLICATION_SERVER_IP_BLOCKS)
# Disable automatic database migrations
gitlab_rails['auto_migrate'] = false
```
NOTE: **Note:** The role `postgres_role` was introduced with GitLab 10.3
NOTE: **Note:** The role `postgres_role` was introduced with GitLab 10.3
1. [Reconfigure GitLab] for the changes to take effect.
1. Note the PostgreSQL node's IP address or hostname, port, and
......@@ -194,9 +194,9 @@ When using default setup, minimum configuration requires:
- `CONSUL_PASSWORD_HASH`. This is a hash generated out of consul username/password pair.
Can be generated with:
```sh
sudo gitlab-ctl pg-password-md5 CONSUL_USERNAME
```
```sh
sudo gitlab-ctl pg-password-md5 CONSUL_USERNAME
```
- `CONSUL_SERVER_NODES`. The IP addresses or DNS records of the Consul server nodes.
......@@ -237,9 +237,9 @@ We will need the following password information for the application's database u
- `POSTGRESQL_PASSWORD_HASH`. This is a hash generated out of the username/password pair.
Can be generated with:
```sh
sudo gitlab-ctl pg-password-md5 POSTGRESQL_USERNAME
```
```sh
sudo gitlab-ctl pg-password-md5 POSTGRESQL_USERNAME
```
##### Pgbouncer information
......@@ -250,9 +250,9 @@ When using default setup, minimum configuration requires:
- `PGBOUNCER_PASSWORD_HASH`. This is a hash generated out of pgbouncer username/password pair.
Can be generated with:
```sh
sudo gitlab-ctl pg-password-md5 PGBOUNCER_USERNAME
```
```sh
sudo gitlab-ctl pg-password-md5 PGBOUNCER_USERNAME
```
- `PGBOUNCER_NODE`, is the IP address or a FQDN of the node running Pgbouncer.
......@@ -288,7 +288,6 @@ Make sure you install the necessary dependencies from step 1,
add GitLab package repository from step 2.
When installing the GitLab package, do not supply `EXTERNAL_URL` value.
#### Configuring the Database nodes
1. Make sure to [configure the Consul nodes](consul.md).
......@@ -296,53 +295,54 @@ When installing the GitLab package, do not supply `EXTERNAL_URL` value.
1. On the master database node, edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section:
```ruby
# Disable all components except PostgreSQL and Repmgr and Consul
roles ['postgres_role']
# PostgreSQL configuration
postgresql['listen_address'] = '0.0.0.0'
postgresql['hot_standby'] = 'on'
postgresql['wal_level'] = 'replica'
postgresql['shared_preload_libraries'] = 'repmgr_funcs'
# Disable automatic database migrations
gitlab_rails['auto_migrate'] = false
# Configure the consul agent
consul['services'] = %w(postgresql)
# START user configuration
# Please set the real values as explained in Required Information section
#
# Replace PGBOUNCER_PASSWORD_HASH with a generated md5 value
postgresql['pgbouncer_user_password'] = 'PGBOUNCER_PASSWORD_HASH'
# Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value
postgresql['sql_user_password'] = 'POSTGRESQL_PASSWORD_HASH'
# Replace X with value of number of db nodes + 1
postgresql['max_wal_senders'] = X
# Replace XXX.XXX.XXX.XXX/YY with Network Address
postgresql['trust_auth_cidr_addresses'] = %w(XXX.XXX.XXX.XXX/YY)
repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 XXX.XXX.XXX.XXX/YY)
# Replace placeholders:
#
# Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
# with the addresses gathered for CONSUL_SERVER_NODES
consul['configuration'] = {
retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z)
}
#
# END user configuration
```
> `postgres_role` was introduced with GitLab 10.3
```ruby
# Disable all components except PostgreSQL and Repmgr and Consul
roles ['postgres_role']
# PostgreSQL configuration
postgresql['listen_address'] = '0.0.0.0'
postgresql['hot_standby'] = 'on'
postgresql['wal_level'] = 'replica'
postgresql['shared_preload_libraries'] = 'repmgr_funcs'
# Disable automatic database migrations
gitlab_rails['auto_migrate'] = false
# Configure the consul agent
consul['services'] = %w(postgresql)
# START user configuration
# Please set the real values as explained in Required Information section
#
# Replace PGBOUNCER_PASSWORD_HASH with a generated md5 value
postgresql['pgbouncer_user_password'] = 'PGBOUNCER_PASSWORD_HASH'
# Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value
postgresql['sql_user_password'] = 'POSTGRESQL_PASSWORD_HASH'
# Replace X with value of number of db nodes + 1
postgresql['max_wal_senders'] = X
# Replace XXX.XXX.XXX.XXX/YY with Network Address
postgresql['trust_auth_cidr_addresses'] = %w(XXX.XXX.XXX.XXX/YY)
repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 XXX.XXX.XXX.XXX/YY)
# Replace placeholders:
#
# Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
# with the addresses gathered for CONSUL_SERVER_NODES
consul['configuration'] = {
retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z)
}
#
# END user configuration
```
> `postgres_role` was introduced with GitLab 10.3
1. On secondary nodes, add all the configuration specified above for primary node
to `/etc/gitlab/gitlab.rb`. In addition, append the following configuration
to inform gitlab-ctl that they are standby nodes initially and it need not
attempt to register them as primary node
```
# HA setting to specify if a node should attempt to be master on initialization
repmgr['master_on_initialization'] = false
......@@ -367,31 +367,31 @@ Select one node as a primary node.
1. Open a database prompt:
```sh
gitlab-psql -d gitlabhq_production
```
```sh
gitlab-psql -d gitlabhq_production
```
1. Enable the `pg_trgm` extension:
```sh
CREATE EXTENSION pg_trgm;
```
```sh
CREATE EXTENSION pg_trgm;
```
1. Exit the database prompt by typing `\q` and Enter.
1. Verify the cluster is initialized with one node:
```sh
gitlab-ctl repmgr cluster show
```
```sh
gitlab-ctl repmgr cluster show
```
The output should be similar to the following:
The output should be similar to the following:
```
Role | Name | Upstream | Connection String
----------+----------|----------|----------------------------------------
* master | HOSTNAME | | host=HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr
```
```
Role | Name | Upstream | Connection String
----------+----------|----------|----------------------------------------
* master | HOSTNAME | | host=HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr
```
1. Note down the hostname/ip in the connection string: `host=HOSTNAME`. We will
refer to the hostname in the next section as `MASTER_NODE_NAME`. If the value
......@@ -402,43 +402,43 @@ Select one node as a primary node.
1. Set up the repmgr standby:
```sh
gitlab-ctl repmgr standby setup MASTER_NODE_NAME
```
Do note that this will remove the existing data on the node. The command
has a wait time.
The output should be similar to the following:
```console
# gitlab-ctl repmgr standby setup MASTER_NODE_NAME
Doing this will delete the entire contents of /var/opt/gitlab/postgresql/data
If this is not what you want, hit Ctrl-C now to exit
To skip waiting, rerun with the -w option
Sleeping for 30 seconds
Stopping the database
Removing the data
Cloning the data
Starting the database
Registering the node with the cluster
ok: run: repmgrd: (pid 19068) 0s
```
```sh
gitlab-ctl repmgr standby setup MASTER_NODE_NAME
```
Do note that this will remove the existing data on the node. The command
has a wait time.
The output should be similar to the following:
```console
# gitlab-ctl repmgr standby setup MASTER_NODE_NAME
Doing this will delete the entire contents of /var/opt/gitlab/postgresql/data
If this is not what you want, hit Ctrl-C now to exit
To skip waiting, rerun with the -w option
Sleeping for 30 seconds
Stopping the database
Removing the data
Cloning the data
Starting the database
Registering the node with the cluster
ok: run: repmgrd: (pid 19068) 0s
```
1. Verify the node now appears in the cluster:
```sh
gitlab-ctl repmgr cluster show
```
```sh
gitlab-ctl repmgr cluster show
```
The output should be similar to the following:
The output should be similar to the following:
```
Role | Name | Upstream | Connection String
----------+---------|-----------|------------------------------------------------
* master | MASTER | | host=MASTER_NODE_NAME user=gitlab_repmgr dbname=gitlab_repmgr
standby | STANDBY | MASTER | host=STANDBY_HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr
```
```
Role | Name | Upstream | Connection String
----------+---------|-----------|------------------------------------------------
* master | MASTER | | host=MASTER_NODE_NAME user=gitlab_repmgr dbname=gitlab_repmgr
standby | STANDBY | MASTER | host=STANDBY_HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr
```
Repeat the above steps on all secondary nodes.
......@@ -487,15 +487,15 @@ attributes set, but the following need to be set.
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
# Disable PostgreSQL on the application node
postgresql['enable'] = false
```ruby
# Disable PostgreSQL on the application node
postgresql['enable'] = false
gitlab_rails['db_host'] = 'PGBOUNCER_NODE'
gitlab_rails['db_port'] = 6432
gitlab_rails['db_password'] = 'POSTGRESQL_USER_PASSWORD'
gitlab_rails['auto_migrate'] = false
```
gitlab_rails['db_host'] = 'PGBOUNCER_NODE'
gitlab_rails['db_port'] = 6432
gitlab_rails['db_password'] = 'POSTGRESQL_USER_PASSWORD'
gitlab_rails['auto_migrate'] = false
```
1. [Reconfigure GitLab] for the changes to take effect.
......@@ -655,45 +655,45 @@ After deploying the configuration follow these steps:
1. On `10.6.0.21`, our primary database
Enable the `pg_trgm` extension
Enable the `pg_trgm` extension
```sh
gitlab-psql -d gitlabhq_production
```
```sh
gitlab-psql -d gitlabhq_production
```
```
CREATE EXTENSION pg_trgm;
```
```
CREATE EXTENSION pg_trgm;
```
1. On `10.6.0.22`, our first standby database
Make this node a standby of the primary
Make this node a standby of the primary
```sh
gitlab-ctl repmgr standby setup 10.6.0.21
```
```sh
gitlab-ctl repmgr standby setup 10.6.0.21
```
1. On `10.6.0.23`, our second standby database
Make this node a standby of the primary
Make this node a standby of the primary
```sh
gitlab-ctl repmgr standby setup 10.6.0.21
```
```sh
gitlab-ctl repmgr standby setup 10.6.0.21
```
1. On `10.6.0.31`, our application server
Set gitlab-consul's pgbouncer password to `toomanysecrets`
Set gitlab-consul's pgbouncer password to `toomanysecrets`
```sh
gitlab-ctl write-pgpass --host 127.0.0.1 --database pgbouncer --user pgbouncer --hostuser gitlab-consul
```
```sh
gitlab-ctl write-pgpass --host 127.0.0.1 --database pgbouncer --user pgbouncer --hostuser gitlab-consul
```
Run database migrations
Run database migrations
```sh
gitlab-rake gitlab:db:configure
```
```sh
gitlab-rake gitlab:db:configure
```
#### Example minimal setup
......@@ -830,16 +830,16 @@ standby nodes.
1. Ensure the old master node is not still active.
1. Login to the server that should become the new master and run:
```sh
gitlab-ctl repmgr standby promote
```
```sh
gitlab-ctl repmgr standby promote
```
1. If there are any other standby servers in the cluster, have them follow
the new master server:
```sh
gitlab-ctl repmgr standby follow NEW_MASTER
```
```sh
gitlab-ctl repmgr standby follow NEW_MASTER
```
#### Restore procedure
......@@ -849,42 +849,42 @@ after it has been restored to service.
- If you want to remove the node from the cluster, on any other node in the
cluster, run:
```sh
gitlab-ctl repmgr standby unregister --node=X
```
```sh
gitlab-ctl repmgr standby unregister --node=X
```
where X is the value of node in `repmgr.conf` on the old server.
where X is the value of node in `repmgr.conf` on the old server.
To find this, you can use:
To find this, you can use:
```sh
awk -F = '$1 == "node" { print $2 }' /var/opt/gitlab/postgresql/repmgr.conf
```
```sh
awk -F = '$1 == "node" { print $2 }' /var/opt/gitlab/postgresql/repmgr.conf
```
It will output something like:
It will output something like:
```
959789412
```
```
959789412
```
Then you will use this id to unregister the node:
Then you will use this id to unregister the node:
```sh
gitlab-ctl repmgr standby unregister --node=959789412
```
```sh
gitlab-ctl repmgr standby unregister --node=959789412
```
- To add the node as a standby server:
```sh
gitlab-ctl repmgr standby follow NEW_MASTER
gitlab-ctl restart repmgrd
```
```sh
gitlab-ctl repmgr standby follow NEW_MASTER
gitlab-ctl restart repmgrd
```
CAUTION: **Warning:** When the server is brought back online, and before
you switch it to a standby node, repmgr will report that there are two masters.
If there are any clients that are still attempting to write to the old master,
this will cause a split, and the old master will need to be resynced from
scratch by performing a `gitlab-ctl repmgr standby setup NEW_MASTER`.
CAUTION: **Warning:** When the server is brought back online, and before
you switch it to a standby node, repmgr will report that there are two masters.
If there are any clients that are still attempting to write to the old master,
this will cause a split, and the old master will need to be resynced from
scratch by performing a `gitlab-ctl repmgr standby setup NEW_MASTER`.
#### Alternate configurations
......@@ -927,13 +927,13 @@ the previous section:
1. On the current master node, create a password for the `gitlab` and
`gitlab_repmgr` user:
```sh
gitlab-psql -d template1
template1=# \password gitlab_repmgr
Enter password: ****
Confirm password: ****
template1=# \password gitlab
```
```sh
gitlab-psql -d template1
template1=# \password gitlab_repmgr
Enter password: ****
Confirm password: ****
template1=# \password gitlab
```
1. On each database node:
......@@ -947,9 +947,9 @@ the previous section:
1. Create a `.pgpass` file. Enter the `gitlab_repmgr` password twice to
when asked:
```sh
gitlab-ctl write-pgpass --user gitlab_repmgr --hostuser gitlab-psql --database '*'
```
```sh
gitlab-ctl write-pgpass --user gitlab_repmgr --hostuser gitlab-psql --database '*'
```
1. On each pgbouncer node, edit `/etc/gitlab/gitlab.rb`:
1. Ensure `gitlab_rails['db_password']` is set to the plaintext password for
......@@ -977,7 +977,7 @@ If you enable Monitoring, it must be enabled on **all** database servers.
## Troubleshooting
### Consul and PostgreSQL changes not taking effect.
### Consul and PostgreSQL changes not taking effect
Due to the potential impacts, `gitlab-ctl reconfigure` only reloads Consul and PostgreSQL, it will not restart the services. However, not all changes can be activated by reloading.
......
doc/administration/high_availability/gitlab.md
View file @ 735e1f2c
......@@ -7,33 +7,33 @@
1. If necessary, install the NFS client utility packages using the following
commands:
```
# Ubuntu/Debian
apt-get install nfs-common
```
# Ubuntu/Debian
apt-get install nfs-common
# CentOS/Red Hat
yum install nfs-utils nfs-utils-lib
```
# CentOS/Red Hat
yum install nfs-utils nfs-utils-lib
```
1. Specify the necessary NFS shares. Mounts are specified in
`/etc/fstab`. The exact contents of `/etc/fstab` will depend on how you chose
to configure your NFS server. See [NFS documentation](nfs.md) for the various
options. Here is an example snippet to add to `/etc/fstab`:
```
10.1.0.1:/var/opt/gitlab/.ssh /var/opt/gitlab/.ssh nfs4 defaults,soft,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
10.1.0.1:/var/opt/gitlab/gitlab-rails/uploads /var/opt/gitlab/gitlab-rails/uploads nfs4 defaults,soft,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
10.1.0.1:/var/opt/gitlab/gitlab-rails/shared /var/opt/gitlab/gitlab-rails/shared nfs4 defaults,soft,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
10.1.0.1:/var/opt/gitlab/gitlab-ci/builds /var/opt/gitlab/gitlab-ci/builds nfs4 defaults,soft,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
10.1.0.1:/var/opt/gitlab/git-data /var/opt/gitlab/git-data nfs4 defaults,soft,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
```
```
10.1.0.1:/var/opt/gitlab/.ssh /var/opt/gitlab/.ssh nfs4 defaults,soft,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
10.1.0.1:/var/opt/gitlab/gitlab-rails/uploads /var/opt/gitlab/gitlab-rails/uploads nfs4 defaults,soft,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
10.1.0.1:/var/opt/gitlab/gitlab-rails/shared /var/opt/gitlab/gitlab-rails/shared nfs4 defaults,soft,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
10.1.0.1:/var/opt/gitlab/gitlab-ci/builds /var/opt/gitlab/gitlab-ci/builds nfs4 defaults,soft,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
10.1.0.1:/var/opt/gitlab/git-data /var/opt/gitlab/git-data nfs4 defaults,soft,rsize=1048576,wsize=1048576,noatime,nofail,lookupcache=positive 0 2
```
1. Create the shared directories. These may be different depending on your NFS
mount locations.
```
mkdir -p /var/opt/gitlab/.ssh /var/opt/gitlab/gitlab-rails/uploads /var/opt/gitlab/gitlab-rails/shared /var/opt/gitlab/gitlab-ci/builds /var/opt/gitlab/git-data
```
```
mkdir -p /var/opt/gitlab/.ssh /var/opt/gitlab/gitlab-rails/uploads /var/opt/gitlab/gitlab-rails/shared /var/opt/gitlab/gitlab-ci/builds /var/opt/gitlab/git-data
```
1. Download/install GitLab Omnibus using **steps 1 and 2** from
[GitLab downloads](https://about.gitlab.com/downloads). Do not complete other
......@@ -46,52 +46,52 @@
added NFS mounts in the default data locations. Additionally the UID and GIDs
given are just examples and you should configure with your preferred values.
```ruby
external_url 'https://gitlab.example.com'
# Prevent GitLab from starting if NFS data mounts are not available
high_availability['mountpoint'] = '/var/opt/gitlab/git-data'
# Disable components that will not be on the GitLab application server
roles ['application_role']
nginx['enable'] = true
# PostgreSQL connection details
gitlab_rails['db_adapter'] = 'postgresql'
gitlab_rails['db_encoding'] = 'unicode'
gitlab_rails['db_host'] = '10.1.0.5' # IP/hostname of database server
gitlab_rails['db_password'] = 'DB password'
# Redis connection details
gitlab_rails['redis_port'] = '6379'
gitlab_rails['redis_host'] = '10.1.0.6' # IP/hostname of Redis server
gitlab_rails['redis_password'] = 'Redis Password'
# Ensure UIDs and GIDs match between servers for permissions via NFS
user['uid'] = 9000
user['gid'] = 9000
web_server['uid'] = 9001
web_server['gid'] = 9001
registry['uid'] = 9002
registry['gid'] = 9002
```
```ruby
external_url 'https://gitlab.example.com'
# Prevent GitLab from starting if NFS data mounts are not available
high_availability['mountpoint'] = '/var/opt/gitlab/git-data'
# Disable components that will not be on the GitLab application server
roles ['application_role']
nginx['enable'] = true
# PostgreSQL connection details
gitlab_rails['db_adapter'] = 'postgresql'
gitlab_rails['db_encoding'] = 'unicode'
gitlab_rails['db_host'] = '10.1.0.5' # IP/hostname of database server
gitlab_rails['db_password'] = 'DB password'
# Redis connection details
gitlab_rails['redis_port'] = '6379'
gitlab_rails['redis_host'] = '10.1.0.6' # IP/hostname of Redis server
gitlab_rails['redis_password'] = 'Redis Password'
# Ensure UIDs and GIDs match between servers for permissions via NFS
user['uid'] = 9000
user['gid'] = 9000
web_server['uid'] = 9001
web_server['gid'] = 9001
registry['uid'] = 9002
registry['gid'] = 9002
```
1. [Enable monitoring](#enable-monitoring)
> **Note:** To maintain uniformity of links across HA clusters, the `external_url`
on the first application server as well as the additional application
servers should point to the external url that users will use to access GitLab.
In a typical HA setup, this will be the url of the load balancer which will
route traffic to all GitLab application servers in the HA cluster.
>
> **Note:** When you specify `https` in the `external_url`, as in the example
above, GitLab assumes you have SSL certificates in `/etc/gitlab/ssl/`. If
certificates are not present, Nginx will fail to start. See
[Nginx documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
for more information.
>
> **Note:** It is best to set the `uid` and `gid`s prior to the initial reconfigure
of GitLab. Omnibus will not recursively `chown` directories if set after the initial reconfigure.
> **Note:** To maintain uniformity of links across HA clusters, the `external_url`
on the first application server as well as the additional application
servers should point to the external url that users will use to access GitLab.
In a typical HA setup, this will be the url of the load balancer which will
route traffic to all GitLab application servers in the HA cluster.
>
> **Note:** When you specify `https` in the `external_url`, as in the example
above, GitLab assumes you have SSL certificates in `/etc/gitlab/ssl/`. If
certificates are not present, Nginx will fail to start. See
[Nginx documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
for more information.
>
> **Note:** It is best to set the `uid` and `gid`s prior to the initial reconfigure
of GitLab. Omnibus will not recursively `chown` directories if set after the initial reconfigure.
## First GitLab application server
......@@ -114,12 +114,12 @@ need some extra configuration.
secondary servers **prior to** running the first `reconfigure` in the steps
above.
```ruby
gitlab_shell['secret_token'] = 'fbfb19c355066a9afb030992231c4a363357f77345edd0f2e772359e5be59b02538e1fa6cae8f93f7d23355341cea2b93600dab6d6c3edcdced558fc6d739860'
gitlab_rails['otp_key_base'] = 'b719fe119132c7810908bba18315259ed12888d4f5ee5430c42a776d840a396799b0a5ef0a801348c8a357f07aa72bbd58e25a84b8f247a25c72f539c7a6c5fa'
gitlab_rails['secret_key_base'] = '6e657410d57c71b4fc3ed0d694e7842b1895a8b401d812c17fe61caf95b48a6d703cb53c112bc01ebd197a85da81b18e29682040e99b4f26594772a4a2c98c6d'
gitlab_rails['db_key_base'] = 'bf2e47b68d6cafaef1d767e628b619365becf27571e10f196f98dc85e7771042b9203199d39aff91fcb6837c8ed83f2a912b278da50999bb11a2fbc0fba52964'
```
```ruby
gitlab_shell['secret_token'] = 'fbfb19c355066a9afb030992231c4a363357f77345edd0f2e772359e5be59b02538e1fa6cae8f93f7d23355341cea2b93600dab6d6c3edcdced558fc6d739860'
gitlab_rails['otp_key_base'] = 'b719fe119132c7810908bba18315259ed12888d4f5ee5430c42a776d840a396799b0a5ef0a801348c8a357f07aa72bbd58e25a84b8f247a25c72f539c7a6c5fa'
gitlab_rails['secret_key_base'] = '6e657410d57c71b4fc3ed0d694e7842b1895a8b401d812c17fe61caf95b48a6d703cb53c112bc01ebd197a85da81b18e29682040e99b4f26594772a4a2c98c6d'
gitlab_rails['db_key_base'] = 'bf2e47b68d6cafaef1d767e628b619365becf27571e10f196f98dc85e7771042b9203199d39aff91fcb6837c8ed83f2a912b278da50999bb11a2fbc0fba52964'
```
1. Run `touch /etc/gitlab/skip-auto-reconfigure` to prevent database migrations
from running on upgrade. Only the primary GitLab application server should
......
doc/administration/high_availability/monitoring_node.md
View file @ 735e1f2c
......@@ -20,44 +20,44 @@ Omnibus:
1. Edit `/etc/gitlab/gitlab.rb` and add the contents:
```ruby
external_url 'http://gitlab.example.com'
# Enable Prometheus
prometheus['enable'] = true
prometheus['listen_address'] = '0.0.0.0:9090'
prometheus['monitor_kubernetes'] = false
# Enable Grafana
grafana['enable'] = true
grafana['admin_password'] = 'toomanysecrets'
# Enable service discovery for Prometheus
consul['enable'] = true
consul['monitoring_service_discovery'] = true
# Replace placeholders
# Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
# with the addresses of the Consul server nodes
consul['configuration'] = {
retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z),
}
# Disable all other services
gitlab_rails['auto_migrate'] = false
alertmanager['enable'] = false
gitaly['enable'] = false
gitlab_monitor['enable'] = false
gitlab_workhorse['enable'] = false
nginx['enable'] = true
postgres_exporter['enable'] = false
postgresql['enable'] = false
redis['enable'] = false
redis_exporter['enable'] = false
sidekiq['enable'] = false
unicorn['enable'] = false
node_exporter['enable'] = false
```
```ruby
external_url 'http://gitlab.example.com'
# Enable Prometheus
prometheus['enable'] = true
prometheus['listen_address'] = '0.0.0.0:9090'
prometheus['monitor_kubernetes'] = false
# Enable Grafana
grafana['enable'] = true
grafana['admin_password'] = 'toomanysecrets'
# Enable service discovery for Prometheus
consul['enable'] = true
consul['monitoring_service_discovery'] = true
# Replace placeholders
# Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
# with the addresses of the Consul server nodes
consul['configuration'] = {
retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z),
}
# Disable all other services
gitlab_rails['auto_migrate'] = false
alertmanager['enable'] = false
gitaly['enable'] = false
gitlab_monitor['enable'] = false
gitlab_workhorse['enable'] = false
nginx['enable'] = true
postgres_exporter['enable'] = false
postgresql['enable'] = false
redis['enable'] = false
redis_exporter['enable'] = false
sidekiq['enable'] = false
unicorn['enable'] = false
node_exporter['enable'] = false
```
1. Run `sudo gitlab-ctl reconfigure` to compile the configuration.
......
doc/administration/high_availability/nfs.md
View file @ 735e1f2c
......@@ -42,8 +42,8 @@ maintaining ID mapping without LDAP, in most cases you should enable numeric UID
and GIDs (which is off by default in some cases) for simplified permission
management between systems:
- [NetApp instructions](https://library.netapp.com/ecmdocs/ECMP1401220/html/GUID-24367A9F-E17B-4725-ADC1-02D86F56F78E.html)
- For non-NetApp devices, disable NFSv4 `idmapping` by performing opposite of [enable NFSv4 idmapper](https://wiki.archlinux.org/index.php/NFS#Enabling_NFSv4_idmapping)
- [NetApp instructions](https://library.netapp.com/ecmdocs/ECMP1401220/html/GUID-24367A9F-E17B-4725-ADC1-02D86F56F78E.html)
- For non-NetApp devices, disable NFSv4 `idmapping` by performing opposite of [enable NFSv4 idmapper](https://wiki.archlinux.org/index.php/NFS#Enabling_NFSv4_idmapping)
### Improving NFS performance with GitLab
......@@ -87,10 +87,10 @@ on an Linux NFS server, do the following:
1. On the NFS server, run:
```sh
echo 0 > /proc/sys/fs/leases-enable
sysctl -w fs.leases-enable=0
```
```sh
echo 0 > /proc/sys/fs/leases-enable
sysctl -w fs.leases-enable=0
```
1. Restart the NFS server process. For example, on CentOS run `service nfs restart`.
......
doc/administration/high_availability/pgbouncer.md
View file @ 735e1f2c
......@@ -105,39 +105,39 @@ It is recommended to run pgbouncer alongside the `gitlab-rails` service, or on i
1. On your database node, ensure the following is set in your `/etc/gitlab/gitlab.rb`
```ruby
postgresql['pgbouncer_user_password'] = 'PGBOUNCER_USER_PASSWORD_HASH'
postgresql['sql_user_password'] = 'SQL_USER_PASSWORD_HASH'
postgresql['listen_address'] = 'XX.XX.XX.Y' # Where XX.XX.XX.Y is the ip address on the node postgresql should listen on
postgresql['md5_auth_cidr_addresses'] = %w(AA.AA.AA.B/32) # Where AA.AA.AA.B is the IP address of the pgbouncer node
```
```ruby
postgresql['pgbouncer_user_password'] = 'PGBOUNCER_USER_PASSWORD_HASH'
postgresql['sql_user_password'] = 'SQL_USER_PASSWORD_HASH'
postgresql['listen_address'] = 'XX.XX.XX.Y' # Where XX.XX.XX.Y is the ip address on the node postgresql should listen on
postgresql['md5_auth_cidr_addresses'] = %w(AA.AA.AA.B/32) # Where AA.AA.AA.B is the IP address of the pgbouncer node
```
1. Run `gitlab-ctl reconfigure`
**Note:** If the database was already running, it will need to be restarted after reconfigure by running `gitlab-ctl restart postgresql`.
**Note:** If the database was already running, it will need to be restarted after reconfigure by running `gitlab-ctl restart postgresql`.
1. On the node you are running pgbouncer on, make sure the following is set in `/etc/gitlab/gitlab.rb`
```ruby
pgbouncer['enable'] = true
pgbouncer['databases'] = {
gitlabhq_production: {
host: 'DATABASE_HOST',
user: 'pgbouncer',
password: 'PGBOUNCER_USER_PASSWORD_HASH'
}
}
```
```ruby
pgbouncer['enable'] = true
pgbouncer['databases'] = {
gitlabhq_production: {
host: 'DATABASE_HOST',
user: 'pgbouncer',
password: 'PGBOUNCER_USER_PASSWORD_HASH'
}
}
```
1. Run `gitlab-ctl reconfigure`
1. On the node running unicorn, make sure the following is set in `/etc/gitlab/gitlab.rb`
```ruby
gitlab_rails['db_host'] = 'PGBOUNCER_HOST'
gitlab_rails['db_port'] = '6432'
gitlab_rails['db_password'] = 'SQL_USER_PASSWORD'
```
```ruby
gitlab_rails['db_host'] = 'PGBOUNCER_HOST'
gitlab_rails['db_port'] = '6432'
gitlab_rails['db_password'] = 'SQL_USER_PASSWORD'
```
1. Run `gitlab-ctl reconfigure`
......@@ -147,28 +147,28 @@ It is recommended to run pgbouncer alongside the `gitlab-rails` service, or on i
> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3786) in GitLab 12.0.
If you enable Monitoring, it must be enabled on **all** pgbouncer servers.
If you enable Monitoring, it must be enabled on **all** pgbouncer servers.
1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration:
1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration:
```ruby
# Enable service discovery for Prometheus
consul['enable'] = true
consul['monitoring_service_discovery'] = true
```ruby
# Enable service discovery for Prometheus
consul['enable'] = true
consul['monitoring_service_discovery'] = true
# Replace placeholders
# Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
# with the addresses of the Consul server nodes
consul['configuration'] = {
retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z),
}
# Replace placeholders
# Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
# with the addresses of the Consul server nodes
consul['configuration'] = {
retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z),
}
# Set the network addresses that the exporters will listen on
node_exporter['listen_address'] = '0.0.0.0:9100'
pgbouncer_exporter['listen_address'] = '0.0.0.0:9188'
```
# Set the network addresses that the exporters will listen on
node_exporter['listen_address'] = '0.0.0.0:9100'
pgbouncer_exporter['listen_address'] = '0.0.0.0:9188'
```
1. Run `sudo gitlab-ctl reconfigure` to compile the configuration.
1. Run `sudo gitlab-ctl reconfigure` to compile the configuration.
### Interacting with pgbouncer
......@@ -190,6 +190,7 @@ pgbouncer=#
The password you will be prompted for is the PGBOUNCER_USER_PASSWORD
To get some basic information about the instance, run
```shell
pgbouncer=# show databases; show clients; show servers;
name | host | port | database | force_user | pool_size | reserve_pool | pool_mode | max_connections | current_connections
......
doc/administration/high_availability/redis.md
View file @ 735e1f2c
......@@ -47,28 +47,28 @@ Omnibus:
1. Edit `/etc/gitlab/gitlab.rb` and add the contents:
```ruby
## Enable Redis
redis['enable'] = true
## Disable all other services
sidekiq['enable'] = false
gitlab_workhorse['enable'] = false
unicorn['enable'] = false
postgresql['enable'] = false
nginx['enable'] = false
prometheus['enable'] = false
alertmanager['enable'] = false
pgbouncer_exporter['enable'] = false
gitlab_monitor['enable'] = false
gitaly['enable'] = false
redis['bind'] = '0.0.0.0'
redis['port'] = '6379'
redis['password'] = 'SECRET_PASSWORD_HERE'
gitlab_rails['auto_migrate'] = false
```
```ruby
## Enable Redis
redis['enable'] = true
## Disable all other services
sidekiq['enable'] = false
gitlab_workhorse['enable'] = false
unicorn['enable'] = false
postgresql['enable'] = false
nginx['enable'] = false
prometheus['enable'] = false
alertmanager['enable'] = false
pgbouncer_exporter['enable'] = false
gitlab_monitor['enable'] = false
gitaly['enable'] = false
redis['bind'] = '0.0.0.0'
redis['port'] = '6379'
redis['password'] = 'SECRET_PASSWORD_HERE'
gitlab_rails['auto_migrate'] = false
```
1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
1. Note the Redis node's IP address or hostname, port, and
......@@ -359,37 +359,37 @@ The prerequisites for a HA Redis setup are the following:
1. SSH into the **master** Redis server.
1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab
package you want using **steps 1 and 2** from the GitLab downloads page.
- Make sure you select the correct Omnibus package, with the same version
and type (Community, Enterprise editions) of your current install.
- Do not complete any other steps on the download page.
- Make sure you select the correct Omnibus package, with the same version
and type (Community, Enterprise editions) of your current install.
- Do not complete any other steps on the download page.
1. Edit `/etc/gitlab/gitlab.rb` and add the contents:
```ruby
# Specify server role as 'redis_master_role'
roles ['redis_master_role']
```ruby
# Specify server role as 'redis_master_role'
roles ['redis_master_role']
# IP address pointing to a local IP that the other machines can reach to.
# You can also set bind to '0.0.0.0' which listen in all interfaces.
# If you really need to bind to an external accessible IP, make
# sure you add extra firewall rules to prevent unauthorized access.
redis['bind'] = '10.0.0.1'
# IP address pointing to a local IP that the other machines can reach to.
# You can also set bind to '0.0.0.0' which listen in all interfaces.
# If you really need to bind to an external accessible IP, make
# sure you add extra firewall rules to prevent unauthorized access.
redis['bind'] = '10.0.0.1'
# Define a port so Redis can listen for TCP requests which will allow other
# machines to connect to it.
redis['port'] = 6379
# Define a port so Redis can listen for TCP requests which will allow other
# machines to connect to it.
redis['port'] = 6379
# Set up password authentication for Redis (use the same password in all nodes).
redis['password'] = 'redis-password-goes-here'
```
# Set up password authentication for Redis (use the same password in all nodes).
redis['password'] = 'redis-password-goes-here'
```
1. Only the primary GitLab application server should handle migrations. To
prevent database migrations from running on upgrade, add the following
configuration to your `/etc/gitlab/gitlab.rb` file:
```
gitlab_rails['auto_migrate'] = false
```
```
gitlab_rails['auto_migrate'] = false
```
1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
......@@ -402,42 +402,42 @@ The prerequisites for a HA Redis setup are the following:
1. SSH into the **slave** Redis server.
1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab
package you want using **steps 1 and 2** from the GitLab downloads page.
- Make sure you select the correct Omnibus package, with the same version
and type (Community, Enterprise editions) of your current install.
- Do not complete any other steps on the download page.
- Make sure you select the correct Omnibus package, with the same version
and type (Community, Enterprise editions) of your current install.
- Do not complete any other steps on the download page.
1. Edit `/etc/gitlab/gitlab.rb` and add the contents:
```ruby
# Specify server role as 'redis_slave_role'
roles ['redis_slave_role']
```ruby
# Specify server role as 'redis_slave_role'
roles ['redis_slave_role']
# IP address pointing to a local IP that the other machines can reach to.
# You can also set bind to '0.0.0.0' which listen in all interfaces.
# If you really need to bind to an external accessible IP, make
# sure you add extra firewall rules to prevent unauthorized access.
redis['bind'] = '10.0.0.2'
# IP address pointing to a local IP that the other machines can reach to.
# You can also set bind to '0.0.0.0' which listen in all interfaces.
# If you really need to bind to an external accessible IP, make
# sure you add extra firewall rules to prevent unauthorized access.
redis['bind'] = '10.0.0.2'
# Define a port so Redis can listen for TCP requests which will allow other
# machines to connect to it.
redis['port'] = 6379
# Define a port so Redis can listen for TCP requests which will allow other
# machines to connect to it.
redis['port'] = 6379
# The same password for Redis authentication you set up for the master node.
redis['password'] = 'redis-password-goes-here'
# The same password for Redis authentication you set up for the master node.
redis['password'] = 'redis-password-goes-here'
# The IP of the master Redis node.
redis['master_ip'] = '10.0.0.1'
# The IP of the master Redis node.
redis['master_ip'] = '10.0.0.1'
# Port of master Redis server, uncomment to change to non default. Defaults
# to `6379`.
#redis['master_port'] = 6379
```
# Port of master Redis server, uncomment to change to non default. Defaults
# to `6379`.
#redis['master_port'] = 6379
```
1. To prevent reconfigure from running automatically on upgrade, run:
```
sudo touch /etc/gitlab/skip-auto-reconfigure
```
```
sudo touch /etc/gitlab/skip-auto-reconfigure
```
1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
1. Go through the steps again for all the other slave nodes.
......@@ -487,89 +487,89 @@ multiple machines with the Sentinel daemon.
1. **You can omit this step if the Sentinels will be hosted in the same node as
the other Redis instances.**
[Download/install](https://about.gitlab.com/downloads-ee) the
Omnibus GitLab Enterprise Edition package using **steps 1 and 2** from the
GitLab downloads page.
- Make sure you select the correct Omnibus package, with the same version
the GitLab application is running.
- Do not complete any other steps on the download page.
[Download/install](https://about.gitlab.com/downloads-ee) the
Omnibus GitLab Enterprise Edition package using **steps 1 and 2** from the
GitLab downloads page.
- Make sure you select the correct Omnibus package, with the same version
the GitLab application is running.
- Do not complete any other steps on the download page.
1. Edit `/etc/gitlab/gitlab.rb` and add the contents (if you are installing the
Sentinels in the same node as the other Redis instances, some values might
be duplicate below):
```ruby
roles ['redis_sentinel_role']
# Must be the same in every sentinel node
redis['master_name'] = 'gitlab-redis'
# The same password for Redis authentication you set up for the master node.
redis['master_password'] = 'redis-password-goes-here'
# The IP of the master Redis node.
redis['master_ip'] = '10.0.0.1'
# Define a port so Redis can listen for TCP requests which will allow other
# machines to connect to it.
redis['port'] = 6379
# Port of master Redis server, uncomment to change to non default. Defaults
# to `6379`.
#redis['master_port'] = 6379
## Configure Sentinel
sentinel['bind'] = '10.0.0.1'
# Port that Sentinel listens on, uncomment to change to non default. Defaults
# to `26379`.
# sentinel['port'] = 26379
## Quorum must reflect the amount of voting sentinels it take to start a failover.
## Value must NOT be greater then the amount of sentinels.
##
## The quorum can be used to tune Sentinel in two ways:
## 1. If a the quorum is set to a value smaller than the majority of Sentinels
## we deploy, we are basically making Sentinel more sensible to master failures,
## triggering a failover as soon as even just a minority of Sentinels is no longer
## able to talk with the master.
## 1. If a quorum is set to a value greater than the majority of Sentinels, we are
## making Sentinel able to failover only when there are a very large number (larger
## than majority) of well connected Sentinels which agree about the master being down.s
sentinel['quorum'] = 2
## Consider unresponsive server down after x amount of ms.
# sentinel['down_after_milliseconds'] = 10000
## Specifies the failover timeout in milliseconds. It is used in many ways:
##
## - The time needed to re-start a failover after a previous failover was
## already tried against the same master by a given Sentinel, is two
## times the failover timeout.
##
## - The time needed for a slave replicating to a wrong master according
## to a Sentinel current configuration, to be forced to replicate
## with the right master, is exactly the failover timeout (counting since
## the moment a Sentinel detected the misconfiguration).
##
## - The time needed to cancel a failover that is already in progress but
## did not produced any configuration change (SLAVEOF NO ONE yet not
## acknowledged by the promoted slave).
##
## - The maximum time a failover in progress waits for all the slaves to be
## reconfigured as slaves of the new master. However even after this time
## the slaves will be reconfigured by the Sentinels anyway, but not with
## the exact parallel-syncs progression as specified.
# sentinel['failover_timeout'] = 60000
```
```ruby
roles ['redis_sentinel_role']
# Must be the same in every sentinel node
redis['master_name'] = 'gitlab-redis'
# The same password for Redis authentication you set up for the master node.
redis['master_password'] = 'redis-password-goes-here'
# The IP of the master Redis node.
redis['master_ip'] = '10.0.0.1'
# Define a port so Redis can listen for TCP requests which will allow other
# machines to connect to it.
redis['port'] = 6379
# Port of master Redis server, uncomment to change to non default. Defaults
# to `6379`.
#redis['master_port'] = 6379
## Configure Sentinel
sentinel['bind'] = '10.0.0.1'
# Port that Sentinel listens on, uncomment to change to non default. Defaults
# to `26379`.
# sentinel['port'] = 26379
## Quorum must reflect the amount of voting sentinels it take to start a failover.
## Value must NOT be greater then the amount of sentinels.
##
## The quorum can be used to tune Sentinel in two ways:
## 1. If a the quorum is set to a value smaller than the majority of Sentinels
## we deploy, we are basically making Sentinel more sensible to master failures,
## triggering a failover as soon as even just a minority of Sentinels is no longer
## able to talk with the master.
## 1. If a quorum is set to a value greater than the majority of Sentinels, we are
## making Sentinel able to failover only when there are a very large number (larger
## than majority) of well connected Sentinels which agree about the master being down.s
sentinel['quorum'] = 2
## Consider unresponsive server down after x amount of ms.
# sentinel['down_after_milliseconds'] = 10000
## Specifies the failover timeout in milliseconds. It is used in many ways:
##
## - The time needed to re-start a failover after a previous failover was
## already tried against the same master by a given Sentinel, is two
## times the failover timeout.
##
## - The time needed for a slave replicating to a wrong master according
## to a Sentinel current configuration, to be forced to replicate
## with the right master, is exactly the failover timeout (counting since
## the moment a Sentinel detected the misconfiguration).
##
## - The time needed to cancel a failover that is already in progress but
## did not produced any configuration change (SLAVEOF NO ONE yet not
## acknowledged by the promoted slave).
##
## - The maximum time a failover in progress waits for all the slaves to be
## reconfigured as slaves of the new master. However even after this time
## the slaves will be reconfigured by the Sentinels anyway, but not with
## the exact parallel-syncs progression as specified.
# sentinel['failover_timeout'] = 60000
```
1. To prevent database migrations from running on upgrade, run:
```
sudo touch /etc/gitlab/skip-auto-reconfigure
```
```
sudo touch /etc/gitlab/skip-auto-reconfigure
```
Only the primary GitLab application server should handle migrations.
Only the primary GitLab application server should handle migrations.
1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
1. Go through the steps again for all the other Sentinel nodes.
......@@ -593,20 +593,20 @@ which ideally should not have Redis or Sentinels on it for a HA setup.
1. SSH into the server where the GitLab application is installed.
1. Edit `/etc/gitlab/gitlab.rb` and add/change the following lines:
```
## Must be the same in every sentinel node
redis['master_name'] = 'gitlab-redis'
## The same password for Redis authentication you set up for the master node.
redis['master_password'] = 'redis-password-goes-here'
## A list of sentinels with `host` and `port`
gitlab_rails['redis_sentinels'] = [
{'host' => '10.0.0.1', 'port' => 26379},
{'host' => '10.0.0.2', 'port' => 26379},
{'host' => '10.0.0.3', 'port' => 26379}
]
```
```ruby
## Must be the same in every sentinel node
redis['master_name'] = 'gitlab-redis'
## The same password for Redis authentication you set up for the master node.
redis['master_password'] = 'redis-password-goes-here'
## A list of sentinels with `host` and `port`
gitlab_rails['redis_sentinels'] = [
{'host' => '10.0.0.1', 'port' => 26379},
{'host' => '10.0.0.2', 'port' => 26379},
{'host' => '10.0.0.3', 'port' => 26379}
]
```
1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
......@@ -791,31 +791,34 @@ cache, queues, and shared_state. To make this work with Sentinel:
1. Set the appropriate variable in `/etc/gitlab/gitlab.rb` for each instance you are using:
```ruby
gitlab_rails['redis_cache_instance'] = REDIS_CACHE_URL
gitlab_rails['redis_queues_instance'] = REDIS_QUEUES_URL
gitlab_rails['redis_shared_state_instance'] = REDIS_SHARED_STATE_URL
```
```ruby
gitlab_rails['redis_cache_instance'] = REDIS_CACHE_URL
gitlab_rails['redis_queues_instance'] = REDIS_QUEUES_URL
gitlab_rails['redis_shared_state_instance'] = REDIS_SHARED_STATE_URL
```
**Note**: Redis URLs should be in the format: `redis://:PASSWORD@SENTINEL_MASTER_NAME`
1. PASSWORD is the plaintext password for the Redis instance
1. SENTINEL_MASTER_NAME is the Sentinel master name (e.g. `gitlab-redis-cache`)
1. PASSWORD is the plaintext password for the Redis instance
1. SENTINEL_MASTER_NAME is the Sentinel master name (e.g. `gitlab-redis-cache`)
1. Include an array of hashes with host/port combinations, such as the following:
```ruby
gitlab_rails['redis_cache_sentinels'] = [
{ host: REDIS_CACHE_SENTINEL_HOST, port: PORT1 },
{ host: REDIS_CACHE_SENTINEL_HOST2, port: PORT2 }
]
gitlab_rails['redis_queues_sentinels'] = [
{ host: REDIS_QUEUES_SENTINEL_HOST, port: PORT1 },
{ host: REDIS_QUEUES_SENTINEL_HOST2, port: PORT2 }
]
gitlab_rails['redis_shared_state_sentinels'] = [
{ host: SHARED_STATE_SENTINEL_HOST, port: PORT1 },
{ host: SHARED_STATE_SENTINEL_HOST2, port: PORT2 }
]
```
```ruby
gitlab_rails['redis_cache_sentinels'] = [
{ host: REDIS_CACHE_SENTINEL_HOST, port: PORT1 },
{ host: REDIS_CACHE_SENTINEL_HOST2, port: PORT2 }
]
gitlab_rails['redis_queues_sentinels'] = [
{ host: REDIS_QUEUES_SENTINEL_HOST, port: PORT1 },
{ host: REDIS_QUEUES_SENTINEL_HOST2, port: PORT2 }
]
gitlab_rails['redis_shared_state_sentinels'] = [
{ host: SHARED_STATE_SENTINEL_HOST, port: PORT1 },
{ host: SHARED_STATE_SENTINEL_HOST2, port: PORT2 }
]
```
1. Note that for each persistence class, GitLab will default to using the
configuration specified in `gitlab_rails['redis_sentinels']` unless
overridden by the settings above.
......@@ -879,12 +882,12 @@ in order for the HA setup to work as expected.
Before proceeding with the troubleshooting below, check your firewall rules:
- Redis machines
- Accept TCP connection in `6379`
- Connect to the other Redis machines via TCP in `6379`
- Accept TCP connection in `6379`
- Connect to the other Redis machines via TCP in `6379`
- Sentinel machines
- Accept TCP connection in `26379`
- Connect to other Sentinel machines via TCP in `26379`
- Connect to the Redis machines via TCP in `6379`
- Accept TCP connection in `26379`
- Connect to other Sentinel machines via TCP in `26379`
- Connect to the Redis machines via TCP in `6379`
### Troubleshooting Redis replication
......@@ -952,38 +955,38 @@ To make sure your configuration is correct:
1. SSH into your GitLab application server
1. Enter the Rails console:
```
# For Omnibus installations
sudo gitlab-rails console
```
# For Omnibus installations
sudo gitlab-rails console
# For source installations
sudo -u git rails console production
```
# For source installations
sudo -u git rails console production
```
1. Run in the console:
```ruby
redis = Redis.new(Gitlab::Redis::SharedState.params)
redis.info
```
```ruby
redis = Redis.new(Gitlab::Redis::SharedState.params)
redis.info
```
Keep this screen open and try to simulate a failover below.
Keep this screen open and try to simulate a failover below.
1. To simulate a failover on master Redis, SSH into the Redis server and run:
```bash
# port must match your master redis port, and the sleep time must be a few seconds bigger than defined one
redis-cli -h localhost -p 6379 DEBUG sleep 20
```
```bash
# port must match your master redis port, and the sleep time must be a few seconds bigger than defined one
redis-cli -h localhost -p 6379 DEBUG sleep 20
```
1. Then back in the Rails console from the first step, run:
```
redis.info
```
```
redis.info
```
You should see a different port after a few seconds delay
(the failover/reconnect time).
You should see a different port after a few seconds delay
(the failover/reconnect time).
## Changelog
......
doc/administration/high_availability/redis_source.md
View file @ 735e1f2c
......@@ -49,22 +49,22 @@ Assuming that the Redis master instance IP is `10.0.0.1`:
1. [Install Redis](../../install/installation.md#7-redis).
1. Edit `/etc/redis/redis.conf`:
```conf
## Define a `bind` address pointing to a local IP that your other machines
## can reach you. If you really need to bind to an external accessible IP, make
## sure you add extra firewall rules to prevent unauthorized access:
bind 10.0.0.1
## Define a `port` to force redis to listen on TCP so other machines can
## connect to it (default port is `6379`).
port 6379
## Set up password authentication (use the same password in all nodes).
## The password should be defined equal for both `requirepass` and `masterauth`
## when setting up Redis to use with Sentinel.
requirepass redis-password-goes-here
masterauth redis-password-goes-here
```
```conf
## Define a `bind` address pointing to a local IP that your other machines
## can reach you. If you really need to bind to an external accessible IP, make
## sure you add extra firewall rules to prevent unauthorized access:
bind 10.0.0.1
## Define a `port` to force redis to listen on TCP so other machines can
## connect to it (default port is `6379`).
port 6379
## Set up password authentication (use the same password in all nodes).
## The password should be defined equal for both `requirepass` and `masterauth`
## when setting up Redis to use with Sentinel.
requirepass redis-password-goes-here
masterauth redis-password-goes-here
```
1. Restart the Redis service for the changes to take effect.
......@@ -75,25 +75,25 @@ Assuming that the Redis slave instance IP is `10.0.0.2`:
1. [Install Redis](../../install/installation.md#7-redis).
1. Edit `/etc/redis/redis.conf`:
```conf
## Define a `bind` address pointing to a local IP that your other machines
## can reach you. If you really need to bind to an external accessible IP, make
## sure you add extra firewall rules to prevent unauthorized access:
bind 10.0.0.2
```conf
## Define a `bind` address pointing to a local IP that your other machines
## can reach you. If you really need to bind to an external accessible IP, make
## sure you add extra firewall rules to prevent unauthorized access:
bind 10.0.0.2
## Define a `port` to force redis to listen on TCP so other machines can
## connect to it (default port is `6379`).
port 6379
## Define a `port` to force redis to listen on TCP so other machines can
## connect to it (default port is `6379`).
port 6379
## Set up password authentication (use the same password in all nodes).
## The password should be defined equal for both `requirepass` and `masterauth`
## when setting up Redis to use with Sentinel.
requirepass redis-password-goes-here
masterauth redis-password-goes-here
## Set up password authentication (use the same password in all nodes).
## The password should be defined equal for both `requirepass` and `masterauth`
## when setting up Redis to use with Sentinel.
requirepass redis-password-goes-here
masterauth redis-password-goes-here
## Define `slaveof` pointing to the Redis master instance with IP and port.
slaveof 10.0.0.1 6379
```
## Define `slaveof` pointing to the Redis master instance with IP and port.
slaveof 10.0.0.1 6379
```
1. Restart the Redis service for the changes to take effect.
1. Go through the steps again for all the other slave nodes.
......@@ -110,56 +110,57 @@ master with IP `10.0.0.1` (some settings might overlap with the master):
1. [Install Redis Sentinel](https://redis.io/topics/sentinel)
1. Edit `/etc/redis/sentinel.conf`:
```conf
## Define a `bind` address pointing to a local IP that your other machines
## can reach you. If you really need to bind to an external accessible IP, make
## sure you add extra firewall rules to prevent unauthorized access:
bind 10.0.0.1
## Define a `port` to force Sentinel to listen on TCP so other machines can
## connect to it (default port is `6379`).
port 26379
## Set up password authentication (use the same password in all nodes).
## The password should be defined equal for both `requirepass` and `masterauth`
## when setting up Redis to use with Sentinel.
requirepass redis-password-goes-here
masterauth redis-password-goes-here
## Define with `sentinel auth-pass` the same shared password you have
## defined for both Redis master and slaves instances.
sentinel auth-pass gitlab-redis redis-password-goes-here
## Define with `sentinel monitor` the IP and port of the Redis
## master node, and the quorum required to start a failover.
sentinel monitor gitlab-redis 10.0.0.1 6379 2
## Define with `sentinel down-after-milliseconds` the time in `ms`
## that an unresponsive server will be considered down.
sentinel down-after-milliseconds gitlab-redis 10000
## Define a value for `sentinel failover_timeout` in `ms`. This has multiple
## meanings:
##
## * The time needed to re-start a failover after a previous failover was
## already tried against the same master by a given Sentinel, is two
## times the failover timeout.
##
## * The time needed for a slave replicating to a wrong master according
## to a Sentinel current configuration, to be forced to replicate
## with the right master, is exactly the failover timeout (counting since
## the moment a Sentinel detected the misconfiguration).
##
## * The time needed to cancel a failover that is already in progress but
## did not produced any configuration change (SLAVEOF NO ONE yet not
## acknowledged by the promoted slave).
##
## * The maximum time a failover in progress waits for all the slaves to be
## reconfigured as slaves of the new master. However even after this time
## the slaves will be reconfigured by the Sentinels anyway, but not with
## the exact parallel-syncs progression as specified.
sentinel failover_timeout 30000
```
```conf
## Define a `bind` address pointing to a local IP that your other machines
## can reach you. If you really need to bind to an external accessible IP, make
## sure you add extra firewall rules to prevent unauthorized access:
bind 10.0.0.1
## Define a `port` to force Sentinel to listen on TCP so other machines can
## connect to it (default port is `6379`).
port 26379
## Set up password authentication (use the same password in all nodes).
## The password should be defined equal for both `requirepass` and `masterauth`
## when setting up Redis to use with Sentinel.
requirepass redis-password-goes-here
masterauth redis-password-goes-here
## Define with `sentinel auth-pass` the same shared password you have
## defined for both Redis master and slaves instances.
sentinel auth-pass gitlab-redis redis-password-goes-here
## Define with `sentinel monitor` the IP and port of the Redis
## master node, and the quorum required to start a failover.
sentinel monitor gitlab-redis 10.0.0.1 6379 2
## Define with `sentinel down-after-milliseconds` the time in `ms`
## that an unresponsive server will be considered down.
sentinel down-after-milliseconds gitlab-redis 10000
## Define a value for `sentinel failover_timeout` in `ms`. This has multiple
## meanings:
##
## * The time needed to re-start a failover after a previous failover was
## already tried against the same master by a given Sentinel, is two
## times the failover timeout.
##
## * The time needed for a slave replicating to a wrong master according
## to a Sentinel current configuration, to be forced to replicate
## with the right master, is exactly the failover timeout (counting since
## the moment a Sentinel detected the misconfiguration).
##
## * The time needed to cancel a failover that is already in progress but
## did not produced any configuration change (SLAVEOF NO ONE yet not
## acknowledged by the promoted slave).
##
## * The maximum time a failover in progress waits for all the slaves to be
## reconfigured as slaves of the new master. However even after this time
## the slaves will be reconfigured by the Sentinels anyway, but not with
## the exact parallel-syncs progression as specified.
sentinel failover_timeout 30000
```
1. Restart the Redis service for the changes to take effect.
1. Go through the steps again for all the other Sentinel nodes.
......@@ -180,21 +181,21 @@ setup:
[resque.yml.example][resque], and uncomment the Sentinel lines, pointing to
the correct server credentials:
```yaml
# resque.yaml
production:
url: redis://:redi-password-goes-here@gitlab-redis/
sentinels:
-
host: 10.0.0.1
port: 26379 # point to sentinel, not to redis port
-
host: 10.0.0.2
port: 26379 # point to sentinel, not to redis port
-
host: 10.0.0.3
port: 26379 # point to sentinel, not to redis port
```
```yaml
# resque.yaml
production:
url: redis://:redi-password-goes-here@gitlab-redis/
sentinels:
-
host: 10.0.0.1
port: 26379 # point to sentinel, not to redis port
-
host: 10.0.0.2
port: 26379 # point to sentinel, not to redis port
-
host: 10.0.0.3
port: 26379 # point to sentinel, not to redis port
```
1. [Restart GitLab][restart] for the changes to take effect.
......@@ -232,23 +233,23 @@ or a failover promotes a different **Master** node.
1. In `/etc/redis/redis.conf`:
```conf
bind 10.0.0.1
port 6379
requirepass redis-password-goes-here
masterauth redis-password-goes-here
```
```conf
bind 10.0.0.1
port 6379
requirepass redis-password-goes-here
masterauth redis-password-goes-here
```
1. In `/etc/redis/sentinel.conf`:
```conf
bind 10.0.0.1
port 26379
sentinel auth-pass gitlab-redis redis-password-goes-here
sentinel monitor gitlab-redis 10.0.0.1 6379 2
sentinel down-after-milliseconds gitlab-redis 10000
sentinel failover_timeout 30000
```
```conf
bind 10.0.0.1
port 26379
sentinel auth-pass gitlab-redis redis-password-goes-here
sentinel monitor gitlab-redis 10.0.0.1 6379 2
sentinel down-after-milliseconds gitlab-redis 10000
sentinel failover_timeout 30000
```
1. Restart the Redis service for the changes to take effect.
......@@ -256,24 +257,24 @@ or a failover promotes a different **Master** node.
1. In `/etc/redis/redis.conf`:
```conf
bind 10.0.0.2
port 6379
requirepass redis-password-goes-here
masterauth redis-password-goes-here
slaveof 10.0.0.1 6379
```
```conf
bind 10.0.0.2
port 6379
requirepass redis-password-goes-here
masterauth redis-password-goes-here
slaveof 10.0.0.1 6379
```
1. In `/etc/redis/sentinel.conf`:
```conf
bind 10.0.0.2
port 26379
sentinel auth-pass gitlab-redis redis-password-goes-here
sentinel monitor gitlab-redis 10.0.0.1 6379 2
sentinel down-after-milliseconds gitlab-redis 10000
sentinel failover_timeout 30000
```
```conf
bind 10.0.0.2
port 26379
sentinel auth-pass gitlab-redis redis-password-goes-here
sentinel monitor gitlab-redis 10.0.0.1 6379 2
sentinel down-after-milliseconds gitlab-redis 10000
sentinel failover_timeout 30000
```
1. Restart the Redis service for the changes to take effect.
......@@ -281,24 +282,24 @@ or a failover promotes a different **Master** node.
1. In `/etc/redis/redis.conf`:
```conf
bind 10.0.0.3
port 6379
requirepass redis-password-goes-here
masterauth redis-password-goes-here
slaveof 10.0.0.1 6379
```
```conf
bind 10.0.0.3
port 6379
requirepass redis-password-goes-here
masterauth redis-password-goes-here
slaveof 10.0.0.1 6379
```
1. In `/etc/redis/sentinel.conf`:
```conf
bind 10.0.0.3
port 26379
sentinel auth-pass gitlab-redis redis-password-goes-here
sentinel monitor gitlab-redis 10.0.0.1 6379 2
sentinel down-after-milliseconds gitlab-redis 10000
sentinel failover_timeout 30000
```
```conf
bind 10.0.0.3
port 26379
sentinel auth-pass gitlab-redis redis-password-goes-here
sentinel monitor gitlab-redis 10.0.0.1 6379 2
sentinel down-after-milliseconds gitlab-redis 10000
sentinel failover_timeout 30000
```
1. Restart the Redis service for the changes to take effect.
......@@ -306,20 +307,20 @@ or a failover promotes a different **Master** node.
1. Edit `/home/git/gitlab/config/resque.yml`:
```yaml
production:
url: redis://:redi-password-goes-here@gitlab-redis/
sentinels:
-
host: 10.0.0.1
port: 26379 # point to sentinel, not to redis port
-
host: 10.0.0.2
port: 26379 # point to sentinel, not to redis port
-
host: 10.0.0.3
port: 26379 # point to sentinel, not to redis port
```
```yaml
production:
url: redis://:redi-password-goes-here@gitlab-redis/
sentinels:
-
host: 10.0.0.1
port: 26379 # point to sentinel, not to redis port
-
host: 10.0.0.2
port: 26379 # point to sentinel, not to redis port
-
host: 10.0.0.3
port: 26379 # point to sentinel, not to redis port
```
1. [Restart GitLab][restart] for the changes to take effect.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7