Commit 34bda361 authored by Wei-Meng Lee's avatar Wei-Meng Lee

Move PostgreSQL HA documentation

Move PostgreSQL HA documentation into a new PostgreSQL section
parent e46ff054
# Configure GitLab using an external PostgreSQL service
If you're hosting GitLab on a cloud provider, you can optionally use a
managed service for PostgreSQL. For example, AWS offers a managed Relational
Database Service (RDS) that runs PostgreSQL.
Alternatively, you may opt to manage your own PostgreSQL instance or cluster
separate from the Omnibus GitLab package.
If you use a cloud-managed service, or provide your own PostgreSQL instance:
1. Set up PostgreSQL according to the
[database requirements document](../install/requirements.md#database).
1. Set up a `gitlab` username with a password of your choice. The `gitlab` user
needs privileges to create the `gitlabhq_production` database.
1. If you are using a cloud-managed service, you may need to grant additional
roles to your `gitlab` user:
- Amazon RDS requires the [`rds_superuser`](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.html#Appendix.PostgreSQL.CommonDBATasks.Roles) role.
- Azure Database for PostgreSQL requires the [`azure_pg_admin`](https://docs.microsoft.com/en-us/azure/postgresql/howto-create-users#how-to-create-additional-admin-users-in-azure-database-for-postgresql) role.
---
redirect_to: 'postgresql/external.md'
---
1. Configure the GitLab application servers with the appropriate connection details
for your external PostgreSQL service in your `/etc/gitlab/gitlab.rb` file:
```ruby
# Disable the bundled Omnibus provided PostgreSQL
postgresql['enable'] = false
# PostgreSQL connection details
gitlab_rails['db_adapter'] = 'postgresql'
gitlab_rails['db_encoding'] = 'unicode'
gitlab_rails['db_host'] = '10.1.0.5' # IP/hostname of database server
gitlab_rails['db_password'] = 'DB password'
```
For more information on GitLab HA setups, refer to [configuring GitLab for HA](high_availability/gitlab.md).
1. Reconfigure for the changes to take effect:
# Configure GitLab using an external PostgreSQL service
```shell
sudo gitlab-ctl reconfigure
```
This content has been moved to a [new location](postgresql/external.md).
......@@ -468,7 +468,7 @@ The replication process is now complete.
PostgreSQL connections. We recommend using PgBouncer if you use GitLab in a
high-availability configuration with a cluster of nodes supporting a Geo
**primary** node and another cluster of nodes supporting a Geo **secondary** node. For more
information, see [High Availability with Omnibus GitLab](../../high_availability/database.md#high-availability-with-omnibus-gitlab-premium-only).
information, see [High Availability with Omnibus GitLab](../../postgresql/replication_and_failover.md).
For a Geo **secondary** node to work properly with PgBouncer in front of the database,
it will need a separate read-only user to make [PostgreSQL FDW queries](https://www.postgresql.org/docs/11/postgres-fdw.html)
......
......@@ -45,7 +45,7 @@ Because of the additional complexity involved in setting up this configuration
for PostgreSQL and Redis, it is not covered by this Geo multi-server documentation.
For more information about setting up a multi-server PostgreSQL cluster and Redis cluster using the omnibus package see the multi-server documentation for
[PostgreSQL](../../high_availability/database.md) and
[PostgreSQL](../../postgresql/replication_and_failover.md) and
[Redis](../../high_availability/redis.md), respectively.
NOTE: **Note:**
......@@ -94,7 +94,7 @@ NOTE: **Note:** PostgreSQL and Redis should have already been disabled on the
application servers, and connections from the application servers to those
services on the backend servers configured, during normal GitLab multi-server set up. See
multi-server configuration documentation for
[PostgreSQL](../../high_availability/database.md#configuring-the-application-nodes)
[PostgreSQL](../../postgresql/replication_and_failover.md#configuring-the-application-nodes)
and [Redis](../../high_availability/redis.md#example-configuration-for-the-gitlab-application).
### Step 2: Configure the **primary** database
......
......@@ -7,7 +7,7 @@ type: reference
As part of its High Availability stack, GitLab Premium includes a bundled version of [Consul](https://www.consul.io/) that can be managed through `/etc/gitlab/gitlab.rb`. Consul is a service networking solution. When it comes to [GitLab Architecture](../../development/architecture.md), Consul utilization is supported for configuring:
1. [Monitoring in Scaled and Highly Available environments](monitoring_node.md)
1. [PostgreSQL High Availability with Omnibus](database.md#high-availability-with-omnibus-gitlab-premium-only)
1. [PostgreSQL High Availability with Omnibus](../postgresql/replication_and_failover.md)
A Consul cluster consists of multiple server agents, as well as client agents that run on other nodes which need to talk to the Consul cluster.
......@@ -27,7 +27,7 @@ When installing the GitLab package, do not supply `EXTERNAL_URL` value.
On each Consul node perform the following:
1. Make sure you collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step, before executing the next step.
1. Make sure you collect [`CONSUL_SERVER_NODES`](../postgresql/replication_and_failover.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step, before executing the next step.
1. Edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section:
......
......@@ -24,7 +24,7 @@ Continue configuration of other components by going back to the
> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/3786) in GitLab 12.0.
1. Make sure to collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z`
1. Make sure to collect [`CONSUL_SERVER_NODES`](../postgresql/replication_and_failover.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z`
1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration:
......
......@@ -142,7 +142,7 @@ migrations performed.
If you enable Monitoring, it must be enabled on **all** GitLab servers.
1. Make sure to collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z`
1. Make sure to collect [`CONSUL_SERVER_NODES`](../postgresql/replication_and_failover.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z`
1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration:
......@@ -203,7 +203,7 @@ for more details.
Read more on high-availability configuration:
1. [Configure the database](database.md)
1. [Configure the database](../postgresql/replication_and_failover.md)
1. [Configure Redis](redis.md)
1. [Configure NFS](nfs.md)
1. [Configure the load balancers](load_balancer.md)
......@@ -117,7 +117,7 @@ It is strongly recommend that multi-node deployments configure load balancers to
Read more on high-availability configuration:
1. [Configure the database](database.md)
1. [Configure the database](../postgresql/replication_and_failover.md)
1. [Configure Redis](redis.md)
1. [Configure NFS](nfs.md)
1. [Configure the GitLab application servers](gitlab.md)
......
......@@ -22,7 +22,7 @@ Omnibus:
package you want using **steps 1 and 2** from the GitLab downloads page.
- Do not complete any other steps on the download page.
1. Make sure to collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z`
1. Make sure to collect [`CONSUL_SERVER_NODES`](../postgresql/replication_and_failover.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z`
1. Edit `/etc/gitlab/gitlab.rb` and add the contents:
......
......@@ -146,7 +146,7 @@ Additionally, this configuration is specifically warned against in the
>to the NFS server can cause data corruption problems.
For supported database architecture, please see our documentation on
[Configuring a Database for GitLab HA](database.md).
[Configuring a Database for GitLab HA](../postgresql/replication_and_failover.md).
## NFS Client mount options
......@@ -284,7 +284,7 @@ are empty before attempting a restore. Read more about the
Read more on high-availability configuration:
1. [Configure the database](database.md)
1. [Configure the database](../postgresql/replication_and_failover.md)
1. [Configure Redis](redis.md)
1. [Configure the GitLab application servers](gitlab.md)
1. [Configure the load balancers](load_balancer.md)
......
......@@ -12,124 +12,7 @@ In a HA setup, it's recommended to run a PgBouncer node separately for each data
### Running PgBouncer as part of an HA GitLab installation
1. Make sure you collect [`CONSUL_SERVER_NODES`](database.md#consul-information), [`CONSUL_PASSWORD_HASH`](database.md#consul-information), and [`PGBOUNCER_PASSWORD_HASH`](database.md#pgbouncer-information) before executing the next step.
1. One each node, edit the `/etc/gitlab/gitlab.rb` config file and replace values noted in the `# START user configuration` section as below:
```ruby
# Disable all components except PgBouncer and Consul agent
roles ['pgbouncer_role']
# Configure PgBouncer
pgbouncer['admin_users'] = %w(pgbouncer gitlab-consul)
# Configure Consul agent
consul['watchers'] = %w(postgresql)
# START user configuration
# Please set the real values as explained in Required Information section
# Replace CONSUL_PASSWORD_HASH with with a generated md5 value
# Replace PGBOUNCER_PASSWORD_HASH with with a generated md5 value
pgbouncer['users'] = {
'gitlab-consul': {
password: 'CONSUL_PASSWORD_HASH'
},
'pgbouncer': {
password: 'PGBOUNCER_PASSWORD_HASH'
}
}
# Replace placeholders:
#
# Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
# with the addresses gathered for CONSUL_SERVER_NODES
consul['configuration'] = {
retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z)
}
#
# END user configuration
```
NOTE: **Note:**
`pgbouncer_role` was introduced with GitLab 10.3.
1. Run `gitlab-ctl reconfigure`
1. Create a `.pgpass` file so Consul is able to
reload PgBouncer. Enter the `PGBOUNCER_PASSWORD` twice when asked:
```shell
gitlab-ctl write-pgpass --host 127.0.0.1 --database pgbouncer --user pgbouncer --hostuser gitlab-consul
```
#### PgBouncer Checkpoint
1. Ensure each node is talking to the current master:
```shell
gitlab-ctl pgb-console # You will be prompted for PGBOUNCER_PASSWORD
```
If there is an error `psql: ERROR: Auth failed` after typing in the
password, ensure you previously generated the MD5 password hashes with the correct
format. The correct format is to concatenate the password and the username:
`PASSWORDUSERNAME`. For example, `Sup3rS3cr3tpgbouncer` would be the text
needed to generate an MD5 password hash for the `pgbouncer` user.
1. Once the console prompt is available, run the following queries:
```shell
show databases ; show clients ;
```
The output should be similar to the following:
```plaintext
name | host | port | database | force_user | pool_size | reserve_pool | pool_mode | max_connections | current_connections
---------------------+-------------+------+---------------------+------------+-----------+--------------+-----------+-----------------+---------------------
gitlabhq_production | MASTER_HOST | 5432 | gitlabhq_production | | 20 | 0 | | 0 | 0
pgbouncer | | 6432 | pgbouncer | pgbouncer | 2 | 0 | statement | 0 | 0
(2 rows)
type | user | database | state | addr | port | local_addr | local_port | connect_time | request_time | ptr | link | remote_pid | tls
------+-----------+---------------------+---------+----------------+-------+------------+------------+---------------------+---------------------+-----------+------+------------+-----
C | pgbouncer | pgbouncer | active | 127.0.0.1 | 56846 | 127.0.0.1 | 6432 | 2017-08-21 18:09:59 | 2017-08-21 18:10:48 | 0x22b3880 | | 0 |
(2 rows)
```
#### Configure the internal load balancer
If you're running more than one PgBouncer node as recommended, then at this time you'll need to set up a TCP internal load balancer to serve each correctly. This can be done with any reputable TCP load balancer.
As an example here's how you could do it with [HAProxy](https://www.haproxy.org/):
```plaintext
global
log /dev/log local0
log localhost local1 notice
log stdout format raw local0
defaults
log global
default-server inter 10s fall 3 rise 2
balance leastconn
frontend internal-pgbouncer-tcp-in
bind *:6432
mode tcp
option tcplog
default_backend pgbouncer
backend pgbouncer
mode tcp
option tcp-check
server pgbouncer1 <ip>:6432 check
server pgbouncer2 <ip>:6432 check
server pgbouncer3 <ip>:6432 check
```
Refer to your preferred Load Balancer's documentation for further guidance.
This content has been moved to a [new location](../postgresql/replication_and_failover.md#configuring-the-pgbouncer-node).
### Running PgBouncer as part of a non-HA GitLab installation
......
......@@ -750,7 +750,7 @@ gitlab_rails['redis_sentinels'] = [
If you enable Monitoring, it must be enabled on **all** Redis servers.
1. Make sure to collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z`
1. Make sure to collect [`CONSUL_SERVER_NODES`](../postgresql/replication_and_failover.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z`
1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration:
......@@ -1002,7 +1002,7 @@ Changes to Redis HA over time.
Read more on High Availability:
1. [High Availability Overview](README.md)
1. [Configure the database](database.md)
1. [Configure the database](../postgresql/replication_and_failover.md)
1. [Configure NFS](nfs.md)
1. [Configure the GitLab application servers](gitlab.md)
1. [Configure the load balancers](load_balancer.md)
......@@ -21,9 +21,9 @@ To enable the PostgreSQL Server Exporter:
```
NOTE: **Note:**
If PostgreSQL Server Exporter is configured on a separate node, make sure the local
address is [listed in `trust_auth_cidr_addresses`](../../high_availability/database.md#network-information), or the
exporter can't connect to the database.
If PostgreSQL Server Exporter is configured on a separate node, make sure that the local
address is [listed in `trust_auth_cidr_addresses`](../../postgresql/replication_and_failover.md#network-information) or the
exporter will not be able to connect to the database.
1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to
take effect.
......
# Configure GitLab using an external PostgreSQL service
If you're hosting GitLab on a cloud provider, you can optionally use a
managed service for PostgreSQL. For example, AWS offers a managed Relational
Database Service (RDS) that runs PostgreSQL.
Alternatively, you may opt to manage your own PostgreSQL instance or cluster
separate from the Omnibus GitLab package.
If you use a cloud-managed service, or provide your own PostgreSQL instance:
1. Set up PostgreSQL according to the
[database requirements document](../../install/requirements.md#database).
1. Set up a `gitlab` username with a password of your choice. The `gitlab` user
needs privileges to create the `gitlabhq_production` database.
1. If you are using a cloud-managed service, you may need to grant additional
roles to your `gitlab` user:
- Amazon RDS requires the [`rds_superuser`](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.html#Appendix.PostgreSQL.CommonDBATasks.Roles) role.
- Azure Database for PostgreSQL requires the [`azure_pg_admin`](https://docs.microsoft.com/en-us/azure/postgresql/howto-create-users#how-to-create-additional-admin-users-in-azure-database-for-postgresql) role.
1. Configure the GitLab application servers with the appropriate connection details
for your external PostgreSQL service in your `/etc/gitlab/gitlab.rb` file:
```ruby
# Disable the bundled Omnibus provided PostgreSQL
postgresql['enable'] = false
# PostgreSQL connection details
gitlab_rails['db_adapter'] = 'postgresql'
gitlab_rails['db_encoding'] = 'unicode'
gitlab_rails['db_host'] = '10.1.0.5' # IP/hostname of database server
gitlab_rails['db_password'] = 'DB password'
```
For more information on GitLab HA setups, refer to [configuring GitLab for HA](../high_availability/gitlab.md).
1. Reconfigure for the changes to take effect:
```shell
sudo gitlab-ctl reconfigure
```
This diff is collapsed.
# Standalone PostgreSQL using Omnibus GitLab **(CORE ONLY)**
1. SSH into the PostgreSQL server.
1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab
package you want using **steps 1 and 2** from the GitLab downloads page.
- Do not complete any other steps on the download page.
1. Generate a password hash for PostgreSQL. This assumes you will use the default
username of `gitlab` (recommended). The command will request a password
and confirmation. Use the value that is output by this command in the next
step as the value of `POSTGRESQL_PASSWORD_HASH`.
```shell
sudo gitlab-ctl pg-password-md5 gitlab
```
1. Edit `/etc/gitlab/gitlab.rb` and add the contents below, updating placeholder
values appropriately.
- `POSTGRESQL_PASSWORD_HASH` - The value output from the previous step
- `APPLICATION_SERVER_IP_BLOCKS` - A space delimited list of IP subnets or IP
addresses of the GitLab application servers that will connect to the
database. Example: `%w(123.123.123.123/32 123.123.123.234/32)`
```ruby
# Disable all components except PostgreSQL
roles ['postgres_role']
repmgr['enable'] = false
consul['enable'] = false
prometheus['enable'] = false
alertmanager['enable'] = false
pgbouncer_exporter['enable'] = false
redis_exporter['enable'] = false
gitlab_exporter['enable'] = false
postgresql['listen_address'] = '0.0.0.0'
postgresql['port'] = 5432
# Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value
postgresql['sql_user_password'] = 'POSTGRESQL_PASSWORD_HASH'
# Replace XXX.XXX.XXX.XXX/YY with Network Address
# ????
postgresql['trust_auth_cidr_addresses'] = %w(APPLICATION_SERVER_IP_BLOCKS)
# Disable automatic database migrations
gitlab_rails['auto_migrate'] = false
```
NOTE: **Note:** The role `postgres_role` was introduced with GitLab 10.3
1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
1. Note the PostgreSQL node's IP address or hostname, port, and
plain text password. These will be necessary when configuring the GitLab
application servers later.
1. [Enable monitoring](replication_and_failover.md#enable-monitoring)
Advanced configuration options are supported and can be added if
needed.
......@@ -25,7 +25,7 @@ requirements, this is the ideal solution.
NOTE: **Note:**
You can also optionally configure GitLab to use an
[external PostgreSQL service](../external_database.md) or an
[external PostgreSQL service](../postgresql/external.md) or an
[external object storage service](../high_availability/object_storage.md) for
added performance and reliability at a reduced complexity cost.
......
......@@ -126,8 +126,8 @@ As long as at least one of each component is online and capable of handling the
By adding automatic failover for database systems, you can enable higher uptime
with additional database nodes. This extends the default database with
cluster management and failover policies.
[PgBouncer](../../development/architecture.md#pgbouncer) in conjunction with
[Repmgr](../high_availability/database.md) is recommended.
[PgBouncer in conjunction with Repmgr](../postgresql/replication_and_failover.md)
is recommended.
### Instance level replication with GitLab Geo **(PREMIUM ONLY)**
......@@ -163,7 +163,7 @@ column.
| [Consul](../../development/architecture.md#consul) ([3](#footnotes)) | Service discovery and health checks/failover | [Consul configuration](../high_availability/consul.md) **(PREMIUM ONLY)** | Yes |
| [PostgreSQL](../../development/architecture.md#postgresql) | Database | [PostgreSQL configuration](https://docs.gitlab.com/omnibus/settings/database.html) | Yes |
| [PgBouncer](../../development/architecture.md#pgbouncer) | Database connection pooler | [PgBouncer configuration](../high_availability/pgbouncer.md#running-pgbouncer-as-part-of-a-non-ha-gitlab-installation) **(PREMIUM ONLY)** | Yes |
| Repmgr | PostgreSQL cluster management and failover | [PostgreSQL and Repmgr configuration](../high_availability/database.md) | Yes |
| Repmgr | PostgreSQL cluster management and failover | [PostgreSQL and Repmgr configuration](../postgresql/replication_and_failover.md) | Yes |
| [Redis](../../development/architecture.md#redis) ([3](#footnotes)) | Key/value store for fast data lookup and caching | [Redis configuration](../high_availability/redis.md) | Yes |
| Redis Sentinel | Redis | [Redis Sentinel configuration](../high_availability/redis.md) | Yes |
| [Gitaly](../../development/architecture.md#gitaly) ([2](#footnotes)) ([7](#footnotes)) | Provides access to Git repositories | [Gitaly configuration](../gitaly/index.md#run-gitaly-on-its-own-server) | Yes |
......
......@@ -31,7 +31,7 @@ This section is for links to information elsewhere in the GitLab documentation.
- Destructively reseeding the GitLab database.
- Guidance around updating packaged PostgreSQL, including how to stop it happening automatically.
- [More about external PostgreSQL](../external_database.md)
- [More about external PostgreSQL](../postgresql/external.md)
- [Running Geo with external PostgreSQL](../geo/replication/external_database.md)
......@@ -45,8 +45,8 @@ This section is for links to information elsewhere in the GitLab documentation.
- Managing Omnibus PostgreSQL versions [from the development docs](https://docs.gitlab.com/omnibus/development/managing-postgresql-versions.html)
- [PostgreSQL scaling](../high_availability/database.md)
- including [troubleshooting](../high_availability/database.md#troubleshooting) `gitlab-ctl repmgr-check-master` and PgBouncer errors
- [PostgreSQL scaling](../postgresql/replication_and_failover.md)
- including [troubleshooting](../postgresql/replication_and_failover.md#troubleshooting) `gitlab-ctl repmgr-check-master` and PgBouncer errors
- [Developer database documentation](../../development/README.md#database-guides) - some of which is absolutely not for production use. Including:
- understanding EXPLAIN plans
......
......@@ -116,7 +116,7 @@ database has reached the target time.
On GitLab.com, Consul and Patroni work together to coordinate failovers with
the read replicas. [Omnibus ships with repmgr instead of
Consul](../administration/high_availability/database.md).
Patroni](../administration/postgresql/replication_and_failover.md).
#### Load-balancing
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment